Prevent abort in situations with recursive checkpoints

In situations where there were multiple checkpoints queued and the
first one causes the thread to suspend itself again then the
RunCheckpointFunction function will hit a LOG(FATAL) and abort. This
is because the recursive checkpoint will clear out the checkpoint
backlog and the first RunCheckpointFunction invocation will
unexpectedly find itself without any more checkpoints to run and
abort.

To fix this, and simplify the code at the same time, we have changed
the RunCheckpointFunction method to (as its name suggests) only run a
single checkpoint function. It will pop this function off the stack of
pending checkpoints and run it relying on the caller to ensure that
all pending checkpoints are handled. This is fine since, due to the
multithreaded nature of checkpoints, the caller must call
RunCheckpointFunction in a loop anyway to ensure it does not advance
until all checkpoints have been handled.

We add test 203-multi-checkpoints that tests that the checkpoint
system does not fall over if there are multiple checkpoints some of
which can suspend.

Bug: 67838964
Test: ./test.py --host -j50
Change-Id: Ib6a3e083e6069d4839647d194bee6849d973633e

0 files changed, 0 insertions, 0 deletions