Fix a problem and keep the check

A problem you can repeat exactly is a problem you can fix and check again. This guide follows one problem from replay to a kept check that your CI runs.

Replay the problem

1. Read the problem in the output of geo check: the consequence first, the rule with what was expected and what was observed, and the numbered steps. It also says what the broken runs had in common.
2. Replay it with the finding’s identifier. Geometer repeats the run from a clean start, with the model’s recorded answers. If you have already changed your code, the run can no longer be repeated on it: the replay says it cannot replay on this version, and geo verify is how you compare your change with the original run.

geo replay <finding>
geo replay <finding> --clean-worker

--clean-worker replays the portable recipe on a second clean worker, with a fresh records folder and fresh containers from a copy of the project at the pinned revision.

Some problems cannot be repeated live, for example when the task they came from is no longer in the project; the command says why in words. A smaller run can sometimes be found with geo shrink <finding>, which tries to minimize the finding and replays the result before accepting it. A flaky result is reported, not accepted.

Change your code

Change your agent in your own editor. The order is: read what happened, change your code, check your fix, keep the check. A typical fix for a lost reply is to look up the first attempt before trying again, or to send an idempotency key so a repeat is ignored.

Edit the code where it is. You do not need to keep a copy of the old version or run anything first: when a check finds a problem, Geometer saves a recording of the run that broke the rule, while your code is still unchanged. geo verify compares your edited code with that recording.

Check your fix

Run geo verify. Geometer replays the recorded run on your code as it is now, confirms the run reached the same point, and tries a bounded set of nearby cases. The verdict is one of reproduced (still happens), no_longer_reproduced (fixed within what was checked), inconclusive (could not tell) or incompatible (cannot check on this version), followed by the number of runs, how many exercised the risky path and how many broke the rule. See Checking a fix.

geo verify <finding>
geo verify <finding> --path /path/to/changed/checkout

If your code changed before Geometer saved a recording of the problem (for example, a problem found by an older version), there is nothing to compare with. geo verify then says so in a sentence, and does not say anything about your fix either way. Run the check again to see whether the problem still happens.

It exits with 0 when the problem no longer reproduces, 1 when it still does, and 2 when the result is inconclusive or the version is incompatible.

Keep the check

Run geo regression keep once the fix has passed geo verify, because what you keep is the fix. Geometer stores the check in its records and writes a reviewable file under .geometer/regressions/ in your project. Keeping the same problem twice does no harm.

geo regression keep <finding>
geo regression list

The command shows what it will keep (the problem, the rule it broke, the folder the check is written to) and asks yes or no. You answer in a terminal window. It refuses to run when it is not attached to one, and there is no option that skips the question: keeping a check is your decision, so a program or an assistant cannot make it for you.

Commit the new file together with your fix. It is a plain file you can read in review. It is the only change to your project that Geometer makes for you.

If your fix refuses a duplicate request with an error instead of ignoring it, a kept check can explicitly allow that outcome:

geo regression keep <finding> --allow retry=rejection

The example allows the node retry an outcome of rejection that it did not show when the problem was recorded. Allowing an outcome is your explicit decision, and the question names the outcome you are allowing before you answer.

Run the kept checks

geo regression run

This re-executes your kept checks on the current checkout. It exits with 1 when any is reproduced, and 2 when any is inconclusive or incompatible. geo ci runs the same kept checks after its automatic plan. See Run kept checks in CI.

After a kept check

Keeping a check changes your project’s files, so the project’s version changes. Geometer treats the version without kept checks as the same version for a certificate, so keeping a check does not flip an issued certificate to “changed”.

• Run kept checks in CI (Make the kept check run on every change.)
• Issue a certificate (Report what you checked.)