Repository navigation
test kit and CI: a command has the time its case has left; the pr tier's gate judges what it runs; the unit shards follow the longest shard - #998
Merged
Conversation
…where no scope states one Design 2026-11-20 §5.8 (K1) gives a command of the kit the time its case has left. mtest 0.11.1 holds a command to that time, but the kit still passed ten minutes as each command's own, and the smaller of the two held: `sets/985`'s pack of every system ended at ten minutes in a pr-tier shard of two (run 44937152470), its suite within the 45 minutes it states. A command that states no time now takes the case's remaining time, and ten minutes where neither the case nor a scope states one. The two commands raised to 30 minutes for the same reason (images 1033, trees 955) no longer state a time; `images` states its suite's time from measurement (30 minutes). Cases 1141 and 1142 assert the time the kit gives a command. The design record states the finding (17).
…is the full tier's (mtest 0.11.3) The surface ledger's requirements are met by passing cases of the run's group, and the pr tier runs the jobs and cases a change affects: 14 items only the full tier's jobs use had no passing case, so the gate failed on every pr run whatever the change (run 44937152470). mtest 0.11.3 judges a ledger for a set that runs every case and notes it for another; ci.yml passes the run's set to the gate. The contributing skill states that a pull request adding a surface item confirms its case with ci:full; the design record states the finding (17).
…st of them, not from the programs' durations over four workers
…jobs kept, since the imported records state no shard
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Design 2026-11-20 §5.8 (K1) gives a command of the test kit the time its case has left. mtest 0.11.1 holds a command to that time, but the kit still passed ten minutes as each command's own, and the smaller of the two held. In the pr-tier run of #997 (44937152470, two Linux shards)
sets/985packed every system besideimagesandstackon a four-processor runner; the step ended at ten minutes while its suite was well within the 45 minutes it states (setstook 2140 s there, 1162 s in a run of three shards).mcpplibs::mtest::time_left()); where neither the case nor a scope around it states one, ten minutes bound it, since the runner's limit would end the program without the case's report.imagesstates its suite's time from measurement (30 minutes; 1102 s in a shard of two, 707 s in a shard of three, of the run's 1200 s).e2e/test, protocol) assert the time the kit gives a command; with the previous kit 1141 fails.The engine and the released 2026.11.21.1 are unchanged: the pull request changes the test kit, its cases, CI and their records.
The pr tier's gate
The surface ledger's requirements are met by passing cases of the run's group, and the
prtier runs the jobs and cases a change affects. In run 44937152470 (#997, a change ofrelease.ymlalone) the gate listed 14 items only thefulltier's jobs use (coff-defon Windows,dyndep --split-moduleandlink-output --asidecalled from inside the build graph, reasons reached on some rows), so the gate failed on everyprrun whatever the change..github/versions.envpins it (the bundle's sha256 is the release's).ci.ymlpasses the run's set to the gate (gate --set "$SET"):fulljudges the budget and the surface,prthe budget.ci:full.The unit shards' count
Main's full run of #997 (44945453323) took two Linux unit shards, 2544 and 2432 s, and
treesran past the 40 minutes it states (1369 s in a run of three shards);sets,imagesandstackran in one shard together. The count was the programs' durations of main's last run summed, over what four workers do in 30 minutes: the run of three shards (1236, 834 and 1438 s) gave two. That measure assumes four workers always busy and a program's duration independent of what runs beside it; the openkal programs build in parallel within the machine's one budget, so in fewer shards each takes longer, and a shard's time is that of its longest programs and of those beside them.ci-unit.ymltakes the shards of main's last run, N, and the longest of them, W: N × W of shard time at 30 minutes a shard (both runs give three), at least one, at most eight; four with no history of shards. N and W are read from the stores the shard jobs kept (mtest-unit-<name>-<shard>-a<attempt>): a job's store is imported as mtest's run records, which state no shard, and the history also holds the subsystems job's runs.