feat(entity,storage): rework speculation tree model and store#231
Open
behinddwalls wants to merge 2 commits into
Open
feat(entity,storage): rework speculation tree model and store#231behinddwalls wants to merge 2 commits into
behinddwalls wants to merge 2 commits into
Conversation
This was referenced Jun 9, 2026
927e240 to
6bc8426
Compare
380f0b1 to
3eddc3d
Compare
6bc8426 to
7b49586
Compare
67da1e9 to
0ee54c9
Compare
3053dd4 to
58cabeb
Compare
behinddwalls
added a commit
that referenced
this pull request
Jul 7, 2026
## Summary Capture how SubmitQueue speculates before the interfaces land. Speculation is modeled as two layers: **decision seams** — enumeration and **scoring** *describe* the tree of possible bets, selection and prioritization *act* on it — and **signal-driven limit policies** that decide *how much* to allow at each step (dependency limit, selection limit, prioritization limit), scaling with the build system's resources. A path's score is dynamic — recomputed as dependencies land, fail, or have their builds pass — not frozen at enumeration. Covers: the problem (dependent batches serialize behind multi-minute builds), the Base/Head path-and-tree model, where speculation sits in the orchestrator pipeline, the status-vs-action ownership split, the dependency limit as an eligibility gate (batches wait for predecessors to land rather than trimming their base), dynamic rescoring, and how prioritization rations the queue's builds against its budget. Design only; entities, store, and extension interfaces follow in later PRs in this stack. Adds doc/rfc/submitqueue/speculation.md and links it from the RFC index. ### Changes since review Reframed from two seams (enumeration + selection) to describe/act decisions + limits, addressing @sbalabanov's review: - selector resource-awareness → the new **prioritization** layer owns the queue-wide build budget; the selector stays a pure per-batch policy - "variable depth, not a single knob" → the **dependency limit** is a signal-driven policy, not a constant - clarified the dependency limit is an **eligibility gate** on active in-flight dependencies (a batch waits until predecessors land), not a base trim - **dynamic scoring**: split scoring out of enumeration into a **scorer** that recomputes each path's score as dependencies land/fail/pass — enumeration is now structure-only - entire tree persisted (the selector reads all path statuses + scores, so it can cancel a path it earlier asked to build); the redundant fallback path is cancelled on a winning bet; slash-free example names; trimmed impl-detail bullets - recorded as open: extending rescoring to weigh how long a batch has waited ## Test Plan Doc-only. Rendered locally; ASCII diagrams and tables verified; no hard-wrapped prose. ## Issues ## Stack 1. @ #230 1. #231 1. #232
Reshape the speculation tree data model around the Base/Head path that the build stage actually consumes, and align its store with the speculation design.
entity: SpeculationPath{Base, Head} becomes the unit; SpeculationPathInfo carries the path plus its Score, controller-owned Status (candidate/selected/building/passed/failed/cancelled), and BuildID. Score is scorer-computed and controller-persisted dynamic state — recomputed on every respeculate, not set at enumeration. Adds SpeculationPathAction and SpeculationPathDecision for the selector seam. SpeculationTree.Speculations becomes Paths. The Build entity uses the shared SpeculationPath (Head = the batch under verification) and drops its own Score, which was never populated or read; the prioritizer seam re-adds a build priority when it lands.
storage: SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo) becomes Update(ctx, SpeculationTree) — symmetric with Create, keyed by tree.BatchID. The MySQL column speculations is renamed paths to match the entity field. MySQL impl, mock, and storage integration coverage (create/get round-trip, duplicate, whole-tree overwrite, not-found) added/updated.
7b49586 to
4606cc7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Why?
The speculation design (
doc/rfc/submitqueue/speculation.md, merged in #230) makes a path's score a dynamic, scorer-owned prediction — recomputed on every respeculate as dependencies land, dependency builds pass, and siblings fail — not a fixed number stamped at enumeration. The prior store model drifted from that in three ways: it documentedScoreas the enumerator's, it carried aScoreon theBuildentity that was never populated or read, and it left the MySQL column name out of step with the renamed entity field. This aligns the entity + store with the design.What?
entity:
SpeculationPath{Base, Head}is the unit;SpeculationPathInfocarries the path plus itsScore, controller-ownedStatus(candidate/selected/building/passed/failed/cancelled), andBuildID.Scoreis now documented as scorer-computed and controller-persisted dynamic state — recomputed on every respeculate, not set at enumeration. AddsSpeculationPathActionandSpeculationPathDecisionfor the selector seam.SpeculationTree.SpeculationsbecomesPaths. TheBuildentity uses the sharedSpeculationPath(Head = the batch under verification) and drops its ownScore, which was never populated or read; the prioritizer seam re-adds a build priority when it lands.storage:
SpeculationTreeStore.UpdateSpeculations(batchID, []SpeculationInfo)becomesUpdate(ctx, SpeculationTree)— symmetric withCreate, keyed bytree.BatchID. The MySQL columnspeculationsis renamedpathsto match the entity field. MySQL impl, mock, and storage integration coverage added/updated.Test Plan
make test— 65/65 passErrAlreadyExists, whole-tree overwrite viaUpdate, andGet/Updateon a missing batch →ErrNotFoundmake gazelle/make fmt— no drift