Capstone: the source-grounded research assistant

The capstone guide

One page for the whole capstone: what each piece is, how the pieces fit, and what to do after every session. Keep it open next to your repository.

The one-liner and the goal

The pieces, and how they fit

There are seven pieces. Only two of them judge your agent: the grader and the defence.

Read this twice. As shipped, uv run bootcamp check cap01 prints cap01: 4/5 passed. The first four checks pass before you write anything, because they run the course's reference pipeline on a scripted fake model. They never touch your agent. Only cap01-e5, your ranked issue list, is yours. So a green notebook says the contract can be met. Your agent is judged by the grader and the defence.

What counts, and for what

Three different things, measured three different ways. Keep them apart.

The notebook scorecards are formative: they find the gap you would rather not meet on stage. The practice grader is formative too. Only the defence grades you, and only the private set certifies you.

The architecture

Read it top to bottom. The top row is the course's reference pipeline, and your agent starts as a copy of it. The middle row is the four ways a run may end without an answer; the dashed one is yours to build. The bottom row is what a reviewer reads, and the check that reads each piece.

How it works, step by step

bootcamp check chNN runs in your course folder. pytest and the grader run in your repository. The contract tests pass as shipped, because your agent starts as the course's pipeline. Their job is to stay green while you change it.

The full list of what each session gives you is on the road to the capstone.

Start today, in 15 minutes

From nothing to a first green in your own repository.

1. Update your course folder. In your course folder, run git pull, then uv sync.

2. Create your repository. Put it next to the course folder, never inside it:

   uv run bootcamp capstone new ../my-capstone
   cd ../my-capstone
   

3. Put it on GitHub, as a public repository.

   With the GitHub CLI (gh auth login once, as in handing work in):

   gh repo create my-capstone --public --source . --push
   

   Without gh, in your browser:

   1. Open https://github.com/new.
   2. Name it my-capstone. Choose Public. Leave "Add a README file" unticked, because your repository already has one.
   3. Click Create repository. Then, in your terminal:

   git remote add origin https://github.com/<you>/my-capstone.git
   git push -u origin main
   

   Do not drag the folder into GitHub's "upload files" page. That route ignores .gitignore, so it can upload your .env.

4. Run the tests. uv run pytest prints 4 passed, 2 skipped, 3 xfailed. That is your first green. The 3 xfailed are behaviours the starter agent does not have yet (an injected instruction, a provider error, a timeout), each marked with the session that teaches the fix. When you add one, its test fails on purpose until you remove the mark: a pass you earned. The 2 skipped are yours to write in sessions 11 and 14.

5. Run the practice grader, and read which gates fail. uv run bootcamp capstone grade. With no .env, your agent runs on the fake model and prints score: 3/10 (30%), NOT YET, and critical safety gate failed. The 3 refusal cases pass. Every other case fails on citation_recall, claim_support, no_review_flag, because the fake model cannot read the passages. That is your starting line.

6. Write your first entry in docs/ISSUES.md. One row: rank, issue, impact. The impact is what ranks it. Use the same shape as cap01-e5.

7. Commit and push. git add -A, git commit -m "first issue", git push. Open the Actions tab on GitHub and watch CI run the tests and the grader.

8. Later, submit. When the final opens, uv run bootcamp capstone submit hands in your answers. See submit the final assignment.

With a model in your .env, your score is different, and it can change between two runs of the same code. Always write down which model produced a number. Only the fake model gives the same number on every machine.

Submit the final assignment

Run it inside your capstone repository, once the final opens. The address of the course app is in the course announcement.

export DEV3PACK_API_BASE=<the address from the announcement>
uv run bootcamp capstone submit --github <you> --dry-run
uv run bootcamp capstone submit --github <you>

1. It checks your repository first. Everything must be committed and pushed. Your submission links to one commit on GitHub, so that commit must be the code that answers.
2. It simulates locally. It runs the practice set, prints your practice score and the model it ran on, and changes nothing. On the fake model it warns you: the fake only refuses, so your score would be about 30%.
3. It answers the final set. Your agent answers each question. A question that crashes or times out becomes a flagged refusal, and the run goes on.
4. It opens the pull request to the submissions repository, the same way bootcamp submit does. Without gh, it prints the browser steps. --dry-run prints the bundle and opens nothing.

The pull request merges itself once its check passes. Your score arrives a few minutes later, in finals/<you>/result.json in the submissions repository. Submit as often as you like. The leaderboard shows your best score. The certificate needs more than 30% and every critical question passed.

Reading the grader

Each question passes only when every gate on it passes. The last column of the grader's output lists the gates that failed.

Gates on a question your agent must answer

Gates on a question your agent must refuse

Gates on every question

The verdict at the bottom

What a good showcase repository looks like

A reviewer or an employer opens README.md first, and reads it top to bottom. Put these sections in this order.

Also good: a "Sources" line from session 13, the rollback sentence from session 14, and a "Credits" section naming any repository you learned from.

What never goes in it

The defence

Six minutes, hard-timed. Every minute has a file in your repository behind it.

The failure you draw is one you already met: empty retrieval, a citation that supports nothing, a tool that times out mid-loop, an order hidden in a document, a call that repeats until the budget ends it, or an illegal move in the state machine. Failing safely is a pass: a refusal the caller can read, a trace that shows where it stopped, and one accurate sentence. Saying it worked when the trace says otherwise is the one answer that fails.

The room scores six areas, on evidence:

What each area looks for is on what done means.

Rules

The capstone contract

The full brief is on the capstone page.

No solutions are published for the capstone or the final assignment. The capstone is yours to build, and a published answer would make every showcase the same.

The private set stays private. Session 7 showed why: a system tuned on the cases it is graded on measures its own homework. The practice set is public so you can build. The graded set is one you cannot tune to, so its score means something to the person reading your certificate.

Previous: What done means · Next: The road to the capstone