The animated scenes show on a laptop or desktop screen.

CI/CD

Claude Code in CI/CD

PhysicsWallah's platform team runs Claude Code on every pull request to the service that scores the PW app's mock tests. This chapter follows pull request #4127, which makes a skipped question score -1 instead of 0, through a review in the session that wrote the code, a review job that waits six hours for a reply, a review that arrives as one long comment and a review that repeats itself on every push. It ends with the tests that catch the bug before Sunday's mock.

What this chapter shows you

  1. A separate Claude session reviews each pull request. The session that wrote the change approved it, and a new session found the bug on line 88.
  2. The -p flag runs the review with nobody at the keyboard. Without it the review job waited six hours for a reply and was cancelled.
  3. Findings arrive as inline comments, once each. A schema puts each finding under its line, and with the last run's findings in the prompt, each finding is posted once.
  4. With CLAUDE.md and the existing tests in context, Claude writes only the missing tests. Claude proposes four tests in place of fourteen, and two of them fail on line 88.
  5. Check yourself. Choose what PR #4127's review job needs, then answer one question in the exam's style.

Part 1: A separate Claude session reviews each pull request

PhysicsWallah prepares students for JEE, NEET and UPSC through its app, its website and its offline coaching centres. Students sit mock tests in the PW app, and a platform team of about 12 engineers owns test-engine, the service that scores them. On a Sunday all-India mock test, several lakh students submit their attempts within a few hours. For each attempt, the score_attempt() calls score_question() once for every one of the 75 questions. JEE Main marks a multiple-choice question +4 when it is correct, -1 when it is incorrect and 0 when it is left unattempted.

About 40 a day reach test-engine and the app's other . A pull request is an engineer's proposed change to the code, shown as a of the lines added and removed, and it is checked and reviewed before it is merged into the main code.

CI runs the checks it is given on every pull request.

is a set of checks that run automatically on every pull request before it can join the main code, such as building the app, running the tests and running the , a tool that checks the code's layout and simple mistakes without running the code. Continuous deployment (CD) then ships the changes that pass. This chapter is about the CI half, which is where reviews code and writes tests.

Claude Code is the program that runs on the engineer's laptop or on a CI runner. It sends each request to Claude, the model, and carries out the file edits and commands that Claude asks for. One conversation with Claude Code, from its first message to its last, is a .

On Thursday at 10:12, an engineer opens PR #4127. Working with Claude Code on a laptop, the engineer has score_question() in scoring.py and the mock-test timer in timer.py. When a student skips a question, the app passes no answer, which Python writes as . The refactor removed the two lines that returned 0 when the answer was None, so line 88 now returns 4 for a correct answer and -1 for everything else, including a skipped question.

On GitHub, a in the folder .github/workflows/ lists the that run on every pull request. , GitHub's service that runs these jobs, starts each one on a fresh , and the pull request page shows each job as a check with a tick or a cross. The tests job runs all 212 tests in the repository. The nine in tests/test_scoring.py check scoring, and they cover correct and incorrect answers but not a skipped one.

CI pipeline

PR #4127 enters the CI pipeline, and the build job builds test-engine from its code and passes.

The tests job runs the existing tests. They check a correct answer and an incorrect answer, and no test checks a skipped question, so nothing catches that a skipped question now scores -1. The tests job passes.

The lint job checks the code's layout and passes. The team has no review job yet, so no job reads the change to score_question(), and GitHub marks the pull request ready to merge.

Every job passed PR #4127, and the bug on line 88 got through because no job was written to look for it.

Each job passed because each one checked only what it was written to check, and no job read the change to score_question() itself. On GitHub the three jobs run at the same time, each on its own runner.

CI checks on PR #4127

In Files changed, two removed lines show in red: the check that returned 0 for a skipped answer. Line 88 now returns 4 for a correct answer and -1 for everything else.

The checks finish: build in 1m 02s, tests in 3m 40s with 212 passed, and lint in 18s. All checks have passed, and the Merge button turns green.

The tests job ran tests/test_scoring.py. Its nine tests check correct and incorrect answers, totals, ranks and negative marks per subject, and none of them checks a skipped question.

If #4127 merges, a student who skipped 18 of 75 questions in Sunday's Mock Test 07 scores 165 instead of 183, and the phone shows All India Rank 9,870 in place of 6,412.

The tests job reported 212 passed, and none of the 212 checks a skipped question. The team decides to add a Claude Code review job that reads the change itself.

A review in the writer's session repeats the writer's reasoning.

A session is one Claude Code conversation. Its holds everything said and done in that conversation: the requests, Claude's reasoning, the files it read and the edits it made. Session context isolation means running a review in a new session, which has none of the history of the session that wrote the code.

The engineer's first idea is the obvious one: ask the writer's session, which is the Claude Code session that made the refactor, to review its own change. Its session ID is 7f3c9a. While refactoring, that session wrote that skipped questions are filtered out before score_question(), so the check for a skipped answer was . The belief is false, because score_attempt() scores all 75 questions, skipped ones included.

When the same session reviews its own change, that reasoning is still in its context, and Claude judges the code with it, even when the marking rules are in the same context.

Session context isolation

In the writer's session, Claude rewrites score_question() and removes the check for a skipped answer. While it works, it writes down its reason: skipped questions are filtered out earlier, so the check is dead code. That reason is now in the session's context.

The engineer asks the same session to check its work against the marking rules. Claude reads line 88 with its own reason still in the context and answers that the code looks correct.

A new session gets the diff and the marking rules, and the writer's reasoning does not reach it. Claude in the new session flags line 88: a skipped question now scores -1 where it should score 0.

If the review continues or resumes the writer's session instead, the session's whole history comes with it, including the reason for deleting the check, and Claude again answers that the code looks correct.

A review in the writer's session judges the code with the writer's reasoning. A review in a new session judges the diff against the rules without it.

Both sessions read the same change with the same marking rules. The writer's session read it with its own reason for deleting the check still in its context, and the new session read it with none of the writer's history.

Self-review in the writer's session

In the engineer's terminal, the request to refactor score_question() goes into the session's context.

Claude's reasoning appears as it works: skipped questions are filtered out before this function, so the None check is dead code. That sentence goes into the context too.

Claude edits scoring.py, removing two lines, and the edit joins the context.

The engineer asks the same session to review its changes against the marking rules: +4, -1 and 0 for unattempted. Claude answers that the refactor is correct and that skipped questions never reach score_question(), which repeats its earlier reasoning.

The review ran in the same context as the refactor, so Claude's answer repeated the reasoning that had removed the check, even though the marking rules were in that context too.

A new session reviews the code as written.

A new session has none of the writer's history, so Claude judges the code as written, with the diff and the marking rules it is given. Anthropic's best-practice guide for Claude Code says that a fresh context improves code review because Claude will not be biased toward code it just wrote.

The engineer can start the review in four ways. The first three reopen or copy the writer's session, so the review starts with the writer's history. The fourth starts a new session with no link to the writer's session.

claude --continue

Reopens the most recent session in this folder. On the engineer's laptop that is the writer's session, so the review continues inside it, with the writer's reasoning in its context.

claude --resume 7f3c9a

Reopens one chosen session by its ID. Here the ID is 7f3c9a, the writer's session, so the review again continues inside it.

claude --resume 7f3c9a --fork-session

Copies session 7f3c9a into a new session with its own ID and continues in the copy. The writer's session stays as it was, and the copy starts with all of its history, the writer's reasoning included.

gh pr diff 4127 | claude -p "Review this diff.
Marking: +4 correct, -1 incorrect, 0 unattempted."

is GitHub's command-line tool, and gh pr diff 4127 prints the diff of PR #4127. The | sends that diff into Claude Code, and Part 2 explains -p. This starts a new session that has none of the writer's history. It receives the pull request's diff and the prompt, along with the project's CLAUDE.md, a file of project instructions that Part 4 covers.

An independent review session

A second terminal starts a new session with claude, and its context holds nothing yet.

The engineer gives the new session the diff of PR #4127 and the marking rules, +4, -1 and 0 for unattempted, and asks for a review of the diff against those rules.

The new session flags line 88: score_question() no longer handles a skipped answer, so None now scores -1, where JEE Main gives 0.

claude --continue reopens the latest session, claude --resume 7f3c9a reopens the writer's session by its ID, and --fork-session copies it under a new ID. A review started in any of the three has the writer's reasoning in its context.

The independent review becomes a job in PhysicsWallah's pipeline, run on every pull request.

Both reviews had the same marking rules. The writer's session also had its own reasoning in context, and its answer repeated that reasoning, while the new session had none of the writer's history and found line 88. Telling the writer's session to be more critical, or turning on , leaves that reasoning in the context, so the exam's answer is an independent review instance with no link to the writer's session.

Claude Code saves each session on the machine where it ran, which is how --continue and --resume can reopen it. A CI runner starts empty and is removed after its job, so it has no saved sessions, and a review job started there begins as a new session. The team keeps it that way by giving the job the diff and the marking rules and nothing from the writer's session.

Anthropic's best-practice guide describes the same arrangement for code in general: one session writes, a second session with a fresh context reviews, and its feedback goes back to the first. PhysicsWallah puts the review into its pipeline as a separate job, and Part 2 follows that job onto a GitHub runner.

Part 2: The -p flag runs the review with nobody at the keyboard

The review job runs claude "Review this pull request for scoring bugs." on a GitHub runner, and Claude Code waits for someone to answer. Who can type that reply?

Interactive Claude Code waits for a person to type.

Typing claude in a opens Claude Code's . It shows an input box, works on what you type, asks before some actions and then waits for your next message.

The -p, short for --print, runs Claude Code in print mode, also called headless mode, which needs nobody at the keyboard. Claude Code takes the prompt and any text handed to it, does the work, prints the result and exits.

A workflow step is written as command lines. A command line names a program and its options, such as claude -p "Review this pull request.", and the is the program that runs it. When two commands sit on one line with the pipe character | between them, the shell takes the text the first command prints and hands it to the second command. In gh pr diff 4127 | claude -p "Review this pull request.", gh pr diff 4127 prints the diff of PR #4127, and the pipe hands that diff to Claude Code along with the prompt. The text a command receives this way is its , and the text it prints is its . When Claude Code exits, it returns an : 0 when the run succeeded and another number when it failed.

The team adds the independent review from Part 1 as a job in a , claude-review.yml. The first version of the job's step runs claude "Review this pull request for scoring bugs." on a GitHub , without -p. A runner is a fresh machine that GitHub starts for one job and removes afterwards, and nobody sits at it.

Claude Code starts in interactive mode and waits for input that no one will send, so the step never finishes. cancels a job that runs longer than its , which is 360 minutes unless the workflow sets another value. For six hours the runner is busy and the review never arrives.

Non-interactive mode

GitHub Actions starts the review job on a runner, and the job runs claude with the prompt and no -p. Claude Code opens in interactive mode and waits for someone to type a reply, and nobody sits at a runner.

Claude Code keeps waiting for input, and the job keeps its runner busy. Nobody can answer on a runner, so PR #4127's review cannot finish.

After 360 minutes, GitHub Actions cancels the job for exceeding its timeout, and PR #4127 can still be merged with no review.

The team changes the step so that Claude Code gets the diff along with the prompt and needs nobody to type: gh pr diff 4127 | claude -p "Review this pull request for scoring bugs.". gh pr diff prints the diff, the pipe | hands it to Claude Code, and -p turns on print mode.

Claude reads the prompt and the diff and writes the review. Claude Code prints it to standard output (stdout), exits with code 0, and the job finishes in 2m 14s.

Without -p, Claude Code waits for a person until the job times out. With -p, it takes the prompt and the diff, prints the review and exits.

Interactive mode needs a person to answer, and print mode needs only the prompt and its input. On a GitHub runner, nobody is there to answer.

A CI job waiting for input

The workflow's step runs claude "Review this pull request for scoring bugs." with no -p. Checkout and the install finish, and Claude Code starts in interactive mode and waits on the runner for someone to answer it.

Nobody sits at the runner, so Claude Code keeps waiting for input while the timer passes 5 minutes, 1 hour and 3 hours.

At 6 hours GitHub Actions cancels the job for exceeding 360 minutes. The "Run claude" step printed nothing.

On the pull request, build, tests and lint are green and claude-review shows cancelled. GitHub blocks merging only until the checks a team marks as required have passed, and the review is not one of them, so the Merge button is still green.

The job reported no review and no failure, only a cancellation after six hours, and the pull request could still be merged.

A team that wants merging to wait for the review marks claude-review as a required check in the repository's settings. GitHub then keeps the Merge button blocked until the review passes, so a cancelled or failed review stops the merge.

The -p flag prints the result and exits.

With -p, the review step becomes gh pr diff 4127 | claude -p "Review this pull request ...". prints the of PR #4127, and the pipe hands it to Claude Code, so Claude receives the prompt and the diff together. Claude Code also reads the repository's , a file of project instructions that Part 4 covers. When the run ends, Claude Code exits with code 0 on success and a non-zero code on failure, and the next step of the job reads both the output and the code.

Nobody can answer a on a runner, so the workflow approves in advance, with --allowedTools, the tools that Claude may use without asking. It can also set safety limits for a run that nobody watches: --max-turns stops the run with an error after a set number of , and --max-budget-usd stops it once its estimated spend reaches a set amount. These limits are a safety stop, and a normal review ends when Claude finishes it.

The official exam guide has a sample question about a CI job that hangs in exactly this way. Its right answer is -p, and it offers three other fixes, none of which works. There is no CLAUDE_HEADLESS and no --batch flag in Claude Code. Feeding Claude Code an empty input with < /dev/null is a shell workaround and not Claude Code's documented way to run without a person; the documented way is -p.

This is the review workflow with -p in place.

on:
  pull_request:
    types: [opened, synchronize]

An event is something that happens on GitHub that can start a workflow. This workflow starts on two events: opened, when a pull request is opened, and synchronize, when new commits are pushed to an open pull request.

jobs:
  review:
    runs-on: ubuntu-latest
    timeout-minutes: 15

One job, review, on a fresh Linux runner. If it runs longer than 15 minutes, GitHub cancels it, in place of the default 360.

    permissions:
      contents: read
      pull-requests: write

GitHub gives each job a key, called a , that lets the job act on the repository. These lines let this job's key read the code and write review comments on the pull request, and nothing else. This token is a password for GitHub and has nothing to do with the tokens a model reads.

    env:
      PR: ${{ github.event.pull_request.number }}
      REPO: ${{ github.repository }}
      HEAD_SHA: ${{ github.event.pull_request.head.sha }}
      ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
      GH_TOKEN: ${{ github.token }}

Values that every step of the job can read: the pull request's number, the 's name, the pull request's latest commit, the Anthropic API key kept as a GitHub , and the job's GitHub token, which gh uses. GitHub does not hand secrets to a run started by a pull request from a fork, which is a copy of the repository under another account, so on such a pull request the review has no API key and fails. PhysicsWallah's engineers open their pull requests from branches of test-engine itself, so the key is always there.

    steps:
      - uses: actions/checkout@v4
      - run: |
          curl -fsSL https://claude.ai/install.sh | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

Copy the repository onto the runner, then install Claude Code. The installer puts the claude program in the folder ~/.local/bin, and the echo line adds that folder to the list of folders where the job's later steps look for programs, so the next step can find claude.

      - run: |
          gh pr diff "$PR" |

The | straight after run: belongs to the workflow file and only says that the step's commands take several lines. The | at the end of the gh pr diff line is the shell's pipe, which hands the diff to the command on the next line.

          claude -p "Review this pull request for scoring bugs.
          Marking: +4 correct, -1 incorrect, 0 unattempted." \

-p runs Claude Code in print mode, so it reviews the diff, prints the review and exits. The prompt gives the marking rules, so the review knows that a skipped question scores 0. The \ at the end says that the command goes on to the next line.

            --allowedTools "Read,Grep" \
            --max-turns 10

The run may use Read, which opens files, and Grep, which searches inside them, without asking. A tool that needs approval and is not listed is denied, because nobody is there to approve it. If the review takes more than 10 turns, the run stops with an error, and a normal review finishes well before that.

Print mode with claude -p

The review step now hands Claude Code the pull request's changes directly. gh pr diff 4127 prints the diff, and the pipe | passes it to Claude Code as its input (stdin).

Claude Code puts the project's CLAUDE.md, the diff and the prompt into one API request and sends it to Claude.

Claude writes the review and sends it back to Claude Code on the runner.

Claude Code prints the review to stdout, where the job's log shows it, and exits with code 0. The review step has taken 2m 14s.

No step waited for a person, so the job moves on to its next step, Post comment, which puts the review on the pull request.

Claude Code ran the same review both times. With -p it took the prompt and the diff, printed the review to stdout and exited, so no step waited for a person.

Nobody on a runner can approve a tool, so a CI job lists in --allowedTools every tool it needs that asks for approval. The review job only reads and searches files, which needs no approval. The nightly test job in Part 4 edits files and runs the tests, so its command line lists Edit and Bash(make test). As of October 2026 (Claude Code 2.1.302 or later), Claude Code denies every other such tool call in a claude -p run, and the allow rules in the repository's own settings file do not count on a runner, so the list has to be on the command line.

Part 3: Findings arrive as inline comments, once each

A paragraph of findings can only be posted as one long comment.

is an answer in a fixed shape that a program can read field by field, such as a list of findings in which each finding has a file, a line and a message.

The review job's next step, Post comment, puts the review on the pull request. The engineers want each finding as an , which sits under one line of one file in the , where they are already looking. GitHub accepts an inline comment only on a line that appears in the pull request's diff. To post one, the posting script, a short program the team writes and runs after the review step, needs the file and the line of each finding, and claude -p prints the review as a paragraph.

--output-format json wraps the run's answer in a object with fixed keys, such as type, subtype, result and session_id. The answer itself is still free text, inside result, so the script still has no file and no line to post to. The flag --json-schema gives Claude Code a , a description of the fields every answer must have, and Claude Code puts an answer that has them in a separate field of its JSON output, named structured_output. That field is where the posting script reads the findings.

Structured output

Claude Code prints the review as one paragraph of text. The posting script finds no file or line for any finding, so it can post only the whole paragraph as one comment.

With --output-format json, the output comes inside a JSON object whose fields the script can read. The review is still one paragraph inside result, so the script still has no file or line.

The team adds --json-schema with a schema of six fields: id, file, line, severity, status and message. Claude Code will now check Claude's answer against it.

Claude's first answer gives F1's line as "near 88", and the schema says a line must be a whole number, so the answer does not match. Claude Code tells Claude what is wrong, Claude answers again with 88, and the script posts F1 under line 88 of scoring.py.

F2 in the same answer already has its file and line, so the script posts it under line 40 of timer.py.

In another run, no answer matches the schema after the retries, and the run ends without structured_output, so the script has nothing to post.

With --json-schema, each finding arrives in structured_output with its own file and line, so the script can post each one under its line.

A script that posts comments reads fields and never reads prose. The script can post a finding inline only when the answer's fields match the schema the script was written for. A review that is one paragraph can only be posted as one long comment, and a run that ends without structured_output gives the script nothing to post, so the script checks for that field before it posts.

Text output and the JSON envelope

The review step saves the plain text output in review.txt. The paragraph mentions line 88 only once, at the start of its third sentence.

The post step sends review.txt as one comment on the Conversation tab. In Files changed, line 88 has no comment.

With --output-format json, the step saves an object in review.json with type, subtype, result, session_id and num_turns. The value of result is the same paragraph, inside quotes.

The script takes result and posts it, and a second long comment lands in Conversation. Line 88 still has no comment.

--output-format json gave the posting step an object it could parse, and the findings inside it were still one paragraph, so the only thing the script could post was the whole paragraph.

A JSON Schema gives each finding its own file and line.

Adding --json-schema gives Claude Code a JSON Schema, and Claude's final answer must match it. Claude Code checks the answer against the schema while the run is still going. If the answer does not match, Claude is told what is wrong and tries again. The answer that matches arrives in a separate field, structured_output, from which a script can read each finding's file and line. If no attempt matches, the run ends without structured_output and Claude Code exits with an error. A run can also end without the field when Claude declines the request, so the posting script checks for the field before it posts anything.

PhysicsWallah writes its own schema and its own posting script. Two of the schema's fields, status and resolved, matter when the review runs again.

{
  "type": "object",
  "properties": {
    "findings": {
      "type": "array",
      "items": {
        "type": "object",

The answer is an object holding findings, a list in which every item is itself an object.

        "properties": {
          "id":       { "type": "string" },
          "file":     { "type": "string" },
          "line":     { "type": "integer" },

Each finding has an id such as F1, which stays the same each time the review runs, the file it is in, and the line, which must be a whole number.

          "severity": { "type": "string",
                        "enum": ["bug", "security", "nit"] },
          "status":   { "type": "string",
                        "enum": ["new", "still_open"] },
          "message":  { "type": "string" }
        },

A severity from three allowed words, bug, security or nit, which means a small point of style; a status, new for a problem reported for the first time or still_open for one that an earlier run reported and the code still has; and the message the engineer reads.

        "required": ["id", "file", "line", "severity",
                     "status", "message"]
      }
    },
    "resolved": { "type": "array",
                  "items": { "type": "string" } }
  },
  "required": ["findings", "resolved"]
}

Every finding must have all six fields. resolved is a list of the ids of earlier findings that are now fixed.

gh pr diff "$PR" |
claude -p "$REVIEW_PROMPT" \
  --output-format json \
  --json-schema "$(cat .github/review-schema.json)" \
  > review.json

Pipe the pull request's diff into the review, run it with the schema file and save the JSON output in review.json. $REVIEW_PROMPT holds the review instructions, including the marking rules: +4 correct, -1 incorrect, 0 unattempted. If the run fails, Claude Code exits with a non-zero code and the step stops here, before anything is posted.

jq -e '.structured_output' review.json > /dev/null || exit 1

reads JSON. If structured_output is missing, the job stops with an error and nothing is posted. > /dev/null discards what jq prints, because only its success or failure matters here.

jq -c '.structured_output.findings[]
       | select(.status == "new")' review.json |
while read -r f; do

Take the findings with status new one at a time. On the first run every finding is new.

  gh api "repos/$REPO/pulls/$PR/comments" \
    -f body="$(echo "$f" | jq -r '"[\(.id)] \(.message)"')" \
    -f commit_id="$HEAD_SHA" \
    -f path="$(echo "$f" | jq -r .file)" \
    -F line="$(echo "$f" | jq -r .line)"
done

Post each finding through GitHub's API as an inline comment: its id and message, on its file and line, in the pull request's latest . The id at the start of each comment lets the next run's script find that comment again. $REPO, $PR and $HEAD_SHA come from the job's env. GitHub accepts such a comment only on a line inside the pull request's diff, which is why both findings point to lines shown in PR #4127's diff.

--json-schema findings as inline comments

The team writes a schema, .github/review-schema.json, in which every finding has an id, a file, a line, a severity, a status and a message, and a list resolved holds the ids of fixed findings.

The review runs with the diff piped in, --output-format json and --json-schema, and saves its output in review.json. Its structured_output field holds two new findings: F1 on scoring.py line 88 and F2 on timer.py line 40.

The posting script first checks that structured_output is present, and the check passes.

For each new finding, the script calls GitHub with its file and line. F1 lands as an inline comment under line 88 of scoring.py, and F2 under line 40 of timer.py.

Another run ends without structured_output, for example because Claude declined the request. The check fails, the job turns red, and the script posts nothing.

The script posted each finding where it belonged because each finding arrived with its file and line in fields the schema fixed. A run that ends without structured_output gives the script nothing to post, and the script stops before posting, because it checks for the field first.

The review posts the same comments again after every push.

A sends new commits to a pull request. Re-reviewing after new commits means running the review again on the updated pull request.

The workflow runs on synchronize, the event for a push to an open pull request, so the review runs again after every push to #4127. Each run is a new claude -p on a fresh , and it starts with no record of the earlier runs. It finds the problems that are still in the code, and the script posts them again.

F1 stays in the code until Saturday. A comment's thread is the comment and the replies under it. On Thursday the engineer replied on F1's thread that skipped questions are filtered out before score_question(). The engineer had read the same reasoning in the writer's session in Part 1 and believed it, so line 88 stayed as it was.

Re-review after new commits

The first review run puts two comments on the pull request: the skipped-answer bug on line 88 and the timer bug on line 40.

The engineer pushes a small fix that another reviewer asked for. The second run is a new session with no record of the first, so it posts both comments again, and the engineer now has two copies of each.

The next push fixes line 40 and breaks line 47. The third run posts the line 88 comment a third time and a new comment on line 47, while the two comments on the fixed line 40 stay open, so the pull request now shows problems that are already fixed.

The team starts again from run 1 with stable ids. Run 1 posts F1 on line 88 and F2 on line 40, and the job saves both findings for the next run.

Run 2 gets the saved findings in its prompt. Both problems are still in the code, so Claude reports F1 and F2 as still_open, and the script posts nothing new.

Run 3 lists F2 in resolved, reports F1 as still_open again and reports the line 47 bug as a new finding, F3, which is the only new comment.

Each run is still a new session. With the last run's findings and their ids in its prompt, it posts a comment only for a new problem.

Each review run still starts as a new session. With the earlier findings in its prompt, it can tell which problems already have comments, so only the new problem gets a new comment.

Duplicate comments after new pushes

Push 1 on Thursday at 10:12 opens the pull request. Run 1 posts F1 under line 88 and F2 under line 40, and the engineer replies on F1: skipped questions are filtered out before score_question().

Push 2 at 15:40 fixes a typo. Run 2 starts with no record of run 1 and posts F1 and F2 again, under the same lines.

Push 3 on Friday at 11:05 fixes line 40 and changes line 47. Run 3 posts F1 a third time and a new finding, F3, under line 47. The two F2 comments stay open under a line that is now fixed.

Line 88 now carries three identical comments, and the review has posted six comments in all.

Every run judged the whole pull request with no record of earlier runs, so every problem still in the code was posted again, and fixed problems kept their old comments.

The last run's findings go into the next run's prompt.

The exam guide's fix is to put the previous run's findings into the next run's prompt and tell Claude to report only issues that are new or still unaddressed. Each finding keeps a stable id across runs. A fixed finding's id goes into resolved, a finding still in the code comes back with status still_open and the same id, and anything else comes back as new. The previous findings come from the last run's review.json, which the job saves as a .

The script posts only the findings with status new, leaves the existing comment of each still_open finding as it is, and resolves the thread of each id in resolved.

This is the prompt for run 3, as the job builds it from run 2's review.json.

Earlier findings on this pull request (from the last review):
F1 scoring.py:88 skipped answer scores -1
F2 timer.py:40 timer ignores pauses

The findings from the last run's review.json, each with its id, file and line.

Review the current diff. For each earlier finding, put its id
in resolved if the code now fixes it, or report it again with
status still_open and the same id. Report other problems with
status new. Do not report anything else.

The instruction Claude follows: account for every earlier id, keep ids stable, and report only new or still-unaddressed issues.

Re-review with prior findings

Run 1 posts F1 and F2 as before and saves its findings as review.json, a build artifact.

After push 2, run 2 is a new session. review.json goes into its prompt, and the prompt asks Claude to report each earlier id again as still_open or list it in resolved, and to report any other problem as new.

Run 2 reports F1 and F2 as still_open and nothing new. Both already have comments, so the script posts nothing and leaves those comments as they are.

After push 3, run 3 gets run 2's findings in its prompt. It reports F1 as still_open, F3 on line 47 as new, and F2 in resolved.

The script resolves F2's thread and posts F3 under line 47. Line 88 keeps its one comment, and the review has posted three comments in all.

The earlier findings in each run's prompt told it what had already been said. The stable ids let the script match each finding to its existing comment, so F1 kept one comment and F2's thread was resolved.

Deleting all of the bot's comments before each run and posting again would lose the engineer's reply on F1 and still repost every open finding. Reviewing only the latest commit's diff would miss F1, because no later commit touched line 88. Raising the severity threshold would post fewer kinds of finding and still repeat each one. The guide's answer is to put the earlier findings in the prompt and report only new or still-unaddressed issues. Resuming run 1's session with --resume would not do the same job. Run 1's session was saved on a runner that no longer exists, and resuming it would also bring back run 1's reasoning about code that has since changed, along with its findings.

As of October 2026, Anthropic also offers two ready-made ways to get inline review comments. The Claude Code GitHub Action can post inline comments for a team, and Code Review, Anthropic's hosted review service on its Team and Enterprise plans, also resolves its own earlier comment when a later push fixes the issue. The exam guide tests the method this chapter builds, a schema and the team's own posting script, which works with claude -p on any CI system.

Part 4: With CLAUDE.md and the existing tests in context, Claude writes only the missing tests

With a fourth push on Friday evening, the engineer fixed line 47, and the next review put F3 in resolved. F1 was still open, because the engineer still believed the writer's reasoning about skipped questions.

Without written standards, the test job writes the easy tests.

is a file of instructions that Claude Code reads at the start of every in a project. A project's CLAUDE.md is committed to the , usually at its root, so everyone who the code gets the same file.

PhysicsWallah also runs a nightly job that asks Claude Code to write for scoring.py. The job writes the tests and opens a pull request with them, and the review job checks that pull request like any other, in its own session. Until now, test-engine's CLAUDE.md had six lines, all build commands.

The repository already has : make_attempt() in tests/factories.py builds a student's attempt, and tests/fixtures/jee_main_mock_07.json holds a 75-question mock with its answer key. Every run of the nightly job starts on a fresh in a new session, so with no written standards Claude decides for itself what is worth testing. It writes tests that only check a or a log line, and it writes attempt JSON by hand. Attempt JSON written by hand repeats work that make_attempt() already does and can differ from the real format.

CLAUDE.md for test generation

On night 1, the nightly job starts on a fresh runner and checks out test-engine. The run gets the prompt, a CLAUDE.md with six build commands, and the fixtures with nothing saying what they are for.

Claude writes 14 tests. Three check only a getter or a log line, and two build attempts from JSON written by hand instead of using the fixture.

On night 2 the job runs on another fresh runner in a new session. It gets the same files, so Claude writes the same kinds of tests.

The team writes its testing standards into the repository's CLAUDE.md: the answer states to test, what a valuable test checks, what not to test, and which fixtures to use.

On night 3 the checkout brings the new CLAUDE.md onto the runner, and Claude Code reads it first. Claude builds attempts with make_attempt() and writes 13 tests. Four cover answer states that had no test, and none tests a getter, logging or hand-written JSON.

A CI run gets only the standards that are in the repository it checks out, so once the team's standards are in CLAUDE.md, every run has them.

The standards reached the third night's run because they were in the repository that the run checked out.

Test generation without project context

The nightly job runs claude -p "Write tests for scoring.py". The repository has a CLAUDE.md with six lines of build commands, the existing tests in tests/test_scoring.py, a factory in tests/factories.py and a fixture file.

Claude proposes 14 tests.

Nine check cases that test_scoring.py already covers, under different names. Three test only a getter or logging. Two build an attempt by writing JSON by hand, while make_attempt() and the fixture file go unused.

None of the 14 tests checks a skipped question.

Claude had the code to test and no project standards, so it tested what was easy to see and repeated tests it had not read, and the case that mattered was missing.

Every CI run reads the project's CLAUDE.md.

claude -p loads CLAUDE.md from the folder it runs in, as an interactive session would. The job's checkout puts the repository in that folder, so the project's CLAUDE.md is read at the start of every CI run. A developer's own ~/.claude/CLAUDE.md stays on that developer's laptop, and no checkout copies it onto a runner.

The team rewrites test-engine's CLAUDE.md. Its testing section holds the three things the exam guide names: , the criteria for a valuable test, and the existing fixtures. The same file carries the marking rules and the review criteria that the review job reads.

## Testing standards (test-engine service)
- Tests use pytest. Run with: make test

How tests are written and run in this service. is the testing tool the team uses, and make test runs all the tests.

- Every scoring change needs a test for each answer state:
  correct, incorrect, unattempted, marked-for-review.

The testing standard: every change to scoring is tested for all four answer states. A question marked for review is one the student has flagged to come back to later.

- A valuable test checks a marking rule or a rank boundary.
  Do not write tests for getters, setters or logging.

The criteria for a valuable test, and the kinds of test not to write. A rank boundary is a case where two ranks meet, such as two students with equal marks sharing one rank. A getter test is one of the kinds not to write.

## Fixtures
- tests/fixtures/jee_main_mock_07.json: 75 questions, key.
- make_attempt(answers=...) in tests/factories.py builds an
  attempt. Use it instead of writing attempt JSON by hand.

The fixtures that already exist and how to use them, so that tests reuse them.

## Review criteria
- Marking: +4 correct, -1 incorrect, 0 unattempted.
- Report bugs in marks, ranks and timers, and security issues.
- Do not report formatting; the linter already runs.

The marking rules the review checks the code against, what the review job should report and what it should leave alone.

CLAUDE.md in every CI run

The engineer commits the new CLAUDE.md on the laptop and pushes it, and the repository's CLAUDE.md grows from 6 lines to 14.

The nightly test job starts on a new runner and checks out the repository. Its files, CLAUDE.md among them, are copied onto the runner.

Claude Code reads CLAUDE.md and sends it to Claude above the prompt. Claude proposes 13 tests, all built with make_attempt(). Four cover answer states that had no test, including test_unattempted_scores_zero, and nine still repeat tests in test_scoring.py.

The next time the review job runs on PR #4127, it starts on another new runner. Its checkout copies the same CLAUDE.md, and its request to Claude starts with the file, this time with the marking rules and the review criteria in view.

Both jobs read the same file from the repository. Writing "only valuable tests" in each job's prompt, raising a coverage target or switching to a bigger model leaves the written standards and the fixtures out of the run, while CLAUDE.md puts them into every run. The nine repeats remain, because CLAUDE.md does not say which cases the tests already cover.

A pull request can change CLAUDE.md itself. The workflow in Part 2 checks out the pull request's code, so the review reads the CLAUDE.md from the pull request. The author of a pull request could then edit the instructions that review their own change, for example by deleting the marking rules, and the review would follow the edited file. A careful pipeline reads the review instructions from the main , which the pull request cannot change before it is merged.

As of October 2026, Anthropic's docs recommend adding the flag --bare to scripted runs, and they say it will become the default for -p in a later release. A run with --bare does not read CLAUDE.md. A team that adds it passes the file to Claude another way, for example with --append-system-prompt-file CLAUDE.md, so the standards still reach every run.

Without the test file, Claude repeats tests that already exist.

Providing the existing tests in means giving Claude the test file itself, so that it can see which cases are already covered and write tests only for the rest.

Of the 13 tests proposed on night 3, four cover the missing cases and nine repeat tests already in tests/test_scoring.py. Claude was not given that file, and neither the prompt nor CLAUDE.md asked it to read the file, so it did not know which cases the file covered.

Existing tests in context

The nightly job runs with the new CLAUDE.md, and tests/test_scoring.py is not in Claude's context. Claude writes 13 tests without knowing which cases the file already covers.

Compared with tests/test_scoring.py, four of the 13 tests cover cases the file was missing, and nine repeat tests that are already in it.

When the job runs again with tests/test_scoring.py given to Claude before it starts, Claude can see the cases the file covers and the four it does not.

Claude writes one test for each missing case: unattempted scores 0, marked for review and answered, marked for review and blank scores 0, and equal marks share a rank.

With the existing test file in its context, Claude writes tests only for the cases the file does not cover.

The existing test file shows which cases are already covered, so with the file in the context the only tests left to write are the missing ones.

The existing test file shows Claude what is already covered.

The file can be piped in on , named in the prompt so that Claude reads it with its Read tool, or both. The prompt then asks for tests only for the cases the file does not cover.

The nightly job runs on the main branch, where the check for a skipped answer is still in place, so its new tests pass there. On Saturday morning the nightly job's pull request was still open, with nine of its 13 tests repeating existing ones. To test #4127's code, the engineer starts the test-writing job by hand on PR #4127's branch, with the existing test file in its context, so the job checks out the refactored code and runs the new tests against it.

claude -p "Here is tests/test_scoring.py. Write tests only for
scoring cases it does not already cover." \

The prompt asks for tests only for the cases the existing file does not cover.

  --allowedTools "Read,Edit,Bash(make test)" \
  < tests/test_scoring.py

Claude may read and edit files and run make test without asking. The < sends the test file in on stdin, so it arrives with the prompt.

Generating only missing tests

The job, started on PR #4127's branch, now hands Claude Code the existing test file as its input (stdin), using < tests/test_scoring.py, and asks for tests only for the cases the file does not cover.

The API request carries CLAUDE.md, then the contents of test_scoring.py with its nine tests, then the prompt.

Claude proposes four tests, one for each missing case, with test_unattempted_scores_zero first.

The job has checked out PR #4127's branch, and make test runs the four tests against its code. Two of the four fail: test_unattempted_scores_zero and test_marked_for_review_without_answer_scores_zero each expect 0 and get -1 at line 88. The other two pass.

With a fifth push on Saturday, the engineer restores the check for a skipped answer and adds the four tests. The next review lists F1 in resolved, and every check is green.

On Sunday, the student's phone will show a score of 183 and All India Rank 6,412.

Claude wrote only the four missing tests because the test file in its context showed the nine that already existed. A bigger model or a larger context window does not help while the file is left out. With the file in the prompt, the run wrote four tests, two of which failed on line 88, and the engineer restored the check.

Part 5: Check yourself

Build PR #4127's review step.

PhysicsWallah's review job must run on a runner with nobody at it, post each finding under its line, stop repeating comments after each push, and judge the code without the writer's reasoning. Select everything the job needs, then check.

Select at least one item first.

Exam question: stop the repeated comments.

Your CI pipeline runs a Claude Code review with claude -p on every push to an open pull request, and a script posts each finding as an inline comment. After three pushes, developers report that one issue on line 88 carries three identical comments, and comments are still open on code that has since been fixed. What is the most effective way to fix this?

PhysicsWallah appears in this chapter only to explain how Claude Code and Claude work. PhysicsWallah has no connection with these tutorials, and the code, systems, numbers and events in this chapter were written for the lesson. They do not describe how PhysicsWallah works, and PhysicsWallah does not use this code.