How to review take-home assignments fairly

To review take-home assignments fairly, write down what good looks like before you open the first submission, then read the diff and run the code for every candidate the same way. Use replays, terminal history and agent transcripts to understand how the work was done, not as the verdict. Have each reviewer record a recommendation alone before any discussion, and write down the reason for the final decision.

Decide your criteria before you read anything

Once you've seen a submission, it shapes what you think the task was testing. Write the criteria first, while every candidate is still a blank page.

Keep it to a short list of things you can point at in the code:

• Correctness. Does it do what the task asked? Do the tests pass, and are the edge cases handled?
• Clarity. Could a teammate pick this up tomorrow? Are names, structure and comments helpful?
• Testing. Did the candidate add or update tests for the change?
• Judgment. Did they stay in scope, make sensible trade-offs and explain them?
• Fit to the codebase. Did they follow the conventions already in the project?

For each one, write a sentence on what "strong" and "weak" look like. Share the list with every reviewer before the first submission comes in.

What not to judge

Leave out things the task didn't ask for: a different framework, perfect formatting, or features beyond scope. Don't reward volume. A small, correct, well-tested change beats a sprawling rewrite.

Read the diff, then run it

Start with the diff against the starting code, not the whole project. It shows exactly what the candidate changed and keeps you from grading code they inherited.

Then run it. Start the app, run the tests, and try the obvious inputs and one or two awkward ones. Code that looks right sometimes isn't, and code that looks odd sometimes works well. Ten minutes of running catches what an hour of reading misses.

Do this in the same order for every candidate, with the same inputs. Consistency is most of what "fair" means.

Use the process as context, not as the verdict

Many tools now show how a submission was built: a keystroke replay, the terminal history, the AI agent conversation. These are useful, but easy to over-read.

• Replays show how someone approached the problem: where they started, what they tried first, when they wrote tests. Use them to answer a question you have about the code, not to mark someone down for a messy first draft.
• Terminal history shows whether they ran the tests, read logs and checked their work. That's a real signal for a role where they'll debug production issues.
• Agent transcripts show how they briefed the agent and whether they checked its output. If agents were allowed, heavy use is not a problem; unchecked use is.

The final code is what they chose to hand in. Judge that first, and use the process to explain it.

How should you treat integrity signals?

Tab switches, pastes from outside the editor and time away are facts, not accusations. There are many ordinary reasons for each: looking up documentation, a message from a child, pasting their own snippet from a notes app.

• Read them next to the code. Fifty lines pasted from outside, followed by code the candidate can't explain, deserves a question. A paste of a URL doesn't.
• Never reject on a signal alone. If something worries you, ask the candidate about it in the follow-up interview.
• Apply the same reading to every candidate. If you only look at signals when you already have doubts, you'll find what you're looking for.

Record recommendations before you discuss

The first opinion voiced in a debrief pulls everyone else toward it. To get independent judgments:

1. Each reviewer reads and runs the submission alone.
2. Each records a recommendation and a few lines of evidence, before reading anyone else's.
3. Only then does the group discuss. Start with the points of disagreement.
4. One person makes the decision and writes down why.

Evidence means specifics: "handled the empty-cart case and added a test for it", not "seems solid".

Write the decision down

Record what was decided, by whom, and the main reasons, tied to your criteria. This helps you give the candidate useful feedback, keeps later rounds consistent, and makes it easier to spot drift when you look back over a hiring round.

Decide how and when the candidate hears back, and stick to it. A short, specific note is a courtesy most candidates remember.

Doing this in Kendor

In Kendor, open Candidates, pick the candidate and choose Open review. Reviewers listed under Default reviewers on the screen are routed every submission, and the Reviewer menu on the review page reassigns one. More in Review a submission.

Read and run the code

The Solution tab shows the submitted files, with Diff vs starter for exactly what changed. To run it, choose Live review: the submission opens in a live room with its own sandbox, where you can start the app and run the tests. The original submission stays unchanged. To walk through the work with the candidate, use Invite candidate next to it.

See how it was built

• Timeline replays the work keystroke by keystroke. Skip idle fast-forwards through quiet stretches.
• Terminal replays what appeared in the candidate's terminal.
• AI agents shows each agent conversation, when agents were allowed.
• Requests lists requests the candidate sent from the editor's Requests panel.
• Comments is where reviewers leave notes for each other; @name notifies a teammate.

Tabs only appear when there's something recorded for them.

Integrity signals

When Proctoring is on for the screen, the Activity tab lists tab switches, external pastes and fullscreen exits, each with its time, and how long the candidate was away in total. When it's off, no integrity signals are collected. Kendor doesn't record the candidate's screen or webcam. See Integrity signals.

Recommendation and decision

Each reviewer picks a Recommendation: Strong yes, Lean yes, Lean no or Strong no, with an optional note. Everyone's recommendations show on the same card, so record yours before you read the others. When reviewers split, Kendor says so and suggests talking it through first.

Then someone makes the Decision: Advance or Reject, with an optional reason that goes into the decision log. Undo this decision reverses it. Nothing is emailed to the candidate automatically when you reject; send the rejection yourself when you're ready, with Send rejection email.