How candidates run tests in a coding screen

Candidates run the project's tests themselves, as often as they like, from the Tests tab in the editor or from the terminal. The Tests tab runs the test command from your kendor.yaml and shows the result. If that command prints TAP, each test gets its own row.

Where tests live

Tests are ordinary files in the project. Put them wherever your test runner expects them, such as tests/, src/__tests__/ or next to the code. Candidates can open and read them, and they can change them too.

Candidates are told before they start: the start page says they can run the project's tests from the Tests tab or the terminal whenever they want.

In a single-file challenge, the candidate only sees the file they edit. The test file stays out of their file tree but still runs when they choose Run tests.

What runs when a candidate chooses Run tests

The Tests tab is in the editor's left-hand bar, behind the beaker icon. Choosing Run tests runs the test.command of one service in your kendor.yaml:

• the service named in kendor.gradeService, if you set it
• otherwise the service under preview.service
• otherwise the only service, if there's just one

version: 1
runtime:
  type: python
  version: "3.12"
services:
  app:
    dev:
      command: uvicorn main:app --host $HOST --port $PORT --reload
      port: 8000
    test:
      command: pytest --tap-stream
preview:
  service: app

The command runs inside the candidate's own sandbox, on their current files, with dependencies already installed. It runs from the project root, or from the service's workDir if it has one. You can also set the command in the screen editor's runtime config form, in the Test command (optional) field, which writes the same kendor.yaml.

Candidates can change some of kendor.yaml for their own session, such as services and install steps, but not the test command. Kendor always runs the one you set.

Several services

For a project with more than one service, such as an API and a worker, list them under kendor.gradeServices. Each service's test command runs in its own folder, and the rows are merged, each prefixed with the service name: api › creates a user. See Multi-service apps.

What candidates see

After a run, the tab shows a summary such as 7/9 passed (78%) and one row per test with its time. Failing tests open automatically to show the failure message. The refresh button in the summary runs the suite again.

If the command printed no TAP, there's a single row named test suite. It passes when the command exits with code 0 and fails otherwise. Candidates can run the command in the terminal to see its full output.

If Kendor can't run the suite at all, for example because there's no test command, the tab shows the error instead of rows.

Get one row per test with TAP

Kendor reads TAP (ok 1 - name / not ok 2 - name lines) from the command's standard output. Anything printed to standard error is ignored when counting, so build noise there can't add or hide a test.

These commands print TAP in Kendor's sandboxes:

pytest --tap-stream

node --test

go test -json ./... | kendor-gotest-tap

cargo test --color never 2>&1 | kendor-cargo-tap

rm -rf target/surefire-reports; mvn -B -q test 1>&2; kendor-junit-tap 'target/surefire-reports/*.xml'

./vendor/bin/phpunit --log-junit /tmp/kendor-phpunit.xml

• Python: Kendor's Python comes with pytest-tap, so --tap-stream works without adding anything to your requirements.
• Node.js: node --test prints TAP when it isn't attached to a terminal, which is how the Tests tab runs it.
• Go, Rust, Java and C++: kendor-gotest-tap, kendor-cargo-tap and kendor-junit-tap are converters that ship in every sandbox. kendor-junit-tap reads JUnit XML reports, so it also works for Gradle (build/test-results/test/*.xml) and for C++ test runners that write JUnit XML.
• PHP: when a single test command at the project root has PHPUnit write its JUnit log to /tmp/kendor-phpunit.xml, Kendor reads each test from that log.

Running tests in the terminal

Candidates can also run the same command, or any other, in the terminal. The output there is whatever the runner prints in a terminal, which for some runners is a friendlier format than TAP. Both run in the same sandbox on the same files.

Reviewers can run them too

On a submission's review page, Open in editor opens the candidate's code in the full editor with its own sandbox. The Tests tab works there as well, so you can run the suite against exactly what they submitted.

If a candidate edited a test file, the Solution tab marks it as changed, and Diff vs starter shows what they changed. See Review a submission.