verodocs

Contracts

vero.events/v1

The native output of vero run (chapter 8.3): newline-delimited JSON, one event per line, written as the run happens and flushed line by line, so a run that is killed leaves a file whose every complete line parses. vero run --events <path> writes it; --events - writes it to stdout and moves the human report to stderr. The file is a contract: its shape changes only with its version.

Carries all seven terminal states without the lossy mapping JUnit forces. JUnit and TAP are exports; this is the record.

Rules

  • Every line is one JSON object with the four common fields below.
  • vero.events/v1 changes only additively: a new event type or a new field may appear; no field is removed, renamed or retyped. A change that needs any of those is vero.events/v2. Consumers must ignore fields and types they do not know.
  • Durations are milliseconds as JSON numbers. A timing phase that did not happen on a request (dns, connect or tls on a reused connection) is the string "n/a", never 0 (chapter 4.3).
  • Bodies are cut at --events-body-cap bytes (default 65536) and the event says so with bodyTruncated: true. Nothing is cut silently. Bytes that are not UTF-8 become U+FFFD.
  • Secrets are scrubbed by value from every line before it is written (C04).
  • Each --repeat iteration is its own run with its own runId, from run.start to run.end.
  • Order: run.start, then per task task.start, per step step.start, the step's request events (or attempt events for a waitFor, or one query event for a sql step, or one fixture event for a fixture step, or one exec event for an exec step, or one ws event for a ws step, or one grpc event for a grpc step), its assert events, step.end, then task.end; tasks that never started have only a task.end; finally run.end. Events of tasks running at the same time interleave.

The tables below are checked by internal/report/contract_test.go against the Go types in internal/report/events.go: a field in one and not the other fails the build.

Common fields

Field Type Present
schema string, always vero.events/v1 always
ts string, RFC 3339 UTC with milliseconds always
runId string, the run's id always
type string, one of the types below always

run.start

Field Type Present
plan string, metadata.name always, may be empty
planFile string, the plan's path as given always
version string, vero's module version and Go toolchain always
jobs number, --jobs always
flags object, every flag set on the command line, name to value always, may be empty
runtimes array of {name, binary, path, version}, each script runtime the plan uses as found on PATH (F01) when the plan has script steps

task.start

Field Type Present
task string always
section string, tasks or finally always
holds array of strings, resources the task holds when it holds any

step.start

Field Type Present
task string always
step number, 1-based always
kind string: http, graphql, ws, grpc, sql, exec, waitFor, script, fixture always

request

One per HTTP exchange; a repeat emits one per request, in send order for serial and completion order for parallel.

Field Type Present
task string always
step number, 1-based always
index number, 0-based position among the step's requests always
method string always
url string always
status number, 0 when no response arrived always
proto string, the protocol that answered: HTTP/1.1 or HTTP/2.0 when a response arrived
requestHeaders array of {name, value}, names lower case, sorted always
requestBody string; a file sent by bodyFile or a multipart part appears as «file <path>, <size> bytes, sha256 <hex>» in place of its bytes always, may be empty
responseHeaders array of {name, value} always, empty when no response
responseBody string, decompressed always, may be empty
bodyTruncated boolean, true when either body was cut here or by the driver's cap always
wireSize number, response body bytes as received (compressed if compressed) always
size number, response body bytes after decompression always
duration object: dns, connect, tls, ttfb, total, each a number or "n/a" always
connection string, new or reused always
error string, the transport error when the exchange failed
timedOut boolean when the deadline fired
phase string, the phase in progress when the deadline fired: dns, connect, tls, write, ttfb, body when timed out
redirects array of {method, url, status}, the hops before this response when followRedirects is on and there were hops; each hop also has a request event of its own, just before
retried boolean, true for an attempt that failed in a way the step's retry covers and was sent again when true

assert

Field Type Present
task string always
step number, 1-based always
index number, 1-based position in the step's assert list always
expr string, as written in the plan always
location string, file:line always
outcome string: held, failed, n/a, error always
error string when outcome is error
values array of {path, value}, every path the assertion read and its value when not held
detail array of strings, the failure explanation (elements that broke an all) when not held

query

One per sql step (D01): the query as written, its bound arguments, and how many rows came back. The rows themselves are in the assert events' values when an assertion fails.

Field Type Present
task string always
step number, 1-based always
connection string, the connection's name in the plan always
query string, the literal query always
args array of strings, the rendered, bound arguments always, may be empty
rows number, rows returned; 0 also when the query failed always
columns array of strings when the query ran
error string when the query failed
timedOut boolean when the step's deadline fired
durationMs number always

fixture

One per fixture step: the statement as written, its bound arguments, and whether the write committed. A fixture step writes reference data through a connection declared writes: true, and this event is the label on that write: a reader of the log can see every row a run put in a database, and through which connection.

Field Type Present
task string always
step number, 1-based always
connection string, the connection's name in the plan always
statement string, the literal statement always
args array of strings, the rendered, bound arguments always, may be empty
rowsAffected number; 0 when it rolled back always
committed boolean; false means nothing was written always
error string when the statement or the commit failed
timedOut boolean when the step's deadline fired
durationMs number always

grpc

One per grpc step (B12, B13), after the call returned or the step ended.

Field Type Present
task string always
step number, 1-based always
target string always
method string, /package.Service/Method always
message string, the request as JSON always; empty for a method that takes a stream
sent array of strings, each request as JSON, in order for a method that takes a stream
code string, the status name; empty when no server answered (error says why) or vero stopped the stream (stopped) always
statusMessage string always, may be empty
response string, the response as JSON with exact numbers always, empty unless the code is OK and the method returns one message
received array of {atMs, body}, each streamed response and when it arrived after the call started when a server stream sent any
stopped boolean, true when vero ended the stream at expect.count or expect.timeout when true
headers array of {name, value} always
trailers array of {name, value} always
bodyTruncated boolean, true when any message or response was cut at --events-body-cap always
durationMs number always
firstMs number, when the first streamed response arrived when a server stream sent any
error string when no server answered or the step timed out
timedOut boolean when the step's deadline or the run's fired

ws

One per ws step (B11), after the socket closed or the step ended.

Field Type Present
task string always
step number, 1-based always
url string always
status number, the upgrade response's status; 101 on success, 0 when none came always
sent array of strings, the messages sent, in order always
received array of {atMs, body}, each message and when it arrived after the upgrade always
closed object {code, reason}, the server's close frame when the server closed
bodyTruncated boolean, true when a message was cut at --events-body-cap always
duration object: connect, tls, upgrade, first, total, each a number or "n/a" always
error string when the step errored or timed out
timedOut boolean when the step's deadline or the run's fired

exec

One per exec step (D04): the process that ran and what it printed.

Field Type Present
task string always
step number, 1-based always
argv array of strings, the rendered command always
dir string, the working directory always
exitCode number, or null when the process was killed or never started (never -1) always
stdout string, as captured, then cut at --events-body-cap always, may be empty
stderr string, likewise always, may be empty
stdoutBytes number, bytes the process wrote to stdout, kept or not always
stderrBytes number, likewise for stderr always
stdoutTruncated boolean, stdout passed the step's maxOutput and the rest was discarded always
stderrTruncated boolean, likewise always
bodyTruncated boolean, stdout or stderr was cut here by the event body cap always
error string, why it did not start or finish when it did not
timedOut boolean when the deadline killed the group
signal string, SIGTERM or SIGKILL, the last signal sent to the group when timed out
durationMs number always

script

One per event a script step's child wrote to stdout, and one per line of its stderr, emitted as they arrive (F02). Secret values are scrubbed like every other line. The protocol is docs/content/contracts/script-v1.md.

Field Type Present
task string always
step number, 1-based always
kind string: log, step, result, or stderr for a stderr line always
level string for log and stderr
msg string for log and stderr
name string for step
status string for step
durationMs number for step
ok boolean for result

attempt

One per try of a waitFor step (chapter 7.3), in order. A wait never produces request or assert events: its tries are not requests the plan counts and its until is not an assertion.

Field Type Present
task string always
step number, 1-based always
attempt number, 1-based always
status number, 0 when the try got no response always
error string, the transport error, such as connection refused when the try failed
ready boolean, whether until held on this try always
durationMs number, this try's total time always

step.end

Field Type Present
task string always
step number, 1-based always
state string, the step's state: passed, failed, errored, timed_out always
reason string when not passed
durationMs number always
attempts number, requests a waitFor made on waitFor steps (C05)
waitedMs number, how long a waitFor waited on waitFor steps (C05)
requests number, requests the step put on the wire, every retry and redirect hop included when the step sent any
retries number, attempts that were retried when any were

task.end

Field Type Present
task string always
section string, tasks or finally always
state string, one of the seven: passed, failed, errored, timed_out, skipped, blocked, not_run always
reason string when not passed
blockedBy string, the upstream task that blocked it when blocked
durationMs number, 0 for a task that never started always
hosts array of strings, the host:port of every request the task sent, ws upgrades and grpc targets included (E02's heuristic reads these with the task.start and task.end times) when it sent any
unverified boolean, the task passed only as passed (unverified): a script step returned ok with no step events (F02) when true

repeat.summary

One per task after the last run of vero run --repeat N (E01), when N is more than 1. It belongs to the whole invocation rather than one run: runId is the last run's, and runIds lists them all.

Field Type Present
task string always
runs number, N always
runIds array of strings, every run's id in order always
counts object, all seven states to how many runs ended the task in each always
flaky boolean, the task did not end in the same state every run always
varied array of {step, index, expr, outcomes, observed}: each assertion whose outcome varied, outcomes mapping held/failed/n/a/error to counts, observed a list of {path, values: [{value, count}], other} with at most 10 distinct values per path, most frequent first, the rest counted in other when flaky and some assertion varied

compare.summary

One per task after vero run --repeat N --compare-jobs (E02, C13), after the last run at the higher job count. Like repeat.summary it belongs to the whole invocation: runId is the last run's. It holds what the terminal's --jobs 1 versus --jobs N table printed.

Field Type Present
task string always
jobs number, the higher job count always
serialRuns number, runs at --jobs 1 always
serialPassed number, of those the task passed always
wideRuns number, runs at --jobs N always
widePassed number, of those the task passed always
verdict string: stable, flaky-wide-only (passed every run at --jobs 1, not at --jobs N) or flaky-both always
hint string, the hint as the terminal printed it, lines separated by \n when the verdict has one

run.end

Field Type Present
counts object, all seven states to their counts, zeros included, finally tasks included always
teardown object, the finally tasks alone, all seven states to their counts (C10) when the plan has finally tasks
tasks number always
unverified number, passed tasks that were passed (unverified); they are in counts.passed too when not zero
exitCode number, the exit code this run calls for (chapter 8.4) always
wallMs number always

This page is docs/content/contracts/events-v1.md in the repository.

verodocs