TestFlow

Article 3 of 5

Test Run

The execution of chosen cases against one version, at one moment.

A test case says what should happen. A test run records what actually happened — on this build, on this environment, today. That is why results live on the run and not on the case.

Run the same cases tomorrow and you get a new run. The old one stays as evidence of what you knew when you shipped.

What it is made of

Name
Usually the purpose plus the build: “Regression Run #4”, “Smoke — Release 1.4”.
Selection
Which cases are included. A smoke run is a handful; a regression run is most of them.
Results
One status per case, plus a comment when something went wrong.

Result statuses

Passed
The actual result matched the expected result. Every step of it.
Failed
The product behaved differently than expected. This is a bug in the product.
Blocked
You could not check it: the environment was down, a dependency was broken, the feature was not deployed. Nothing is known about this case.
Skipped
You deliberately did not run it — out of scope for this run, or not worth the time.
Not Run
Not reached yet. The run is unfinished.

Rules that save you later

  • Failed and Blocked are not the same. Failed says the product is wrong; Blocked says you learned nothing. Marking blocked cases as failed invents bugs that do not exist.
  • Never edit a finished run to make the numbers look better. Fix the product and start a new run.