Unthought.

Findings and evidence

What separates a report that changes something from one that gets filed is whether each claim arrived checkable.

The difference between a report that changes something and one that gets filed is rarely the quality of the thinking inside it. It is whether each claim arrived with enough attached that a skeptical reader could verify it themselves, without taking a single sentence on trust.

A finding states what was observed, where, and when. "The database has no backup" is an assertion. "No automated backup job exists for the production database, and the most recent manual dump in the bucket is dated eleven months ago" is a finding — and the second one survives an argument with the person who believes there is a backup.

Observation and interpretation are separated in the document's structure, not merely in its tone. A reader must be able to accept the fact and reject the conclusion drawn from it, because that is what happens whenever a report meets somebody with more context than its author had.

Severity is expressed as consequence instead of as a number on a scale. "High" is a summary of a judgement whose reasoning has been thrown away; "if this credential is lost, no one can deploy and there is no documented recovery path" is the judgement itself, and it can be argued with.

Every finding carries both what it costs to fix and what it costs to leave. Without the second figure everything on the list reads as urgent, nothing gets sequenced, and the report's ordering — which was the most carefully considered thing about it — is discarded by whoever picks the easiest item first.

Where a finding can be reproduced, the steps are included so a reader can see it for themselves. A finding somebody has personally observed is a finding that stops being contested.

Confidence is stated wherever it is less than certain. "Probably", "in the one instance examined", and "reported by the team but not verified" all belong in a technical report, and each is worth considerably more than a confident sentence that turns out to be wrong — because one wrong confident sentence retroactively devalues every correct one beside it.

What would change the conclusion is named. A finding that no possible evidence could overturn is a position, not a finding, and stating the thing that would falsify it is what marks the difference.

Findings are dated, because a report describes a system at a moment and systems move. A reader picking the document up a year later is entitled to know how much of it may have gone stale, and a report without a date invites being treated as either current or worthless, with no way to tell which.

Exclusions

What this does not cover.

  • Severity scales that report a number in place of the reasoning behind it.
  • Findings stated without the observation they rest on.
  • Undated reports.