Show the Working

ErgoSphere is a library of published assessment methods, implemented in software. For each method we are writing a verification and validation dossier: a controlled document that shows, with dates and evidence, how our implementation was checked against the source that defined it. Completed dossiers are released on request, under NDA. This page is the public record of the programme itself: how it works, and what a method has to survive before we stand behind its numbers.

The reason is a discipline, not a trend. I come from Systems Engineering and from modelling and simulation, two fields built on the same uncomfortable rule: an unverified model is an opinion, however good it looks. Verification and validation are not paperwork that follows the real work; they are what makes a number fit to be stood behind in a report. That has been the quiet voice through every tool ErgoSphere ships, saying that assurance matters more than the next shiny thing. The Dossier Project is that voice, finally given a format and a public record.

Development here moves fast, and speed can read as risk when you cannot see what a method passes through before it ships. We are not going to look slower. We are making the path visible.

A tool returns a number. The number goes into a report, and the report goes to a client, a regulator, sometimes a court. Underneath sits an assumption almost nobody checks: that the number the software produced is the number the published method would have produced.

Assessment methods live in lookup tables, thresholds and scoring rules, transcribed by hand from papers into code. A transcription error in one cell of one table does not crash anything. It throws no exception and fails no ordinary test. It quietly returns a slightly wrong answer, forever.

Every method gets a dossier on the same fixed skeleton. Each dossier opens on a status board of eight components, and each component is marked with one of four words: Complete, Partial, Not started or Not applicable. Nothing else. A reader who gets no further than that board still leaves with an accurate picture.

Amber is releasable. A dated Partial with real evidence behind it is worth more than a tick a reader cannot check, so the first dossiers released will not be all green, and that is by design. Defects found during dossier work are recorded, not quietly fixed in passing: the dossier is evidence, not a repair job.

This is the rule the whole programme hangs on. If the code and the test were both written from the same transcription of the source, then the test only proves the code equals itself. Every cell of that transcription could be wrong, and every test would still pass. That is how heavily tested software can still be incorrect, and it is why a large test count, on its own, proves less than it appears to.

So the canonical test cases are never taken from our own tables. They come from the published source's own worked examples, or they are hand-computed by a person reading the source directly. Two independent paths from the paper to a number, meeting in the middle. When they agree, that agreement means something. When they disagree, we have found something real.

Verification did not start with this page. The testing behind ErgoSphere is long-standing: counted on 30 Aug 2026 there were 7,848 test methods marked [Fact] or [Theory] in the main ErgoSphere.Tests project alone, a further 288 across the simulation and glare-engine projects, and more than 600 distinct work-package references cited in the test code. Method-fidelity audits were being run well before the dossier format existed. What is new is the dossier itself, the externally readable write-up that makes that body of work legible to someone outside the company.

What the write-up demands is easier to show than to describe, so take one method: REBA.

Its scoring tables were checked cell by cell against the published source. Not against our own notes, and not against the tests, which would only prove the code agrees with itself: against the paper. 240 cells compared, all 240 matching. RULA had the same treatment on the same day: 272 cells, all 272 matching. The comparison is recorded, so a reviewer can repeat it rather than take our word for the result.

Then the golden case. REBA case R-01 is a worked example published in the source itself, encoded independently of our tables and run as five automated checks. All five passed. The fixture was then deliberately broken, one expected value altered, to confirm the run goes red and goes red on the right assertion, before being restored and confirmed green again. A test that cannot fail proves nothing, so every golden case gets that treatment.

That is one method, one set of tables, one case, against one of the eight components. Multiply it across a library of methods and the pace of this work is no mystery. It is also why we put no date on the end of it.

Not every method can be checked that way, and the ones that cannot are the more interesting half. REBA and RULA publish lookup tables, so there is arithmetic to reproduce. A method like SHERPA, SCTA or TRACEr publishes a procedure instead: a taxonomy, a sequence of steps, a way of decomposing a task. There is no worked example with an answer at the bottom, so there is nothing to encode as a canonical case, and the dossier says so in the plainest way available. That row reads n/a rather than not done, because it cannot move.

What stands in its place is a process conformance walk, and it has meant something different every time. TRACEr's source publishes 124 taxonomy entries, so the check asserts the published names in the published order. GDTA's publishes a worked 46-row hierarchy, so the check rebuilds 21 of those rows through the tool's own write API and reads back the published outline numbers, which the tool computes rather than stores. SAGAT's publishes nothing reproducible at all, so the check walks its seven published procedure steps and asserts what the tool must refuse to do.

Whether that catches anything is a fair question, and the answer is that it does. The Detection-Response Task is one of these methods, with no canonical case available to it in principle. Walking it found that every DRT report the tool had ever produced stated a hit rate of exactly 100 per cent: misses were counted from a list that only ever held valid responses, so the recommendation written to flag a poor hit rate sat behind a number that could not occur and had never once fired. Found, fixed, guarded by a named test, and written into the dossier. “No number to validate” has never yet meant “nothing to check”.

A dossier is a controlled document. It carries the implementation specification, the provenance of every scoring table, the golden cases with their run records, the boundary behaviour, and the internal review signature. That is the working, and the working is the part worth protecting: it describes how the software is built as much as how it was checked.

So dossiers are not posted for download. A dossier is released on request, under NDA, to the customers, auditors and reviewers who need to verify what they rely on. If you are assessing ErgoSphere for work that has to stand up, ask about the method you actually care about and we will tell you where it stands.

Each method's status board lives inside its own dossier. The Engineering Record publishes the register: that a document exists, the version it carries and the rung it stands on. That is deliberately as far as it goes. A rung is not a verdict on the method. It says how far the paperwork has been taken, not whether the arithmetic is right, and a reader who reads one as the other has learned the wrong thing. Only the dossier answers that, for one method, in front of someone who can ask us follow-up questions. We rank no tool against another, anywhere.