There is a version of AI-assisted verification that sounds appealing and does not survive contact with real documents: upload anything, get a verified company back, no humans involved.
The problem is not that models are bad at reading documents. They are increasingly good at it. The problem is that they are equally confident when they are wrong, and in verification the cost of a confident error is not symmetrical with the cost of a slower answer.
Confidence is not accuracy
An extraction model returns a value and a score. It is tempting to treat the score as a probability of correctness. It is not, quite. It is a measure of how well the input matched patterns the model has seen, which correlates with correctness on typical inputs and decouples from it on atypical ones.
This matters because atypical inputs are exactly where verification earns its keep. The routine document was never the risk. The unusual one is — and that is precisely where the score is least trustworthy.
A model asked to find a registration number in a document that does not contain one will often produce a number, because there are numbers on the page and one of them is more registration-number-shaped than the others. It is doing what it was asked. It has no way to express "this document is not what you think it is" unless the system is designed to let it.
The asymmetry that drives the design
In verification, the two failure modes cost very different amounts.
A false negative — flagging something correct for review — costs a few minutes of a reviewer's time.
A false positive — accepting something incorrect — means an entity was verified on bad evidence. That surfaces months later, in the worst case during an audit or an investigation, and by then it has been relied on by other decisions.
Given that asymmetry, the correct operating point is nowhere near maximum automation. It is well into over-flagging, because the thing you are optimising is not throughput.
What review is actually for
The value of a reviewer is not re-reading what the machine read. If that is what review consists of, it is expensive theatre and you have automated nothing.
The reviewer is there for the judgements the pipeline cannot make:
Does this document make sense as a whole? Extraction reads fields. A person reads a document. An incorporation date after the date of the first filing is individually valid in every field and collectively impossible.
Is this consistent with everything else we hold? The register extract says one thing, the articles say another. Neither is wrong on its face.
Is this plausible? A newly formed company with a director resident in three countries and a corporate shareholder in an opaque jurisdiction may be entirely legitimate. It is also a pattern worth a second look, and no field-level check produces that signal.
Is the evidence sufficient? Not "did the checks pass" but "would I be comfortable defending this decision". That is a judgement with a person's name on it, and it should stay that way.
Designing the loop so it works
Human-in-the-loop fails when it is bolted on. A few things make the difference.
Show the source. A reviewer given a value and a link to the exact page can confirm in seconds. A reviewer given a value alone has to redo the work, and review becomes slower than doing it manually.
Route by confidence, and mean it. High-confidence fields pre-filled, medium flagged for confirmation, low left empty. If everything lands in one undifferentiated queue, the scores are decoration and reviewers stop reading them.
Surface disagreement first. When documents conflict, put that at the top of the screen. It is the highest-value thing a reviewer can look at, and it is the thing manual processes miss most often.
Make rejection easy and specific. A reviewer who can only accept or reject wholesale will accept marginal cases rather than restart the process. Let them reject one field and request one document.
Record the decision, not just the outcome. Who reviewed, what they saw, what they changed, and when. This is what makes the process defensible later, and it is also the training signal for improving the pipeline.
Reviewers get worse when automation gets better
This is the uncomfortable part, and it is well documented outside our field.
As automation accuracy rises, reviewers see fewer errors, and their attention drops accordingly. Someone who confirms a hundred correct extractions develops a reasonable expectation that the hundred-and-first is correct too. The reviewer is least alert exactly when the rare error arrives.
There is no clean solution, but some things help: keep the volume per reviewer low enough for attention to hold, rotate reviewers across case types, sample-audit accepted cases rather than only rejected ones, and design the interface so confirming requires a real action rather than a reflex.
The failure mode to design against is not the model being wrong. It is the model being wrong and the reviewer agreeing without looking.
Where this sits at Dorsko
DorskoVerify is in development, and human review is a designed-in stage rather than a fallback. The capabilities listed on its page — confidence scoring, source-page references, discrepancy detection, human-in-the-loop review — are design targets, not shipped features.
The reason they are grouped that way is the argument above: scoring, referencing and discrepancy detection exist primarily to make the human review stage fast and well-aimed. Without them, review is a bottleneck. With them, it is the part of the pipeline that catches what nothing else can.