Intelligent document processing, built to be checked.

Intelligent document processing - IDP - is often described as OCR with a model attached. That framing causes bad systems. A document pipeline that holds up in production has five distinct stages, and the interesting engineering is in the last three.

Ingest accepts the file from wherever it arrives and normalises it: a phone photo of a certificate, a scanned PDF, a native PDF with a real text layer, an email attachment. Classification determines what the document is, because everything downstream depends on knowing whether this is an invoice, a trade licence, or a delivery note. Extraction pulls the fields that matter for that document type. Validation checks the extracted values against rules and against other systems. Review puts anything uncertain in front of a person. Only then does the result get written anywhere that matters.

A field extracted with high confidence can still be wrong, and a field extracted with low confidence can still be right. Confidence tells you how sure the model is, not whether the value is correct. Validation is the layer that converts one into the other, and it is where most of the durable value in an IDP system sits.

Format validation checks the shape: a tax registration number has a known length, a date is a date, an amount parses. Arithmetic validation checks internal consistency: line items sum to the subtotal, tax is the expected proportion, the total matches. Cross-system validation is the strongest of the three: does this invoice reference a purchase order that exists, for a supplier on file, within the amount that was approved?

A document that passes all three can often proceed unattended. A document that fails any of them should go to a human with the specific failure surfaced - not simply flagged as needing attention.

Several characteristics of documents circulating in the UAE genuinely change the engineering, as opposed to being marketing colour.

Bilingual documents are common: Arabic and English on the same page, sometimes in the same table, with right-to-left and left-to-right text interleaved. This affects OCR configuration, reading order, and how bounding boxes are handled - and it needs testing on real bilingual samples rather than assumed to work.

Expiry is a first-class concept. Trade licences, establishment cards, visas, insurance certificates, and equipment certifications all carry validity windows, and the operationally valuable output is often not the extraction at all but the alert thirty days before a document lapses. That means storing expiry as structured data and building a scheduler over it, not just filing the PDF.

Record retention matters. VAT and corporate tax regimes impose record-keeping obligations, so the archive, its retention period, and its retrievability are part of the system design rather than an afterthought. Confirm the specific obligations that apply to your entity with your own tax or legal advisers - they vary by activity and by free-zone status, and this guide is not a substitute for that advice.

Data residency and access are worth deciding explicitly: where documents are stored, where inference runs, who can retrieve an original, and how long anything is retained. These are architecture decisions with contractual consequences, best settled before a build rather than during one.

The review surface determines whether the system is adopted. It is not an admin table - it is the main product for the people who use it every day, and it should be designed with that seriousness.

The reviewer needs the source document and the extracted values side by side, with each extracted field linked to the region of the page it came from so a value can be verified with a glance rather than a search. Fields that failed validation should be first and should state why. Correcting a field should take one action, and the correction should be stored as training signal and as an audit entry.

Queue design matters as much as screen design. Items should be prioritised by consequence and deadline rather than arrival order, assignment should prevent two people reviewing the same item, and the queue should be visibly bounded so a reviewer can see the end of the work.

The safe rollout is shadow mode. The pipeline runs against real documents in parallel with the existing manual process, and nobody acts on its output. You compare the two, measure disagreement per field and per document type, and find out where the system is genuinely weak on your documents rather than on a vendor's samples.

From there, thresholds are set per field from observed behaviour - typically conservative at first, so more goes to review than strictly needs to. As the correction rate on a given field settles, that field's threshold can be relaxed. Fields with legal or financial consequence often stay gated permanently, and that is a legitimate end state rather than a failure to fully automate.

Retrieval over the archive - answering questions across processed documents rather than one at a time - is worth adding only once extraction and validation are stable. Retrieval over an unreliable archive produces answers that are fluent, sourced, and wrong.