Synthetic Industry

Buyer collection · updated 2026-10-12

For a bookkeeper retyping supplier invoices: get a batch into rows, with the doubtful ones listed

Measure whole invoices, not fields. Check the arithmetic on every invoice, keep an exceptions list, and test on invoices you labelled and the tool has not seen.

Count whole invoices

A tool can be 92% right per field and still get far fewer invoices right in full. In one small public test, three set-ups scored the same pooled field accuracy of 91.7% twice, yet 60% and 75% of invoices were fully right. Ask for the whole-invoice rate beside the field rate.

Keep an exceptions list

Every invoice should end up as rows or as an entry on a list with a reason. Nothing should vanish. If the files in do not equal the invoices out plus the exceptions, something was dropped.

  • The number of files in equals the invoices out plus the exceptions.
  • Lines plus tax equal the total, flagged when they do not.
  • A missing field comes back empty, not as a plausible value.

Expect most trouble outside the reading step

One benchmark summary says most exceptions are purchase-order mismatches and vendor-master problems, not extraction. Better reading will not clear those. They need a person who knows the supplier.

Which job fits

For one batch in up to 40 layouts, the one-off job builds the reading step and tests it on 30 invoices you label and we have not seen. For every month's batch, the monthly service repeats the checks. Staff expense receipts have their own job, because they read less well and carry card digits; pairing receipts with a card statement is a separate finance job.

Sources and limits

  • GitHub: LLM-invoice-extraction Checked 2026-10-12.
    • A public test of 60 synthetic invoices (40 test) found whole-invoice correctness of 60.0%, 75.0% and 87.5% for three prompt strategies while pooled field accuracy was 91.7%, 91.7% and 99.0%; it used one model and the author notes wide confidence intervals.
  • Parseur: AI invoice processing benchmarks Checked 2026-10-12.
    • Reports a top model at 96.50% on clean digital invoices, 92.71% on scanned invoices and 87.46% on receipts, and says most exceptions are not extraction failures but purchase-order mismatches and vendor master data issues.