Beyond the text layer.

Converted, the text travels and the pictures do not, so anything said about a drawing is generated rather than extracted — a claim, not an extraction. These notes set out what would have to be true before such a claim reaches a model, what would count as a result, and who decides. Every direction below is open, and one thread runs through them: the conversion record — including the hash this engine does not take.

§1What the model was shown, six months later

An organisation runs a supplier's drawing pack through a document pipeline, a model reads the result, and a compliance summary quotes a tolerance from it. Six months later the part fails, and someone who owes the organisation nothing asks what the model was shown. Not the drawing — the bytes. Where nothing was recorded, the honest answer is a filename and a date, which is a claim.

Conversion is an event, and it can be recorded like any other. What exists today is more than nothing and less than a record: the sha256 of the source bytes, the converter and its risk rung where one ran, the OCR provider, its egress and a status per page — and no page text, since an audit record carrying the document defeats its purpose.

Three things are absent. No engine-version field appears in the documented response; nothing points a derived statement back to the pixels; and the output side is unhashed — the bytes that went in are fingerprinted, the Markdown that came out is not.

§2OCR reads the glyphs, not the picture

The text a reader sees is one layer of a PDF; figures, scanned annexes, drawings, charts and photographs sit outside it. A picture becomes a figure block holding its position, carrying a caption only if one was printed beside it, never one the engine wrote. A page uploaded natively arrives as an image and its text, which is what lets a reader ask the model about its charts;^[1] the converted page cannot be asked, and that is what conversion gives up.

OCR does not close the gap, because it reads glyphs rather than pictures: over a schematic it recovers the labels and leaves the schematic. The provider is handed a raster rather than the document — rasters in, lines out — and each line is capped at inferred, because certain means the format said so. The difficulty is not local to one engine: a 2026 benchmark on engineering drawings reports models reading a drawing's text far more accurately than its symbols, and calls symbol-centric understanding unsolved^[2] — the authors' word, on their own preprint.

A merged cell has no representation in the Markdown emitted^[3] — Paper 001 §7 gives the specification — so the cell is flattened and the table named in a warning. The tolerances citable from a drawing are those that were in a table, and not all of even those.

// the two sentences that came out newly false
"The longest thing in your estate goes in whole."
"…the tolerances you can cite are the ones that were in a table,
   and they are all of them."

// what the checks returned
type checker · test suite · claims diff → PASS

// what was true, one panel away on the same page
the window decides what fits, and intake caps at 2,000 pages
a merged cell is flattened, so a table can lose a tolerance

Absolutes arrive during positive rewrites, and the strongest sentence in a rewritten section is the likeliest to have overshot.

§3A picture is a claim waiting to be made

Knowing what is in a picture is several jobs wearing one name: reading values off a chart, recovering the instruments and connections of a piping and instrumentation diagram, following a floor plan or an assembly drawing, naming what a photograph of a site shows. They are one kind of thing — every one of them is generated — and they differ sharply in difficulty and in whether anyone can score them.

Extraction is deterministic and checkable; description is generated and is a claim. The same bytes give the same Markdown — asserted in the engine's documentation rather than demonstrated by a published corpus, as §7 records — so a doubter can convert the file again and diff. A description cannot be re-run that way: a second run may write another sentence, and either may assert what is not in the picture. A 2018 study found captioning models "prone to 'hallucinating' objects that are not actually in a scene", and a 2023 study finds the current model class inherits it.^[4]

One vendor's documentation instructs that image interpretations be verified.^[5] So a description reaching the model is labelled as derived, kept below certain — OCR lines are capped at inferred, and a generated sentence has no claim to sit higher — and carries a way back: page, bounding box and the sha256 of the raster. It exists only where the pipeline owns the rasterisation: a natively uploaded PDF is rasterised server-side "at dimensions you don't control", so the coordinates that come back cannot be reliably mapped onto the page. A caption so recorded is locatable, not true.

A derived statement needs its provenance committed with it, and committed injectively: caption, page, box and hash run together by plain concatenation let whoever writes the record move the boundary between them — the defect Paper 002 of The Project Handshake found in its own leaf encoding. A verifier checks the box against the raster, not the sentence against the picture.

§4What is being researched, and what would count as a result

The direction under research is an enhancement tier over the deterministic baseline, and none of it is built: the engine accepts the option today and ignores it. It would act only on blocks already flagged as needing help — figure alt-text, chart-to-table, reconstruction of flattened merges, semantic chunk boundaries — marking each touched block enhanced: true, returning the deterministic output byte-identical when the tier is off, aborting to it at a cost ceiling, and routing model calls through a gateway with an evidence log.

The difficulty is evaluation, and it is uneven:

A piping and instrumentation diagram is drawn in a vocabulary — ANSI/ISA-5.1-2024 establishes "a uniform means of designating instruments and instrumentation systems used for measurement and control"^[8] — and a vocabulary invites structured output rather than prose: node and edge lists, tag-to-value statements, each carrying its box and raster hash. A result would be a measurable margin over the baseline on a corpus with ground truth, and for an estate's own drawings no such corpus exists: the published benchmarks are other people's documents, and annotating an estate's own is work that would have to be paid for before anything could be scored. Marked open, all of it.

§5Where the Markdown should live

Today the engine returns Markdown and a structure map and stores nothing, a decision rather than an omission: the moment an engine embeds, it needs a model, a store and a network, and the property being protected — that documents stay inside the operator's perimeter and jurisdiction — is gone.

So the question belongs to the corpus owner: a document store, one record per source with sections addressable by heading path; a vector index, chunks mapped to embeddings; or both. They are the same kind of thing — derived indexes over a canonical text — since an index adds no information and dropping it deletes nothing, because it rebuilds.^[9] A vector index is derived twice over, the boundaries a pipeline decision and the embedding a model's output.

The record is the canonical Markdown and a hash over it. Every index above that pair is a derived structure that can be dropped and rebuilt from it. The discipline that follows is exact: whatever an index returns must be re-findable in the record, or it has left the chain — and a store that cannot name the bytes its answer came from has quietly become the record itself. A direction, not a decision, taken jointly by those who own the corpus.

§6Repetition is the compressible part

Estates repeat themselves — running headers and footers, boilerplate clauses, recurring tables, template pages. Inside one PDF the engine already strips headers, footers and page numbers, and because that is the one pass that deletes text, dropped lines are warned. Across documents it does not look.

Content-defined chunking sets block boundaries from the bytes rather than from position, so a repeated block gets the same boundaries wherever it sits; content addressing makes a block's identifier the hash of its contents, so duplicates coalesce and corruption shows on read.^[10] A repeated clause is then stored once and referenced by hash — cacheable, and more to the point checkable. The chunk sizes and window widths in that work were tuned for source trees over slow links, so no compression figure appears here: none has been measured on a Markdown corpus.

Prompt caching has the same shape and not the same lifetime: a hit requires byte-identical segments, and the default is five minutes, an hour at additional cost.^[11] A cache is a cost instrument with a horizon in minutes; an archive is a record. What repetition invites is an archive format for Markdown corpora: dictionary compression across documents, block-level deduplication, an index. The dictionary format RFC 8878 specifies travels out of band, identified by an ID the decoder checks, and that specification leaves its provisioning "being explored".^[12]

§7Properties, not promises

Egress declared as a field and written onto the conversion is a property; "documents stay in Europe" is a promise. A policy says where pages are allowed to go, a record says where they went, and the two are indistinguishable while nothing goes wrong.

• Egress as a recorded fact on the conversion, with a status for every page. A declared field, checked before a provider authenticates and written onto the result: enforcement of a declaration, not proof of one. Paper 001 §6 gives the rungs.
• Determinism. The same bytes give the same Markdown — asserted in documentation rather than demonstrated by a published corpus, the property tests that would pin it planned rather than run, and table column alignment dropped on emission, denting idempotence.
• Measurement with the counter the claim is about. Stated against native document upload on one vendor's models; on a vendor billing a flat rate per page the sign reverses. Paper 001 carries the figures and the spread.
• Estate economics on CPUs inside the building. CPU-bound and per core, with no GPU in the path.

The fifth property is the record itself, and its limit belongs in the same breath. Paper 004 of The Project Handshake gives the general form: a record shows what was done, not that it was right. A conversion record shows that a page went to a local binary and how confident that binary reported itself to be, not whether the lines it returned are the lines on the page.

// what a conversion record carries today · fields present where each step ran
meta.source_sha256    60b30efa…d814c89   // rfc9110.pdf, 194 pages
structure.converted   {converter, converter_version, risk, endpoint}
structure.ocr         {provider, provider_version, egress, endpoint, pages.*}
                      // page text deliberately absent
blocks[*].confidence  1.0 certain · 0.75 inferred · 0.45 guessed · 0.2 salvaged
blocks[*].enhanced    // false on every block: the tier does not exist

// needed, and not there
a hash of the Markdown that came out
the engine release, which the documented response has no field for
per derived block: page · bounding box · raster sha256
an open schema anyone can implement and check

Those last four lines are the proposal. What does not exist is an agreed shape: an open schema for conversion records — the fields, the hashes, how a derived block points back to its raster — is the kind of open standard the Foundation being established is meant to steward. These notes propose only that it should exist.

§8An honest ledger of what is open

Collected here, because a reader deciding what to demand is entitled to them together, each with what would settle it and who decides:

1. Whether a generated description can be kept below certain and still be worth having. Settled by a tier shipping with every touched block marked and capped, falsified if the cap is lifted. The engine's owners decide.
2. Whether chart-to-table and diagram-to-graph can be scored on an estate's own documents. Settled by ground truth that does not exist yet, and a margin over the baseline. The estate owner decides, and pays for the annotation.
3. Where the canonical Markdown lives, and what is derived from it. Falsified by an index whose results cannot be re-found in the record. The corpus owner decides.
4. Whether chunking and dictionary compression pay on Markdown corpora. Settled by measurement on a named corpus, of which no figure exists, and falsified if body repetition proves smaller than the furniture already stripped. Whoever runs it decides.
5. Whether a declared egress can be enforced below the engine. Settled at the operating-system boundary, where a service unit can deny a converter the network; the engine declares and records, and cannot prove. The operator decides.
6. Whether an open schema can be written so that two implementations agree. Settled by two independent implementations checking each other's records, and held by a body that owns no product — the Foundation being established, if those establishing it so decide.

Nothing above is committed: the decisions are joint, and these notes take none of them.