documents.md

Documents and ingest

A workspace holds a library of sources: uploaded files, notes written in the app, and the bodies of Studio artifacts. Every source ends up the same way, as a markdown body on its documents row cut into passages that are embedded and indexed, so chat and search treat them alike. The API only stores bytes and enqueues a job; parsing, chunking and embedding happen in the ingest worker, one document at a time, and a document is ready only once it is searchable. Ingest also keeps a file's pictures beside its original, as figures the agent can place in a document, and turns a recording into a timestamped transcript through the speech model (Recordings). Studio indexes an artifact's body itself, in its own worker, with the same chunking and embedding code.

Code: modules/workspaces/, modules/documents/, worker/ingestion/, worker/audio_decode/, worker/jobs.py, shared/queue.py, frontend/src/features/sources/ Decisions: ADR 0003, ADR 0007, ADR 0036, ADR 0058, ADR 0059, ADR 0008, ADR 0009

Workspaces

Method	Path	Does
POST	/workspaces	create; 201
GET	/workspaces	list, oldest first
GET	/workspaces/{workspace_id}	read one
PATCH	/workspaces/{workspace_id}	rename
DELETE	/workspaces/{workspace_id}	delete it and everything in it; 204

• The API seeds "My Workspace" at startup when no workspace exists (seed.py), so a first launch always has one to open into.
• Every workspace has a Library: a managed source root and its root folder, made when the workspace is created, by 0025 for older ones, and on first use by any folder route (managed_root.py).
• A name is trimmed and must be 1 to 200 characters, so a name of spaces fails instead of rendering blank.
• Delete cascades the workspace's documents (with their chunks and index entries), threads, messages and artifacts, then removes data/workspaces/<id>/ after the commit, which a rollback would undo.
• Every route with a workspace in its path resolves it first and answers 404 for an unknown id. A document is looked up within the workspace in its path, so one workspace's id cannot reach another's document.

Document routes

Method	Path	Does
GET	/workspaces/{workspace_id}/documents	list (below)
POST	/workspaces/{workspace_id}/documents	write a note, with an optional document_metadata object and folder_id; 201 with its body, 422 for a field it does not know
POST	/workspaces/{workspace_id}/documents/upload	upload files, with optional folder_id and relative_paths form fields; 201 with created, duplicates and rejected
POST	/workspaces/{workspace_id}/documents/move	{document_ids ≤ 1000, folder_id}: file sources in a folder; 200 with moved and skipped (Folders)
GET	/workspaces/{workspace_id}/documents/{document_id}	one document with its content and document_metadata; content is null for a FILE the worker has not written yet
PATCH	/workspaces/{workspace_id}/documents/{document_id}	rename; edit a note's content
DELETE	/workspaces/{workspace_id}/documents/{document_id}	delete; 409 while processing
POST	/workspaces/{workspace_id}/documents/{document_id}/retry	requeue a failed or cancelled document
POST	/workspaces/{workspace_id}/documents/{document_id}/cancel	stop a pending or processing ingest
GET	/workspaces/{workspace_id}/documents/{document_id}/original	the uploaded file, as an attachment
GET	/workspaces/{workspace_id}/documents/by-chunk/{chunk_id}	a cited chunk and its neighbours, for the citation panel (chat.md)

Chunks have no router of their own.

Listing

• Filters are ?document_type= and ?status=, both repeatable, and ?folder_id=. Paging is ?limit= (default 50, at most 200) and ?offset=; the sources panel reads pages of 200 until one comes short, so it holds every source, not the newest 50. Rows come newest first, by created_at and then id, since one upload batch shares a second, so a fresh upload leads the sources list while it is still indexing.
• ARTIFACT rows are included and the caller filters them out. The sources panel asks for document_type=FILE&document_type=NOTE, so Studio output does not look like something the user uploaded.
• A row carries id, title, document_type, mime_type, status, error_message, folder_id, timestamps and transcription, but no content. The list is polled while ingest runs, and a body per row would ride along on every poll; GET .../documents/{document_id} returns one row with it.
• transcription is null for everything but a recording (mime_type audio/*), which carries {language, progress, awaiting}: the language its transcript was heard in once it is done; {done_s, total_s} read from the job's progress.json while it is pending or processing, so a reload or a late listener sees how far it has come; and whether it failed only for want of a speech model (Recordings).

What is editable

• Title, always: 1 to 500 characters after trimming.
• Content, only for a NOTE. Anything else was extracted from bytes or generated, and an edit would vanish at the next ingest, so PATCH answers 409. This is also the server-side read-only guard ADR 0003 asks for on artifact bodies; renaming and deleting an artifact stay allowed.
• Editing a note's content puts it back to pending and enqueues ingest, because until the worker rebuilds the index, search would keep returning the old text.
• In the sources panel, every row's menu has Rename, which sends title alone so a file is never sent a content it would refuse, and a note's row also has Edit note. Both forms check the title's 1 to 500 characters before sending; the server's answer stays authoritative.

Notes

A note is a document the user writes, with no file behind it. Creating one commits the row as pending and enqueues ingest: nothing needs parsing, but the note is not ready until it is chunked and indexed. A note never touches the filesystem and carries no dedup key, so two notes with the same text are two documents. + › New note in the sources panel header writes one; Edit note reopens it with GET /workspaces/{id}/documents/{doc} and saves with PATCH, and the edited row comes back pending, so the panel's ingestion poll carries it to ready like any upload.

A note may carry document_metadata, stored as given. A plugin names itself there on the notes it adds, with plugin_id, plugin_version, action and run_id; a note the user writes has none, and editing a note leaves it alone. Writing a note with a field the API does not know is refused rather than dropped, so a client that names a field differently finds out at once.

The sources tree

The sources panel shows the Library as a tree (features/sources/tree/). It reads every folder with GET /folders and every file and note page by page, 200 at a time, and builds the tree on the client; the Library's own folder is never a row, so a workspace with no folders is the flat list it always was.

• Rows. Folders come first, by name, then sources newest first. The list is an ARIA tree with one treeitem per row: arrow keys move and open, Space ticks, Enter opens, F2 renames and Delete asks to delete. A name filter, opened from the search button in the header, shows matches with the folders holding them opened; Escape or closing it clears it. A name the column cuts off shows whole in a tooltip on hover, or while its row has keyboard focus (overflow-tooltip.tsx). Rows are memoized, and a re-read of the list keeps the object of every unchanged source (reuse-unchanged.ts), so a poll or an event renders only the rows whose source changed.
• Actions. New folder (from the header's + menu, or inside a folder from its menu), Rename, Move to… and Delete. Move to… is a picker of folders and the keyboard's path; a row dragged onto a folder, or onto a source inside one, moves there too. A folder cannot be moved inside itself. Deleting a folder names how many sources, and filed Studio outputs, go with it, as GET /folders/{folder_id}/summary counts them, and deletes them for good.
• Recordings. A recording, an audio/* source, carries transcription on its row, and its row has a second line (transcription/). While it transcribes, "Transcribing · 23 of 58 min · about 7 min left" over a thin bar: the minutes are transcription.progress, and the time left is the speed of the windows the panel has heard of since this run started; until two have arrived there is no estimate. A recording that made way for other sources reads "Queued · 40% transcribed". One waiting for a speech recognition model reads "Waiting for speech recognition" with Set up, which opens the speech set-up popover (catalog), and shows no error and no Retry: choosing a model queues it again, and the popover stays open until it is closed, so it can say speech recognition is ready. A ready transcript in a language the chosen speech model detected but is not listed for (GET /llm/selection/speech_to_text/language) says "Estonian (detected) · the transcript may be less accurate"; with one curated speech model, there is no other to offer. A ready transcript that is not in English, or whose language was not identified, says search finds it best with questions in its own language and that a chat reads all of it, while the active index's embedder is bge-small. The frontend knows bge-small by its id, bge-small-en-v1.5, since no route says which languages an embedder reads. A recording has no preview: viewers are keyed by MIME type, and none takes audio/* yet.
• Ticks. Each row's tick is tri-state. Ticking a folder ticks everything under it, now and added later; the nearest tick up the tree decides (scope-state.ts). Chat and Studio send the ticks as source_scope, so every ticked source counts, not only the ones loaded.

Upload

• The request is multipart, one files part per file. Each file streams to a temporary file inside the workspace directory in 1 MB reads, hashed with SHA-256 on the way past and never held whole in memory. A file over 500 MB is refused with 413. The temporary file sits on the same filesystem as its destination, because a rename is only atomic within one filesystem.
• Only a plain extension, a dot and 1 to 16 letters or digits, is validated from the client's filename. The file is stored alone in a directory built from row ids, data/workspaces/<workspace>/documents/<document>/, under the client's filename made safe for every platform (original_file.py): the last path part only, with pathvalidate removing characters no filesystem accepts and suffixing Windows reserved names, leading dots dropped so the file is never hidden, cut before the extension so the whole name fits in 120 bytes, which keeps a typical Windows path under its 260-character limit, and ending in the validated extension lowercased. A name with nothing left becomes untitled, and one that would be extracted.md becomes extracted (1).md (see The original file). The name reaches the disk; the path never does.
• Accepted extensions are .pdf, .docx, .pptx, .xlsx, .html, .htm, .csv, .md, .markdown, .txt, .text, .png, .jpg, .jpeg, .tif, .tiff, .bmp and .webp, and the recordings .wav, .mp3, .flac, .ogg and .oga. The bytes must match the extension: the PDF signature; an OOXML zip holding its main part, with at most 10,000 entries and 2 GB unpacked; UTF-8 text with no NUL and few control bytes; an image's magic number; or a recording's: RIFF with a WAVE form, an ID3 tag or an MPEG audio frame header (not AAC's), fLaC, or an Ogg page whose first packet is Vorbis's. An Ogg Opus file is rejected on its own with "Opus support comes in a later update", since the decoder reads Vorbis only. The MIME type stored is the server's, never the client's: audio/wav, audio/mpeg, audio/flac and audio/ogg for recordings. The frontend mirrors the list to filter its file picker and to skip unsupported files, dropped ones included, before uploading.
• The sources panel takes files two ways: its + menu, and files dropped onto the panel, which shows "Drop files to add them as sources" while files are dragged over it (use-sources-drop.ts). Both go through the same upload. Dropped files and rows dragged between folders go through Pragmatic drag and drop, which also scrolls the list while a drag nears its top or bottom, and draws a dragged row as a small chip. The panel takes no drop while an upload runs, as uploading is disabled then, and a drag carrying no files, such as selected text, does not light it up. A file dropped anywhere else in the window is refused (file-drop-guard.ts): unhandled, Chromium opens it in the window, and in Electron that replaces the app, since the main process lets file: navigation through.
• A whole folder comes in two ways: + › Upload folder…, an <input webkitdirectory> whose files carry webkitRelativePath, and a dropped folder, which the panel walks with webkitGetAsEntry() (dropped-files.ts). Either sends each file's path under the folder as relative_paths, and a drop on a folder row, or on a source inside one, adds folder_id. Before sending, the client skips paths through .git, node_modules, __pycache__, .DS_Store, .obsidian and .trash, sets aside unsupported files and any file over 500 MB, which it lists as rejected rather than letting one fail its request with 413, and sends the rest in requests of at most 50 files or 200 MB (upload-plan.ts).
• Files go into the folder_id form field's folder, else the top of the Library. relative_paths, a JSON list index for index with files, carries an added folder's tree: each file lands in the folders its path names below the target (Research/2024/a.pdf makes Research and 2024), made as needed with ensure_folder_path. Levels past the depth cap are joined with " / " into the deepest level there is room for, so a deep tree arrives whole. A list of the wrong length is 422; a folder from another workspace is 404.
• dedup_key is the SHA-256 of the bytes and is unique per folder. Keying on the bytes rather than the filename means the same report saved twice in one folder is one document, while two different files both called report.pdf are two. A file whose bytes sit in another folder is created in the new one too, so a copied folder has no holes, and copying it again adds only what is new.
• A batch is split, not rejected, except that a file over 500 MB fails the whole request with 413 and nothing in it is created. The response lists created, duplicates (each with the existing document's id and its folder_id) and rejected (each with a reason), so a dropped folder holding one known file keeps the rest.
• An upload of more than 20 files enqueues at PRIORITY_BULK (10); every other ingest runs at PRIORITY_INTERACTIVE (100), so a note written during a long folder copy is read first (tasks.py).
• Accepted files become FILE rows in pending, with mime_type, size_bytes and suffix in document_metadata. A recording is accepted with or without a speech model chosen; with none, its ingest waits for one (Recordings). The 500 MB cap holds for recordings too, with no cap on length: at 64 kbps that is about 18 hours. The files move into place, the transaction commits, and only then is one ingest job enqueued per file, because the worker is another process and would look for rows this request had not yet written.
• The jobs are enqueued cheapest first (upload_order.py): text files (.md, .markdown, .txt, .text, read straight off disk), then every other file by size, smallest first, then recordings by size; ties keep upload order. Equal priorities run first in, first out (The queue), so this is the order a dropped batch becomes ready in. It holds within one request, and the frontend sends at most 50 files or 200 MB in each. On two PDFs and a 4.8 KB .md at FP16, the first source was ready after 4.2 s instead of 35.0, and the three after 29.2 s on average instead of 50.4. Size only stands in for the work, so a small file that parses slowly can still hold larger ones behind it.

Folders

Sources sit in folders of the workspace's Library (data model, proposal). Routes are under /workspaces/{workspace_id} (modules/folders/):

Method	Path	Does
GET	/folders	every live folder, flat, the Library's own folder (no parent) first, then by folded name
POST	/folders	{parent_id?, name, role?}; no parent means the top of the Library; 201
PATCH	/folders/{folder_id}	{name?, parent_id?, role?}: rename, move, set or clear the role
GET	/folders/{folder_id}/summary	{folders, sources, artifacts} below it, for the delete dialog
DELETE	/folders/{folder_id}	delete it, its subfolders and every source in them, for good; 204
POST	/folders/{folder_id}/cancel	stop every queued or running read below it; {cancelled}

• A name is trimmed, 1 to 255 characters, and unique among its siblings by name_key, the name in NFC and case-folded in Python, since SQLite's NOCASE folds ASCII only: "Été" beside "ÉTÉ" is 409.
• Library folders go at most 8 levels below the Library's own folder; a create or move past that is 400, and so is a move into the folder itself or below it.
• The Library's own folder cannot be renamed, moved or deleted (409). A folder of another workspace, or one being deleted, is 404.
• Delete answers 409 while any source below is processing. Otherwise it marks the subtree deleting, which takes it out of every scope at once, and cancels its queued reads in the same transaction, so none starts and outlives its batch. It then removes the documents in batches of 500, each its own transaction with its bytes removed after the commit, then the folders. A delete the app stopped partway through is finished on a thread when the API next starts, so start-up does not wait on it; a source still processing there keeps its subtree deleting, out of every scope, until a later start. There is no Trash yet.
• POST /documents/move refuses an id from another workspace (422) and skips a source whose bytes already sit in the target folder, reported as {document_id, reason: "duplicate", duplicate_of}.
• Every change sends a folders event with the folder ids, an upload that made folders from its paths included; a move, a cancel and a delete also send a documents event with the document ids.
• Every constructor files a source: a note and an upload go to the folder named or the top of the Library; an import rebuilds the bundle's folder path under the Library; an artifact starts unfiled.

Retry, cancel and delete

• Retry accepts a failed or cancelled document: it sets pending, clears error_message and enqueues. Without it those states would be terminal, since re-uploading the same bytes is a duplicate.
• Cancel accepts a pending or processing document and answers 409 otherwise. It marks the row cancelled, clears error_message and revokes any queued copy of the job (revoke_pending). A running job is not killed: the pipeline checks between steps and unwinds at the next one, and a long Docling reading between batches of 8 pages, so the batch in flight finishes first (Long Docling readings).
• Delete refuses a processing document with 409. Otherwise the row goes, its chunks cascade, the triggers clear both index tables, and the document's directory is removed after the commit; removing it first would leave a row describing a missing file if the transaction rolled back.
• A ready source can be deleted while its figures-only pass reads it, and on Windows the delete cannot remove the file Docling holds open. documents has no AUTOINCREMENT, so SQLite gives the highest id out again once its row is gone, and the next document would find that file in its directory. So an upload or an import empties its new document's directory before moving the file in (original_file.py), and the pass removes what the delete left once Docling lets go (Figures).
• Delete also accepts an ARTIFACT document, and removes the artifact's files under artifacts/<id>/ as well as documents/<id>/, since a Studio output keeps its rendered blobs under the artifact's own id rather than the document's. DELETE /artifacts/{id} removes the same two (studio.md).

The original file

No filename is stored in a row: the document's directory is the record. The original is the one file in it, so not the figures/ folder, a recording's transcript/, a long Docling reading's docling/ or a long OCR reading's ocr/, that is not hidden, not Thumbs.db or desktop.ini, and not extracted.md. Any other count, an empty or absent directory included, means the file is missing. Directories written before uploads kept their names hold original.<ext> beside an extracted.md nothing reads. They are read as they are and deliberately never rewritten: nothing in the index depends on a file's name, so renaming would only change what the file manager shows, and a bulk rename can fail part way on a file that is open or already deleted. Because extracted.md is never taken for the original, a legacy directory whose original was deleted reads as missing rather than serving Docling's text as the user's file.

GET .../original serves the stored bytes with the document's title as the filename and the stored MIME type, always as an attachment: an HTML or SVG file rendered inline would run its script against the app's origin. It answers 404 when there is no file behind the document, as for a note.

The desktop app previews an original in place of the left sidebar when its MIME type has a source viewer, starting with PDF. The sidebar widens to the detail width while it shows, or stays at the user's own width if that is wider, as the right panel does for an inspected artifact, and the right panel stays as the user left it. Dragging the preview's edge resizes the preview alone, and that width is remembered apart from the sidebar's (column-widths.ts). Other file types still open natively: through the preload bridge, Electron applies the same rule to the directory under the data directory for the given ids, since the main process never calls the backend, and hands it to the operating system, or reveals it in the file manager (document-files.ts). The DocumentRead row carries mime_type from stored metadata only for FILE; notes always report null, even if their user-provided metadata contains a mime_type. The preview fetches the existing /original route, which remains an attachment so directly navigating to HTML or SVG cannot execute it on the app's origin.

Ingest pipeline

ingest_document(document_id) is declared in modules/documents/tasks.py, because Huey binds a task to its queue when it is decorated and the API is what enqueues it. The task body imports the pipeline lazily and Docling is imported only when the converter is first built, so the API never loads Docling or torch. Each job opens its own database engine.

1. Start. begin_job marks the row processing unless it was cancelled, and the worker notifies the API. The job then reads the active embedding index; with none, because onboarding has not chosen an embedder, it fails with no embedding model has been chosen yet.
2. Parse (parsing.py). A NOTE or ARTIFACT already has its markdown in content. A recording, a FILE whose mime_type is audio/*, is transcribed before Docling is reached, which must never read one (Recordings). A FILE with a text extension (.md, .markdown, .txt, .text) is read as UTF-8. A .pdf is read from its own pdfium text layer when every page passes the triage, and Docling reads only the pages that fail it when they are at most half the file (PDFs with a text layer); a scanned page and an uploaded image are read by OCR alone, and Docling reads only a page OCR could not (Scans and photos); a .docx is read without Docling (below); anything else goes to Docling, a PDF longer than 8 pages a batch of pages at a time (Long Docling readings). A .pptx with no slides, such as a template, skips Docling, which refuses a deck with no pages: its text is PowerPoint template with no slides., its slide size and its slide layout names in order, one line each, read with python-pptx (slideless_deck.py), so search finds it and the agent can start a deck from it as a template (studio). A .docx is read as markdown with python-docx and lxml (word_markdown/): 0.13 s for the longest reference contract, which Docling reads in 5.1 s warm, and 0.05 s for a fresh worker's first Word file, which costs 4.4 s through Docling, its import mostly. The body comes in order: headings as Docling decides them (a style id or name with "heading", or the style's own w:outlineLvl; Title is #, Heading N is N+1 #s), list items and numbered headings with the numbers Word renders from numbering.xml, tables one compact line per row with a merged cell's text once and a nested table flattened into its cell, links as markdown links to their address, a space in it as %20 and an unmatched parenthesis escaped, tabs and breaks as spaces and line breaks, tracked changes as if accepted, a deleted table row with them, mc:AlternateContent as its first choice and ruby as its base text, then footnotes and endnotes as [^1]: …. Headers, footers and comments are left out, as Docling's markdown leaves them. The numbers agree with Word 16's ListString on all 1,876 numbered paragraphs of 41 reference files, where Docling's agree on 31%: it restarts each w:num, ignores startOverride and numbering a style inherits along w:basedOn, and writes letters and roman numerals as decimals. test_word_numbering.py rebuilds a fixture for each rule and expects the ListString Word gave. A counter and a w:lvlText come from the file, so a letter, numeral or dagger repeated past 32 times is written in decimal, a start past a 32-bit integer is ignored, and a marker is cut at 255 characters; the longest in the reference files is 10. On the 791 reference Word files Docling reads, the reader keeps a median 100% of Docling's words, and 100% at the 5th percentile, with numbers compared apart and a merged cell counted once. SURFSENSE_LOCAL_DOCX_READER=docling turns it off. A file it cannot read, one with native charts, whose data Docling writes as tables, and one where it finds no text go to Docling as before, so a blank Word or Excel file is still ready with no text. While Docling reads a .docx, for ingest and the figures-only pass alike, python-docx's style lookups are remembered for that one conversion, per document part and per style id and type (docx_style_cache.py): Docling asks for a paragraph's style once per run, and python-docx scans every style in the file for each answer, which was 80% of reading a long contract, 25.8 s against 5.4 s with the cache, idle. The Markdown and the whole DoclingDocument came out identical on 804 Word files; a file that defines few styles gains little. Only the converting thread reads the cache, and python-docx's own get_style is put back once the last conversion running ends, failed or not, so revise_document and Studio's writers never see a remembered style. OCR and table structure are switched on, so a scanned PDF is readable and a table survives as a table. RapidOCR's per-line orientation classifier is off: on noisy scans it turned upright lines 180 degrees and they read back as garbage, and dropping it cut the character error rate on generated scans about eightfold. An image is first turned upright by its EXIF orientation, which Docling ignores, and given a DPI that reads it as one A4 page, never finer than the 216 dpi OCR renders at nor coarser than its own DPI (image_page.py); with none, Docling takes it as 72 dpi and OCR upsampled a photo threefold. Images and PDFs share one set of options, so they share one pipeline and its models. Inference gets one thread per physical core, at most 8, as audio.cpp does, instead of Docling's 4: on an 8-core CPU that parsed generated scans, photos and PDFs 1.4 times faster. The converter is built once per process and its models load on the first conversion, so the first document a worker gives Docling pays for the import and the load, 11 to 17 s on the measuring machine; a worker given only notes, text files, text-layer PDFs and Word files imports it only once a figures-only pass runs. HF_HOME defaults to the models directory, because Docling would otherwise write weights into site-packages, which is read-only in a frozen bundle. When the bundled parser pack is present Docling uses it with RapidOCR, and HF_HUB_OFFLINE=1 is set before Docling is imported, because huggingface_hub reads it only then, so a PDF or an image converts with networking off. The markdown is kept only in content. Beside the original, ingest writes only the pictures Docling found, as figures; a PDF read from its text layer, wholly or but for some pages, gets its figures from the figures-only pass, and a .docx read without Docling drops those an earlier reading kept and leaves its pictures to that pass.
3. Chunk (chunking.py). Chonkie's RecursiveChunker splits at the coarsest boundary that fits: a heading, a paragraph, a line, a sentence, a word, then a bare split so an over-long line still ends. Passages are at most 480 tokens by bge-small's tokenizer, under its 512-token limit with room for the two tokens it adds. That tokenizer is pinned to the chunker rather than taken from the active embedder, so changing the embedder would not re-cut documents and move every chunk id. Pieces under 24 characters are merged into a neighbour while splitting, so only a document shorter than that yields a shorter passage. The tokenizer is passed as an object, because one named by a string would reach the network even offline. Each passage records the lines its first and last characters fall on.
4. Embed (encoder.py). The active index's model, as its spec says, on onnxruntime's CPU provider: the document prefix, tokenize (truncated at the spec's max_tokens), pool, normalise, in batches of 32. For bge-small that is no prefix, 512 tokens, CLS pooling and L2 normalisation, and on x86-64 its FP16 file runs upcast to FP32 in memory, about six times as fast (embedding). No network and no model server. A vector whose width is not the spec's is refused rather than stored.
5. Index (indexing.py). The document's old chunks are deleted, which clears both index tables through the delete trigger, and the new chunks are inserted with their float32 embeddings, followed by one row per chunk in the active index's vector table, and the document records that index in embedding_index_id. The keyword index follows by trigger; the vector cannot, because only ingest holds it. Editing a note therefore stops its old text being findable.
6. Finish. finish_job writes ready unless a cancel won the race, and beside it only what the job produced: a file's extracted markdown on documents.content. A note's content is the user's and is only read, and ingest never writes title, so a rename or a note edit committed while the job ran stands, and the ingest that edit enqueued indexes the new text.

Cancellation is checked after parsing and after embedding, between a recording's windows, between a long Docling reading's batches and every 8 pages of an OCR reading. A recording or a long Docling or OCR reading that gives the queue up rolls back and goes back to pending (pause_job, unless a cancel won), queued again at PRIORITY_BULK for a recording and at the priority it came with for a reading: neither a failure nor a retry. On any other failure the worker rolls back and re-reads the row. If the document was deleted or cancelled meanwhile it stops; otherwise it writes failed with error_message set to the exception type and message, cut to 500 characters (a recording's failures are sentences written for the person, stored as they are), notifies, and re-raises so Huey runs the job again, up to two more times. A file whose bytes Docling cannot load (a ConversionError caused by a DocumentLoadError, such as a damaged Office file) is not run again, because the same bytes are refused the same way: parsing raises UnreadableFileError and the task turns it into Huey's CancelExecution(retry=False). Docling raises ConversionError too for a file it cannot open yet, such as one another program holds, and for a model that fails on every page, such as running out of memory; those are tried again. A recording that would fail the same way again is not run again either (Recordings). A later success clears the message. The sources panel shows it and offers Retry.

PDFs with a text layer

A born-digital PDF already holds its text, and pdfium reads it in milliseconds a page, where Docling takes about 3 s (ADR 0059). pdf_text_layer/ reads it with pypdfium2 while holding shared.pdfium.lock, since pdfium is not thread-safe. It decides page by page (pdf_reading/):

• Every page passes: the file is its text layer.

• Some pages fail, at most half of them: Docling reads only the runs of consecutive failed pages, each run as one conversion with page_range and the usual options (a run longer than 8 pages in batches, below), and the text layer the rest. A run's Markdown takes its pages' place, a blank line apart, as the text layer's pages are (spliced_text.py). Such a PDF keeps no figures at ingest, as a text-read one. The log names the pages and the first three reasons, as Docling reads pages 5, 9 of mixed.pdf: page 5 is ocr:no-text; page 9 is turned 90 degrees.

• Docling reads the whole file, as before and keeping its figures as it reads, when more than half the pages fail (a scan) and OCR-first is off or reads no page as text (Scans and photos), when a form field holds an answer, when pdfium cannot read the file, and when no page that passes is ok:text, since covers and stamps alone leave the content in pixels (docling_pages.py). It logs Docling reads <file>: <reason>.

• Forms. Before any page is read, a form field holding an answer sends the file to Docling. That is a text, combo or list field with a value that is not blank, or a ticked box. The answer lives in the field, which the text layer never shows, while Docling reads it off the page (filled_fields.py). A form nobody filled in is read from its text layer; 11 of the 19 Boston budget PDFs carry one.

• Turned pages. A page with a /Rotate fails, because pdfium returns a turned page's lines last to first.

• The triage. page_triage.py classes every other page by its text, its image objects and, where needed, its drawn path segments. It finds images inside nested forms too, maps each into page space, and clips it to the page's visible box, the media box within the crop box. The tests run in this order:

  Kind	The page	Read by
  ocr:no-text	under 20 non-space characters, over images covering more than 5% of it	Docling
  ok:blank	no characters, no image and at most 40 path segments; with an image or more segments it is ocr:no-text	text layer
  ocr:no-text	all its live text in the top or bottom 12% of the page box, over images covering more than 5% of it or more than 40 path segments. This catches a scan stamped with a header, a Bates number or a page number, and lettering drawn as outlines	Docling
  ok:sparse	under 20 non-space characters otherwise	text layer
  ocr-layer:scanner	at least half its text objects invisible, over images covering more than half the page: a scanner's OCR, which may be wrong	Docling
  ocr:garbled	more than max(6, 3% of its characters) that its fonts cannot map to Unicode, or more than that many meaningless ones (U+FFFD, private use, control codes), or under 30% letters and digits once dot leaders are set aside	Docling
  mixed:big-image	at least 20 characters over images covering more than 65% of it, such as a section cover with its title, where OCR of the photo added only noise	text layer
  ok:text	anything else	text layer

  A file whose pages all pass still goes to Docling when its text is empty, or when no page is ok:text, since covers, titles and stamps alone leave the real content in pixels. Any pdfium error also sends the file to Docling, with a warning.

• The text (reading.py). Each page's text is pdfium's get_text_range(), with line breaks as \n. Pages are joined with a blank line, the representation the A/B measured. Within a page, paragraph_breaks.py adds a blank line where a paragraph ends, so the chunker cuts there and not at an arbitrary line, as it once cut a paper's Table 1 from its caption and header. Each line's baseline comes from pdfium's character origins, under the same lock: a line starts a paragraph when the gap above it is more than 1.3 times the wider line spacing on either side of it (Word's 8 pt after a paragraph and LaTeX's parskip measured 1.49 to 1.5), so a table's evenly spaced rows and the rule under its header stay one block, and when it jumps up to the next column. A caption line (Table 1., Figure 2:, Exhibit 3 –) goes with the nearer of the blocks around it, the table under it in a paper or the one above it in a spec. A line on its own, such as a heading, goes with the block after it. Only \n is ever added: the text is otherwise the A/B's, character for character. Where a word is hyphenated at a line's end, pdfium removes the line break and leaves U+FFFE where the hyphen was. printable_text.py joins the word again when a lowercase letter follows, and writes - otherwise. It also drops C0 control codes, which python-docx refuses in a draft, after the triage has counted them. The text has no Markdown headings and no pipe tables. A table is one row to a line with its values in order, and an empty cell leaves no trace. Two- and three-column pages read down each column in turn.

• Figures. Ingest keeps none for a PDF read from its text layer, wholly or but for some pages; the figures-only pass does (Figures).

• The switch. SURFSENSE_LOCAL_PDF_TEXT_LAYER is read for each PDF (switch.py) and is on by default. off, or any false value pydantic reads, such as 0 or no, in any case, sends every PDF to Docling as before. A value that is neither true nor false logs a warning and means off.

• Measured on the Ryzen 7 5800X with the embedder at FP32. Each file went through the upload route and the pipeline to ready, one at a time, in fresh processes with Docling warm:

  File	Docling	Text layer	CPU-s, Docling / text layer	Passages, Docling / text layer
  Boston Executive Summary, 12 pp	16.7 s	1.4 s	176 / 10	13 / 11
  Boston Operating Budget, 46 pp	125.6 s	9.0 s (7.1 s on a repeat)	1,327 / 63	93 / 62
  hermes spec, LaTeX, 33 pp	73.1 s	3.8 s	863 / 29	51 / 35

  Reading and triage average 7.1 ms a page; Finance's 85 pages take 0.65 s. Paragraph breaks add 0.36 ms a page on the 19 Boston PDFs and 0.29 ms on the gate's second corpus, 1 ms at most; the passage counts above are with them. They packed short paragraphs across pages: the 16 Boston PDFs read from their text layer gave 601 passages instead of 651. In the answer-quality gate's retrieval, the answer was in the top 5 passages for 44 of 50 Boston questions instead of 41, and for 22 of 23 on the second corpus instead of 20, where both Table 1 questions now find its caption, header and rows in one passage. A gap threshold of 1.2 or 1.4 retrieved the same; 1.5 times the page's median spacing, with no caption or heading rule, found 39 of 50. Chunking and embedding take the rest, about 0.11 s a page in all. Of 36 PDFs from the Boston budget, the agent's live tests and that spec, 32 are read from their text layer. Two Boston volumes send Docling a cover photo with a title under 20 characters, Budget Organization an org chart whose page holds only its running header, and a filled site-visit form goes to Docling whole for its answers.

• Measured, failed pages only. Through the upload route, the queue and a real worker.py ingest process, Docling warmed in it first, against the whole file in Docling as before. The machine was not idle: other sessions ran tests, and their CPU is the last column.

  File	Docling whole	Failed pages only	Worker CPU-s, before / after	Other jobs' CPU-s, before / after
  Boston Education, 12 pp, cover photo	13.2 s (15.3 s on a first run)	3.5 s (3.9 s on a first run)	131 / 23	34 / 13
  Boston Capital Planning, 12 pp, cover photo	28.6 s (32.2 s on a first run)	3.4 s (4.9 s on a first run)	315 / 22	78 / 7
  Generated report, 10 pp, a scan at page 5 and a turned table page at 9	17.7 s	12.7 s	143 / 95	50 / 39

  The two volumes' text was exactly the text layer's pages 2 to 12, paragraph breaks included, after Docling's Markdown of page 1 read alone, and the report's its text-layer pages with Docling's Markdown of pages 5 and 9 in their places. In the report, the turned table page is the expensive part.

Scans and photos

A scanned page and an uploaded image are read by OCR alone before Docling is asked (ADR 0062). On 499 real pages, scanned forms, receipts and DocVQA pages, phone photos, book scans and public scanned PDFs, Docling took 4.9 s and 39 CPU-s a page; RapidOCR, the OCR it runs inside, read a whole page alone in about 2 s and 8 CPU-s, and found more of it, since Docling's layout step keeps only the text of the regions it classes. scan_ocr/ holds the readers and the page verdict, and pdf_reading/ocr_first.py the PDF planner.

• Which pages. In a PDF, after the text layer's triage (above): every page with no text layer (ocr:no-text), a scanner's invisible OCR layer (ocr-layer:scanner) or text pdfium cannot map (ocr:garbled); a turned page that is a scan, which pdfium renders upright; and, when no page is running text, the cover and picture pages (mixed:big-image, ok:sparse), whose render includes their title. Pages of running text keep their text layer. Every uploaded image, frame by frame.
• The reader. SURFSENSE_LOCAL_SCAN_OCR chooses: auto, the default, is Apple's Vision on a Mac and RapidOCR alone elsewhere; rapidocr is RapidOCR on a Mac too; off sends every scan and photo to Docling, as before, and so does a value it cannot read.
  • RapidOCR (rapid.py) loads the parser pack's PP-OCRv6 small detection and recognition files, or the identical ones the rapidocr wheel carries, with the line classifier off as Docling has it. One engine per process reads up to 4 pages at once, at most half the physical cores, at one onnxruntime thread each; a sideways page's turned reads share that cap. A page costs about 8 CPU-s however it is split, and one thread a page spends the least: on 100 real pages, one page at a time on 8 threads took 192 s and 1,016 CPU-s, 4 at once on one thread each 209 s and 824 CPU-s, and 8 at once 184 s and 1,436 CPU-s. A larger recognition batch was slower. The engine reads pages up to 2000 px long, which RapidOCR shrinks anything longer to anyway.
  • Vision (vision.py), at its accurate level with language correction and automatic language detection (macOS 13), through PyObjC, not ocrmac, which would switch Docling's own OCR on a Mac with no parser pack. Its fast level found 0.27 of all words and is not used. If Vision cannot start, a Mac uses RapidOCR.
  • Reading order (reading_order.py). Both readers give boxes of text; a visual row of boxes reads left to right as one line, so a receipt's item and its price, and a table's label and its figures, stay together, and two columns of prose read one after the other.
• Render. A PDF page greyscale at 216 dpi, Docling's OCR resolution, at most 2000 px on its long side for RapidOCR, which shrinks anything longer to that anyway (about 170 dpi for A4), holding shared.pdfium.lock only while it renders, so a page renders while earlier ones are read. The renders are closed on the ingest thread whatever stops the reading. An image upright by its EXIF and scaled as an A4 page at 216 dpi, never below its own pixels, as Docling's image page is sized: FUNSD's 754x1000 forms gave Windows OCR 0.49 of their words at their own size and 0.61 scaled, and Vision the same either way.
• The verdict (page.py). A page's OCR text stands unless it is garbled: under 30% of its characters letters and digits, RapidOCR's own confidence under 0.8 (character-weighted; 0.85 or more on every upright real page, 0.56 to 0.79 on sideways photos), or, for Vision, under 30% of its characters in words of three letters or more. A page under 20 characters is blank paper, a photo or a stamp: it keeps what it read, or a cover's own title when the layer had one. Tables stay with OCR: Haiku 5.5, given each page's text whole, answered 0.79 of 374 DocVQA questions from RapidOCR's text and 0.60 from Docling's Markdown.
• Sideways pages (upright.py). A page whose text stands mostly in boxes on end, which RapidOCR reads confidently but across one row, one the engine was unsure of, or, with no confidence, one or two characters a token, is read again turned 90, then 270, then 180 degrees, one turn at a time, and the first turn that reads as upright text replaces it; with none, the page keeps its verdict. All 20 camera captures of printed pages in the corpus came sideways with no EXIF turn.
• Docling. A garbled page goes to Docling in its place, through the splice a mixed PDF's flagged pages take, in batches that give way (below). A PDF with no page that reads as text, and an image with a garbled frame or no frame of text, go to Docling whole, as before. What OCR read of a PDF whose garbled pages Docling then reads is kept in ocr/ in the document's folder until the file is ready, so a Docling reading that gives the queue up resumes without reading every page with OCR again. The log says RapidOCR reads 17 of 20 pages of scan.pdf in 31.2 s; Docling reads 5, 9-10: page 5 is garbled; ....
• Figures. Ingest keeps none for a PDF read this way. A scan's figures wait until the agent or Studio asks for them, as a PDF over 40 pages does: the figures-only pass reads every scanned page with Docling in full, about 39 CPU-s a page, the cost this path removes. A PDF with born-digital pages beside its scanned ones still gets the pass queued at idle, as a text-read PDF does. An image is still its own figure.
• Cancel and giving way. Every 8 pages, about 17 s of RapidOCR's, a PDF's reading checks for a cancel and gives the queue up as a long Docling reading does, to the same sources (below). Before it gives way it keeps the pages read so far in ocr/pages.json in the document's folder, stamped with the original's SHA-256 and the reader's name (ocr_kept.py), and it gives way only once they are on disk, never after its last page. The run that resumes reads only the pages left; the few in flight as it gave way are read again. A reading whose garbled pages go to Docling keeps its pages there too, for a resume should Docling's part give way. The file goes once the source's text is built, before it is cut into passages, so a failure after that reads every page again on Huey's retry; when Docling reads the file whole instead, because no page read as text, OCR failed or OCR is off; when the job is cancelled or fails, in OCR's part or in Docling's after it; and when a cancel lands while it waits in the queue after giving way (ended_in_queue.py). A worker killed mid-reading keeps only what an earlier turn kept, and Retry resumes from it. An image's frames are read without a check.
• The release build runs worker --check-scan-ocr in the frozen worker on every system: each reader reads a rendered line, and an import the spec dropped fails the build.
• Measured on the Ryzen 7 5800X, Windows 11, through the upload route and a real worker.py ingest, one upload at a time: the six public scanned PDFs and three book scans, 199 pages, were all ready in 509 s against 829 s through Docling, a file in 332 s on average against 598 s, at 1,752 worker CPU-s against 7,343; no page went to Docling. 40 receipt photos, receipt scans and forms were all ready in 113 s against 202 s, an image in 65 s on average against 129 s, at 131 worker CPU-s against 1,416.

Figures

Ingest keeps the pictures in each source, so the agent can place a source's chart or logo in a document it makes (agent); a Word file read without Docling gets them from the figures-only pass, which its ingest queues as it ends. Chat and search do not use them.

• Where. Beside the original, in the document's own folder: figures/<n>.png, numbered from 1 in the order the source shows them, and figures/figures.json, written last and replaced whole (source_figures/layout.py). Deleting the source removes its folder and so its figures, and no table or migration is needed.

  {"version": 1, "figures": [{"n": 1, "caption": "Figure 1: Projected yearly costs …", "page": 2, "width": 1104, "height": 620}]}

• Which pictures (figures/pictures.py), from a DoclingDocument: the one the parse step made, before it is exported to Markdown, or, for a PDF read wholly or partly from its text layer, the one the figures-only pass makes:

  • PDF: each picture Docling's layout model finds, drawn charts and pictures in scans included, cropped from its page with pypdfium2 at 2x (144 dpi) after the conversion, with its caption and page. Docling's own generate_picture_images stays off, because it keeps every page's render until the conversion ends: about 7 MiB a page, 574 MiB for 80 A4 pages (figures/pdf_crops.py).
  • Word and PowerPoint: the pixels the file embeds. Docling links no caption for them; PowerPoint gives the slide as the page, Word gives none.
  • An image source is its own one figure, turned upright by its EXIF orientation.
  • Pictures in page headers and footers count, since a letterhead logo is the picture a user asks for. A picture repeated with the same pixels is kept once. A picture that refers to a path or a URL rather than carrying its pixels is skipped, so an uploaded page cannot pull an image from the machine into the library. Every figure is a PNG in RGB, or RGBA when it has transparency, which python-docx and ReportLab both read.
  • A text upload (.md, .txt, .csv, .html) keeps none, and neither does a recording: the figures-only pass would hand it to Docling.

• Never fails an ingest. A failure while keeping figures is logged and recorded as figures.json with no figures and an error, and the text is indexed as usual.

• Names. A figure is <document id>-<n>, built from the document when it is listed, never stored (source_figures/). list_figures() answers a source's figures from its figures.json; figure_file() resolves a name to its PNG only for a source in the caller's workspace.

• A Word file read without Docling (figures/word_figures_pass.py). Its package says where Docling would look: an image related from the main part, a header or a footer, or a w:drawing, which Docling draws only with LibreOffice on PATH (figures/word_pictures.py). With neither, ingest indexes no figures, so asking queues no pass; on 791 reference files that is 519, none of which Docling found a picture in. With an image, ingest drops any earlier figures and, once the file is ready, queues the figures-only pass at bulk priority (10), behind the ingests already queued: text first. With only drawings, nothing is queued until someone asks.

• The figures-only pass. A ready FILE with no figures.json is one that ingest left without figures. That is every PDF read wholly or partly from its text layer, a Word file read without Docling whose pass has not run, and any source ingested before figures were kept. The first list_figures() on it queues extract_figures(document_id) on the ingest queue at PRIORITY_ASKED (30), unless a copy waits at that priority or above already. That is ahead of a folder import, since someone waits on it, and behind a note or an upload at 100, since each pass is a Docling run and one Studio draft or refine asks for every source it is grounded on. It then answers that the figures are being extracted. Until the pass has run, the agent hears "its images are being extracted. Ask again in a minute.", and Studio's shelf offers no figures from that source, so a draft places none (studio). A source still pending or processing gets the same answer and queues nothing.

  • When a PDF read from its text layer, wholly or but for some pages, has image objects on its pages, a turned page's included, and at most 40 pages, ingest queues the pass itself at PRIORITY_IDLE (-10) (figures/idle_pass.py). That is below every ingest and every pass someone asked for, so it runs once nothing else waits. Drawing alone does not queue it, since no count of path segments tells a chart drawn as paths from a table's rules: of 183 agent-run PDFs with no image object, more than 30 segments on a page caught the figures of 2 distinct documents (28 files, 27 of them copies of one report) and queued 97 files where the pass finds none, and more than 40 caught none and queued 66. The pass loads Docling's models into the ingest worker, which a text-read PDF otherwise never does. A longer PDF, about 6 CPU-s a page for the pass, and one with no image wait to be asked. An ask while only the idle or bulk copy waits queues one at 30, so it does not wait out a folder import. Whichever copy runs second finds figures.json and reads nothing (listing.py).
  • The pass reads the original with Docling again, without holding a transaction, and writes the figures, leaving the text and chunks alone (figures/backfill.py). Once Docling has let go of the file, it checks that the same row, by its created_at and dedup_key, is still ready with the same original, and writes nothing, not even a failure, when it is not (figures/read_source.py). A source deleted meanwhile has its directory removed; when a later upload has taken its id and directory, only the file the pass read goes, if it is still that file. On a PDF, each page that shows characters from its text layer is read with Docling's layout model alone, without OCR or TableFormer. A scan (no such character, or raster images over half the page, as on a scan stamped with a digital page number) keeps the full reading, since only OCR reads its captions. Two kinds of page the layout reading finds are then read again in full, alone: one with a picture and a caption only OCR could read, with no characters in the text layer because it is drawn as outlines or pixels, by Docling's own rule for where OCR reads (figures/captions_for_ocr.py), and one with a picture inside a table, which TableFormer can move in the order. Each page is taken from its last reading, in page order (figures/pictures_reading.py, figures/run_reading.py, figures/scanned_pages.py). A page a run's reading lacks is kept as an empty page, so joining the runs leaves every later picture on its own page, as one reading of the whole file would lack that page too (docling_batches/lost_pages.py). Any failure on the way, in the scan check too, and a layout reading Docling only partly made fall back to the full reading of the whole file, which Word and PowerPoint always get. It is not retried, since its failure is recorded in the index. A run is at most 8 pages, so the pass can stop between any two, and a scan of more than 8 pages is read the same way, in full runs, rather than in one conversion.
  • Giving way (figures/queue_turn.py). Between two runs, once the pass has run 20 s and a pending file or note, a recording included, has its ingest queued at the pass's priority or above, it keeps the runs it has read under figures/work/ and queues itself again at the priority it was queued with, through the same once-per-priority rule. An idle pass so gives way to every upload, and an asked-for one, at 30, to notes and uploads but not to a folder import, which it would otherwise run ahead of again and give way to again; a pass that gave way is queued again only while its source is still ready; figures.json is still written last, whole, and emptying the folder for it removes work/. The next copy reads only the runs not kept. The kept runs are named by kind and pages and stamped as a long Docling reading's are, so another file's, or a reading with or without the parser pack, are read again; a source deleted meanwhile does not get its folder back. Resumed after every run, the pass kept the same figures.json and PNG bytes as one run straight through, and as Docling's whole-file reading, on Education, Capital Planning and a 20-page scan.

• Cost. scripts/measure_figure_ingest.py converts a file without and with figures, each in a fresh process. On generated files: an 80-page A4 PDF peaked at 2,519 MiB with figures and 2,517 MiB without, and cropping 20 vector charts from a 20-page PDF took 0.1 seconds. On 12 born-digital PDFs (city budget volumes and sample business reports; 185 pages, 28 figures) the figures-only pass kept the same figures.json and the same PNG bytes as the full reading on every one, in 155 s and 1,191 CPU-seconds against 660 s and 6,895: 0.8 s and 6 CPU-seconds a page against 3.6 s and 37. An 85-page budget volume, mostly tables, took 64 s against 453 s, and a 10-page one whose cover and one more page are raster, 18.5 s against 19.6 s. One of the 185 pages was read again for a caption the text layer holds but a drawn shape overlaps, which OCR reads too; that cost it 2 s. The pass loads a second, layout-only pipeline: about 144 MiB more resident memory in a worker that already holds the full one, and the ingest worker keeps both once it has run a pass. Layout alone loses the caption of a generated scan stamped with a digital page number, which is why such a page keeps the full reading.

Embedding index. Which model embeds is a property of the library, not a preference: a row in embedding_indexes holds the model's spec and names its vector table, which is built at the spec's width when onboarding locks the choice (data-model.md). upgrade_to_head() refuses to start when the table's width differs from the spec's. The per-batch check in embed() is the second line of defence. Studio indexes an artifact's body through the same index.

Before the choice. Onboarding gates the UI but not the API, so the routes that queue work which ends up embedded (writing, uploading, editing or retrying a document, a cloud import, and creating or regenerating a Studio artifact) answer 409 with the code embedding_not_chosen until an index exists.

Models on disk. The bge-small files (model_optimized.onnx and tokenizer.json, from Qdrant/bge-small-en-v1.5-onnx-Q) live in bge-small-en-v1.5/ under the models directory. They are pinned to one revision and checked against their hashes (bundled.py): every existing library was built with that exact file, so a different build would be a different vector space. Packaged builds ship them; a development checkout fetches them with scripts/fetch_embedding_model.py, which refuses a file that does not match. Any other curated embedder downloads into embeddings/<id>/ under the data directory, since the models pack is read-only in a packaged app. Chat answers 503 while the active model's files are missing.

The queue

• ingest is a SqliteHuey queue in huey.db, a file of its own so the consumer's constant polling never contends for the write lock on surfsense.db.
• worker-ingest drains it with one thread. Ingest saturates a CPU and writes to the database the API is serving from, so a second thread would spend its time behind the first one's lock. Studio has its own queue so an import never sits in front of a summary (ADR 0008). An idle consumer polls its queue at least every 0.5 s for ingest and every second for Studio and plugins (consumer.py): Huey's own default backs off to 10 s, which a packaged worker idle for a minute took to start an upload that its text layer then made ready in about a second.
• The queue survives restarts. Electron starts the workers beside the API and only the API migrates, so each consumer waits until the database is at the latest revision before taking a job (wait_for_schema.py); a job queued before an update would otherwise run against the old schema and fail. A job carries only its task's name, so each consumer imports every task before it starts (import_tasks()), and tests/integration/test_registration.py fails when that list falls behind the task modules on disk.
• Jobs are enqueued by upload, note creation, a note content edit and retry, and by import (import.md). The queue takes the highest priority first: interactive work at 100, a bulk copy and an import at 10. Equal priorities run in the order they were queued, since Huey's SQLite storage takes order by priority desc, id. The figures-only pass, extract_figures, shares the queue: at 30 when someone asks for figures, behind every note and upload but ahead of an import; at 10 when a Word file's ingest queues it as it ends; and at -10 when ingest queues it for a PDF read from its text layer (Figures). After 20 s it gives way to a source whose ingest is queued at its priority or above.
• A source the app quit in the middle of ingesting is failed with interrupted when the app closed when the ingest worker next starts, before it takes a job (interrupted_documents.py). Huey drops a job as it starts it, so nothing else would end it, and Retry accepts only a failed or cancelled document. A source still pending is still queued and is left alone.
• A recording's job gives the queue up between windows once it has run a minute and a source that is not a recording is pending (Recordings), and a long Docling reading between batches, or a long OCR reading every 8 pages (Scans and photos), once an upload or note is queued at its own priority or above (Long Docling readings), so a long file never holds every upload and note behind it. The ingest task is handed its own Huey task (context=True), so the job knows the priority it was queued at.
• Each transition the ingest worker makes, to processing, ready or failed, sends a documents event keyed by document id, and a recording's job sends one per transcribed window carrying a transient progress {done_s, total_s}, never stored, and so does each change the API makes itself: a note written, an upload, a rename or edit, a retry, a cancel, and a delete, whose status is deleted. The sources panel reloads its list on each event and whenever a dropped stream is back, except for a transcribed window's event, whose progress it keeps on the row when the row already has the event's status (the sources tree), so a note a plugin writes shows as it is written; it still refetches every 10 seconds while any row is pending or processing, in case a notice was lost (overview.md).

Long Docling readings

A PDF Docling reads whole, and a run of failed pages, longer than 8 pages is converted a batch of 8 pages at a time with page_range (docling_batches/); 8 pages or fewer is one conversion, as before. A PDF with failed runs reads all their batches as one sequence, so what follows also holds between two short runs. A scan reads at 3 to 5 s a page, so once the reading should give way, a waiting upload waits only for the batch in flight. The figures-only pass reads in runs of at most 8 pages too, kept under figures/work/ (Figures).

• Joined. The batches join into one DoclingDocument with DoclingDocument.concatenate, after a paragraph that runs on past a batch's last page is joined again by Docling's own rule: an open end, then a word going on, past headers, footers, tables, pictures, captions and footnotes (edge_paragraphs.py). Without it a budget volume read in 7-page batches split a paragraph at page 7. A whole-file reading keeps its figures from the joined document. Joining numbers each batch's pages on from where the last ended, so a page Docling could not read is first put back into its batch as an empty page, as one conversion of the file lacks it; otherwise every later picture would land on the page before its own (lost_pages.py).

• Checkpoint. Each batch read is kept as its DoclingDocument's JSON, pages-<first>-<last>.json, replaced whole, in docling/work/ in the document's folder: a folder, so it is never taken for the original. A stamp names the format, the original's SHA-256, Docling's version and whether the parser pack is there, since the converter reads OCR without RapidOCR's line classifier only with it; a checkpoint whose stamp differs is dropped (checkpoint.py). Hashing costs 0.26 s per 100 MiB, once a run. A source deleted meanwhile does not get its folder back.

• Giving way (docling_batches/queue_turn.py). Before each batch but the first this job reads, it checks for a cancel and, once the reading has run 60 s, gives the queue up as a recording does, JobYieldedError and pause_job, and is queued again at the priority it came with, behind the source it gave way to. It gives way to a pending file or note whose ingest is queued at its own priority or above (queued_ingests.py), so an upload's scan is not sent behind a folder copy started after it, and a pending row with no job queued holds nothing up. That source must not be a recording, and must have no docling/work/ or ocr/ of its own: long readings, Docling's or OCR's, do not take turns, which would leave all of them waiting until the last batch (waiting_work.py). It gives way only when every batch it read is on disk.

• Resume. The next run reads only the batches not kept. A reading that gave way is pending, so the startup sweep leaves it alone and its queued copy carries on after a restart.

• End. Read to the last batch, failed or cancelled, the job removes docling/. A cancel of a reading waiting in the queue after giving way removes it in the route, the document's or its folder's (ended_in_queue.py), and so does the job when a cancel lands as it gives way; a delete removes it with the folder. A worker killed mid-reading leaves it on purpose and the sweep fails the row, so Retry carries on from the batches kept; it stays until the source is retried or deleted. Huey's own retry after a failure starts over.

• Proved with real Docling: on a generated 20-page scan and on Education, read in batches straight through and resumed after every batch, the Markdown, figures.json and every PNG were identical to one conversion's.

• Measured on that scan through the upload route, the queue and a real ingest worker, Docling warm, against one conversion, while other sessions ran tests on the machine; Docling took 4.5 to 5 s a page under that load:

  	One conversion	Batches
  The scan alone, ready	108.2 s	94.9 s
  Its worker CPU-s	889	827
  Other jobs' CPU-s meanwhile	647	372
  A note written 10 s into the scan, ready after	81.0 s, after the scan (ready at 91.0 s)	61.4 s, before the scan (ready at 90.4 s)

  The gap in the scan's own time is the other jobs' load, not a speedup: batches cost nothing measurable. With the note waiting, the scan gave way once, after its second batch, some 70 s in. The reading runs a minute before it may give way, most of a 20-page scan, so the gain grows with the scan; a note waits at most that minute and the batch in flight, some 40 s on this machine.

Recordings

A recording becomes a source through the speech model chosen for speech_to_text (selection), on the second audio.cpp server, which the worker reaches only through the API's gate (admission). The transcript is the recording's content, so chunking, embedding, search, citations and the agent's sources/ mirror take it unchanged (worker/ingestion/audio/, proposal).

Everything the job keeps lives under the document's transcript/ folder, since the original must stay the only file beside it (recordings/layout.py): transcript/work/ holds audio16k.raw, plan.json, windows.jsonl and progress.json, and transcript/segments.json is written when the transcript is done.

1. Ready? The job asks GET /llm/transcription/status. With no speech model it stops before decoding (below). With the server unavailable, as in the seconds after a model is chosen or a crash, it asks again every 5 s for up to 2 minutes, giving the queue up as a window does, then fails with "The speech recognition engine stopped." and Huey tries again.
2. Decode (worker/audio_decode/). The worker's own binary, started as worker --decode-audio <in> <out> ahead of the consumer's imports, as --run-document-script is, reads the file's length from its headers (an Ogg file's from its last page, since miniaudio's Vorbis reader opens a path with fopen, which on Windows cannot open a name outside the ANSI code page) and checks the data volume for room at 115.2 MB an hour plus 64 MiB, then streams it through pyminiaudio (MIT; WAV, MP3, FLAC, Ogg Vorbis) to 16 kHz mono 16-bit PCM in audio16k.raw.part, renamed to audio16k.raw once every sample is written. It prints its outcome on one line and exits when the worker's end of its stdin closes. A malformed file kills the child, not the consumer, and the job checks for a cancel every second while it waits. An hour of MP3 decodes in 3.6 s. miniaudio's linear resampler cost 0.08 points of word error over scipy's resample_poly on an 8.6-minute 44.1 kHz file (5.91% against 5.83%, under the proposal's 2% relative), so it stays.
3. Speech (speech_regions.py). Silero VAD, through the API's own reader (voice_activity/silero.py) fed a minute at a time off disk, finds speech by its rule (0.5 threshold, at least 250 ms); each span is padded by 200 ms and joined to the next across gaps under 300 ms. It ran at 207 times real time on one thread. Without a staged Silero, or should it fail, all of the recording counts as speech, as a dictated phrase does.
4. Windows (windows.py). Speech packs into windows of at most 28 s, which bounds a request's memory, ending at the longest pause once a window holds 20 s, the one nearest 24 s among equals. A single run past 28 s is cut at its quietest 100 ms, by loudness, between 20 and 28 s. Silence between windows is never sent, and every window keeps its place in the recording. The plan goes to plan.json once; a job that finds it skips the speech pass, and decodes again only while windows remain to send.
5. The loop (window_loop.py). For each window not yet in windows.jsonl: check for cancel, send it as an in-memory WAV to POST /internal/transcription/windows (last when it is the final window and no other recording is pending or processing), append {n, start_s, end_s, text, language, model_id, intent}, write progress.json and send the window's progress event. A dropped connection or 503 transcription_unavailable is asked again after 1, 2, 4, 8 and 16 s, then fails with "The speech recognition engine stopped." 503 waiting is asked again after the cancel and yield checks: a hold or a dictation has no bound, and memory waits are summed in progress.json across yields, failing past 10 minutes with the API's sentence. A window answered by another model than the job began with starts the recording over under the status the API reports then, so one model writes the whole transcript and the language set for the last model is not carried to it.
6. Finish. The transcript becomes content through finish_job. transcript/segments.json keeps {model_id, language, duration_s, windows: [{line, start_s, end_s, language}]}, the line being where the window's paragraph is, counted from 1 as chunks count theirs. The row's metadata gains transcription: {language, duration_s, model}, written with merge_metadata, which reads and writes the one key in a transaction so every other key stands. transcript/work/ is deleted.

On the measuring machine (proposal), an 8.6-minute 16 kHz file went from pending to ready in 60 s in 22 windows, at 5.83% word error.

The transcript

One header line, then one paragraph per window that holds text, each starting with its time (transcript_text.py):

Transcript of meeting.mp3 · 42:10 · Japanese (detected) · Nemotron 3.5 ASR Streaming 0.6B. [hh:mm:ss] marks where each passage starts.

[00:00:00] …

[00:00:26] …

The chunker splits at blank lines, so while a window's paragraph stays under a chunk's 480 tokens, every chunk opens with its time. Measured on Kokoro's sentences at its pace, a 28 s window runs from 78 tokens in English to 214 in Hindi. The header names the language the user set, or else the one most windows were heard in, then the one heard longest, as "Japanese (detected)"; one the model may detect but is not listed for adds "lower accuracy with this model", and none at all reads "language not identified". The model is named as Settings names it, from its curated entry.

Yielding

The ingest queue has one thread, and an hour at the measured 11.5 times real time holds it for some 5 minutes. After each window but the last, and after each 503 waiting, once 60 s have passed in this run and a FILE or NOTE that is not a recording is pending, the job raises JobYieldedError, which pipeline._ingest catches before its failure handler: the row goes back to pending unless cancelled, and the job is queued again at PRIORITY_BULK. Recordings never give way to each other. A yielded recording survives a restart, since the startup sweep fails only processing rows, and resumes from its checkpoint with its decoded audio. A long Docling reading takes its turn through the same code, by its own rule, and never gives way to a recording (Long Docling readings).

Cancel and retry

Cancel is checked between windows without holding the write lock, and lands within one window; the window in flight finishes. A yielded job's queued copy is revoked as any is. No speech model, a memory wait past its limit, no room on disk ("Transcribing needs about 2.5 GB of free disk space."), no speech ("No speech was found in this recording.", from Silero or from a model that wrote nothing), no audio ("This file has no audio.") and a file the decoder cannot read ("This recording could not be decoded.") fail without retry: RecordingRefusedError becomes CancelExecution(retry=False) in tasks.py. Anything else uses Huey's two retries, and every retry, Huey's or the user's, resumes from windows.jsonl when the model id and the language intent match; another model or language starts over. A run that ends the job, a cancel included, deletes audio16k.raw and progress.json and keeps the plan and the windows; so does a cancel of a recording waiting in the queue after giving way, which has no run left to do it (recordings/ended_run.py).

No speech model yet

The upload is accepted. Finding no speech model, the job fails without retry with "Waiting for a speech model." and records transcription: {awaiting: true} in the row's metadata, which the next run removes as it starts; a model deleted between windows does the same and keeps the windows written. Choosing a speech model, by PUT /llm/selection/speech_to_text or an install with select: true, sets every failed row with the flag back to pending, clears the flag and its message, queues each at PRIORITY_BULK and sends each workspace a documents event (recordings/awaiting.py), so nothing is uploaded twice. A row's transcription.awaiting reads true only while it is failed.

Known gaps

• Deleting a folder is permanent; the Trash is designed, not built (proposal).
• Folders on disk (linked roots) are designed, not built.
• The sources tree renders every visible row of the open folders, with no windowing, so a very large open folder costs render time. Which folders are open is not kept across a reload.
• The tree's name filter runs on the client over the full listing, folding case with toLocaleLowerCase, which is close to the server's name_key but not identical; there is no GET /folders/search.
• There is no multi-select with Shift or Ctrl, so Move to… and Delete act on one row at a time.
• Parsing runs on the CPU. Windows and Linux ship CPU-only torch and no CUDA payload (ADR 0012).
• A PDF read from its text layer holds only what that layer holds. Content drawn only in pixels on a page that also has running text is lost. That covers raster org charts under 65% of the page, such as Finance's seven at 12 to 35%, a generated bar chart whose values exist only as pixels, and a picture under a cover title of 20 characters or more. Docling's Markdown lost these too, as <!-- image -->.
• A wrong ToUnicode map is not detected. A generated PDF whose map shifts every letter by one reads as fluent ok:text and is indexed as it is; Docling read it no better.
• A filled form field sends the whole file to Docling, and so do failed pages over half the file. Text spliced from the two readers meets at a page break: a paragraph running from a text-layer page into a page Docling read is cut there, and only Docling's pages carry Markdown headings and pipe tables.
• A cover photo whose title is under 20 characters, or a picture page whose only live text is its running header, is read by Docling, which adds OCR of the photo. Education's and Capital Planning's covers are (titles of 11 and 18 characters), and Budget Organization's org chart on page 3, which Docling's OCR reads as 272 mostly misspelled words.
• A text-read table is lines of values with no column boundaries. On a second corpus the text layer answered 19 of 30 table questions to Docling's 27. Most misses came where the chunker cut a paper's Table 1 from its caption and header, which paragraph breaks now keep in one passage: measured again, it answered 28 of 30 to Docling's 27. The rest came where a table's rows begin with a number, as in the spec's roadmap, 5 Dispatcher … on one line and 6 Gateway slash command … on the next, which the breaks do not change.
• A table whose rows are spaced like paragraphs, as when its cells wrap to two lines, gets a blank line between rows, so a long one can still be cut between rows. A sentence that wraps so that Table 2: starts a line is taken for a caption.
• The figures-only pass gives way to a waiting source, but not to a chat turn, so a local model shares the CPU with it.
• documents has no AUTOINCREMENT, so a deleted source's id goes to the next document. Upload and the figures pass remove what a delete leaves (Retry, cancel and delete), but a file another process holds open through both, such as an original streaming to a download, can still be found in the new document's directory. sqlite_autoincrement needs a migration that rebuilds the table.
• A chart drawn as paths in a text-read PDF with no image object is kept as a figure only once someone asks for figures, since no signal measured yet tells it from a table's rules. Filled or curved path segments, counted on distinct documents, are the next to measure.
• Long Docling readings queued together do not finish in the order queued: each gives way once to the next, which has not given way yet, so the last one finishes first. In a bulk copy, a long reading gives way after its minute to the next file of the copy, queued at the same priority, and goes back behind the rest of the copy, so it is ready after the copy's other files. A long reading sends no progress, which the sources panel shows for a recording only.
• The Word style cache replaces python-docx's DocumentPart.get_style during a conversion, as Docling 2.125.0 and python-docx 1.2.0 read styles. A release that reads them another way makes long Word files slow again without changing their text; STYLE_LOOKUPS in tests/integration/worker/test_docx_style_cache.py pins the lookups on a generated contract to catch it. The fix is not yet offered upstream.
• A picture in a Word or PowerPoint file is kept with no caption, even under a paragraph in Word's Caption style, because Docling links none; only a PDF's figures carry theirs.
• The figures-only pass reads a born-digital page without OCR unless a picture there may have a caption only OCR reads. Other text missing from the text layer, such as a paragraph drawn as outlines between a picture and its caption, only the full reading sees, and it can change which caption a picture gets; the pass matched the full reading on every file measured (Figures).
• The Word reader writes an equation's characters where Docling writes LaTeX, and fences no code. A Word file's figures still wait on Docling's import, in the figures-only pass.
• A Studio draft or refine from a Word source whose figures pass has not run yet places none of its pictures: it offers only the figures on disk (figure_shelf.py), and the pass runs behind the ingests queued before it, a whole folder import included. Asking queues a copy ahead of the import, so a later draft has them.
• Recordings take WAV, MP3, FLAC and Ogg Vorbis only; Opus, M4A, AAC and audio-only WebM wait for a decode-only LGPL FFmpeg (proposal, step 9).
• A recording cannot be played, nor a citation of one opened at its time; GET .../original does not commit before streaming, which a player would need (proposal, step 10).
• OCR reads one script family per converter: PP-OCRv6 covers Latin, Chinese and Japanese, so scanned Korean, Cyrillic, Devanagari or Arabic text is misread.
• A scan read by OCR alone has no Markdown headings or pipe tables; its tables are rows of cells on one line each. A camera capture shot at an angle is read upright, but RapidOCR's rows can mix lines its perspective skews; Vision read the corpus's well. Vision's speed was measured only on a virtual Mac with no Neural Engine.