Collections & knowledge

A collection is a named body of documents an agent can search. It owns the settings that decide how its documents are broken up, embedded and retrieved, which is why the collection — not the document and not the agent — is the unit you configure.

Grouping matters more than it first appears. A single collection holding your product manual, your HR handbook and last quarter’s board minutes will retrieve badly, because every query competes against all three. Split knowledge along the lines your questions actually fall on, and attach the relevant ones per agent.

Creating a collection

curl -s -X POST "$KOVAC_BASE/api/manage/collections" \
  -H "Authorization: Bearer $KOVAC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "handbook"}'

name is the only required field, and must be unique within your tenant — a duplicate returns 409. Everything else has a default:

• chunk_size (1000) and chunk_overlap (200) — how long each indexed passage is, in characters, and how much neighbouring passages share. Overlap exists so a sentence straddling a boundary is not lost to both sides.
• embedding_alias — which registered embedding model turns passages into vectors. Pick it from the models the operator has registered; the model id and the vector width follow from it and are reported back as embedding_model and embedding_dims. Changing it changes what the stored vectors mean, so every document is marked stale and a re-index re-embeds them — reads keep serving the old vectors until it cuts over.
• rerank_top_k (40) and rerank_top_n (8) — how many candidate passages are gathered, and how many survive re-scoring to reach the agent.

Creating and modifying collections requires the editor role or above. Reading them needs only a valid key, whatever its role.

Chunking respects the document’s structure

Passages are never merged across a page or section boundary — whichever strategy you pick, a chunk belongs to exactly one page and one section. That is what makes every chunk’s provenance unambiguous, and it is why a citation can name a page and a section rather than just a file.

chunk_size is a ceiling on how long a chunk may get, not a target every chunk hits. A short section stays a short chunk.

Four strategies

Set chunk_strategy on the collection:

Strategy	What it does	Good for
structure	Packs consecutive passages that share a page and a section into one chunk, up to chunk_size. A heading always travels with the body beneath it.	Most documents. Reports, handbooks, anything with headings.
hierarchical	Packs like structure, then groups those chunks under their section — a parent window of up to parent_size. Search matches the small chunks; when enough of a section matches, the whole section comes back once.	Long documents with real headings, where a question is answered by a section rather than a paragraph. Measure it first (see Improving retrieval quality).
page	One chunk per page, or per slide for a deck.	Slide decks, scanned forms, anything where the page is the unit.
fixed	Splits each passage the parser found on its own and never joins two.	Matching what a collection produced before this setting existed.

fixed is the default for collections created before strategies existed, so their chunks do not change underneath them. New collections created in the UI default to structure.

Why structure is usually the right answer: a PDF parser emits one passage per paragraph and one per heading. Under fixed that makes a long document into hundreds of paragraph-sized chunks, and — more damagingly — every heading becomes a chunk with no body, which retrieves as a bare title matching a query it cannot actually answer.

Sizing in characters or tokens

chunk_size_unit is chars (the default for existing collections) or tokens. Tokens are what the embedding model actually sees, so a token budget is the one that maps onto the model’s context — a 512-token chunk is about 2,000 characters of English, but that ratio moves with the language and the amount of code or punctuation in the text.

New collections created in the UI default to 512 tokens with 64 of overlap.

Match small, return the section

Under hierarchical a document has two granularities. The children are exactly the chunks structure would have made at chunk_size, and they are what search matches on — small, precise, one page and one section each. The parents are their sections: children are grouped into windows keyed on the section path of each child’s last block, packed up to parent_size (four times chunk_size unless you set it, in the collection’s own unit). A section longer than a window becomes several. A parent may run over a page break within its section; its children never do, and a citation always lands on the child that matched — the passage the reranker actually scored, even when what comes back is the whole section around it.

Three retrieval settings decide what an agent receives, and change nothing stored — they take effect on the next search, no re-index needed:

• parent_return: auto (the default) merges the matched passages into their section, after reranking, once at least merge_threshold of the section’s passages matched — two of four at the default 0.5 — and returns the passage alone otherwise. The merge runs on the scores the reranker gave each child: the merged section takes its best-scoring child’s score, and the citation still carries that child’s id.
• parent: always the section, however little of it matched.
• child: the passage only. Useful as the control arm when you measure.

Under every policy an agent can ask for the section explicitly with the search tool’s expand argument, which on a hierarchical collection means “the enclosing section” rather than N neighbouring chunks.

curl -s -X PATCH "$KOVAC_BASE/api/manage/collections/$COLLECTION_ID" \
  -H "Authorization: Bearer $KOVAC_API_KEY" -H "Content-Type: application/json" \
  -d '{"chunk_strategy": "hierarchical", "chunk_size_unit": "tokens", "chunk_size": 256, "parent_size": 1024}'

Changing the strategy or parent_size marks every document stale; apply it with a re-index. The chunk preview takes parent_size too and reports how many parents a document would get (parent_count) and their median size (median_parent_tokens).

Every chunk carries its section

Each chunk records the heading trail it sits under — Handbook › 3 Returns › 3.2 Refunds — and that trail is prepended to the text when the chunk is embedded. A paragraph reading “Refunds are issued within 14 days” is nearly identical to every other document’s refund paragraph until you know which document and which section it came from.

The trail is not part of the chunk’s stored text, so keyword search still matches the body alone. A header repeated across every chunk of a document would otherwise flatten keyword scores across all of them.

Trying a change before committing to it

POST /collections/{id}/documents/{document_id}/chunk-preview chunks a document with parameters you supply and returns the result without writing anything — plus the numbers that make two settings comparable: how many chunks, what share of them are under 50 tokens, and how many are a heading with no body.

curl -s -X POST "$KOVAC_BASE/api/manage/collections/$COLLECTION_ID/documents/$DOC_ID/chunk-preview" \
  -H "Authorization: Bearer $KOVAC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"chunk_strategy": "structure", "chunk_size_unit": "tokens", "chunk_size": 512}'

When the numbers look right, change the collection and POST …/documents/{document_id}/rechunk to apply it. Re-chunking reuses the parser output stored at upload, so it does not re-upload or re-parse the file — it is cheap enough to iterate on.

Tuning retrieval per collection

Two optional knobs, both left unset by default so they follow the platform default:

• hybrid_weight — how much of the ranking comes from meaning-based search versus keyword search, from 0.0 (keywords only) to 1.0 (meaning only). Unset behaves exactly as an even split.
• score_threshold — shown in the UI as Confidence bar (retry & citations). How confident the reranker must be for a result to count. It drives two things and filters nothing: if no result clears it, retrieval runs one broadened retry and returns that pass’s results; and every citation scoring below it is badged weak rather than supported. Raise it for a collection where a wrong answer is worse than no answer. The platform default is 0.60, calibrated against real answerable-vs-unanswerable score distributions. In a search spanning several collections the strictest collection’s value applies to all of them.

Ingesting a document

Upload is multipart/form-data:

curl -s -X POST "$KOVAC_BASE/api/manage/collections/$COLLECTION_ID/documents" \
  -H "Authorization: Bearer $KOVAC_API_KEY" \
  -F "file=@handbook.pdf" \
  -F "source_uri=https://intranet.example.com/handbook.pdf"

file is required. source_uri is optional and is what citations will point at — pass the document’s real home if it has one, so a reader can follow a citation back to the source. Omit it and the platform records upload://<filename>, which identifies the document but does not lead anywhere.

Accepted formats, by family:

Family	Extensions
Documents	.pdf .docx .pptx .html .htm .md .txt .rtf .odt .odp .epub
Spreadsheets and delimited text	.csv .tsv .xlsx .xls .ods
Structured data	.json .yaml .yml .xml
Email	.eml .msg
Images (OCR)	.png .jpg .jpeg .tif .tiff .bmp .gif .webp .jp2
Source files	.py .js .ts .tsx .java .go .rs .rb .php .c .cpp .cs .kt .swift .scala .sh .sql and others

Anything else is rejected with 400 before any work starts. So is a file whose content contradicts its extension — a ZIP renamed to .pdf, an HTML error page saved as .docx. Text formats have no signature to check, so they are never rejected on that ground.

Spreadsheets, CSVs and JSON exports

A spreadsheet is not a document. A 5,000-row ticket export rendered as prose gives an agent 5,000 paragraphs that retrieve for nothing, so these formats have a mode of their own.

records (the default). One chunk per row. Each column gets a role, proposed from the header row and the first 200 rows and editable afterwards:

Role	What it does
index	The value goes into the chunk’s text and is embedded
metadata	The value becomes a filterable key
both	Both
ignore	Dropped

So a ticket export ends up with its subject and body searchable, and its status, priority and opened_on filterable:

{ "status": "open", "priority": "P1", "opened_on": { "gte": "2026-01-01" } }

A metadata column declares itself. The first upload adds those columns to the collection’s metadata schema with a type inferred from the values — a number, a date, a small vocabulary as an enum. It only ever adds: a column the collection already declares keeps the type it has, and your values are validated against that.

sheet. The whole sheet as one table instead, chunked like any other document. Right for a small lookup table inside a report.

The same applies to a JSON or YAML file whose root is an array of objects, and to an array anywhere inside one if you point at it with records_path. A JSON file that is not an array of objects — a config file, say — is flattened by path instead, so a chunk knows it came from database.replicas.

Changing the roles re-parses the document. Roles decide what a row’s text and metadata are, so applying new ones means rebuilding the document from the original — which discards any manual chunk edits and per-block decisions on it. The UI asks before doing it.

Scanned PDFs and images

A PDF with no selectable text, and every image, is read by optical character recognition, and OCR has to be told the script. The default reads English and other Latin-script text; for Cyrillic — Macedonian, Serbian, Bulgarian — choose it, or the text comes back as nonsense.

In the UI, pick Language of scanned text when uploading. A document already uploaded has an OCR tab where the choice can be changed; applying it re-parses the document, with the same warning as changing column roles.

Over the API, send it as ingest_options on the upload:

curl -s -X POST "$KOVAC_BASE/api/manage/collections/$COLLECTION_ID/documents" \
  -H "Authorization: Bearer $KOVAC_API_KEY" \
  -F "file=@scan.pdf" \
  -F 'ingest_options={"mineru_lang": "cyrillic", "tesseract_langs": ["mkd", "srp", "eng"]}'

Option	What it sets	Values
mineru_lang	The primary OCR model	ch (the default: Chinese and English), ch_server, cyrillic, east_slavic, korean, el, arabic, devanagari, th, ta, te, ka
tesseract_langs	The fallback’s languages, used only if the primary fails, most important first	eng, mkd, srp

To change it on an existing document, PATCH .../documents/{document_id}/ingest-options with the same fields and "confirm_reparse": true. A .zip import does not take these options; set them per document afterwards.

Email

An .eml or .msg becomes a document whose text is the message body, with the subject as its heading. The sender, date, subject, recipients and conversation thread are extracted and filterable — author and created are the platform’s own keys, so {"author": "ada@example.com"} works with no schema of your own.

Attachments become their own documents, linked to the message. Each gets its own status, its own chunks and its own viewer, and an attachment that fails to parse does not fail the message. Attachments in formats the platform does not accept are skipped, and named in the ingestion’s warnings.

Ingestion is asynchronous

The response is 202 Accepted:

{ "document_id": "...", "status": "parsing" }

Parsing, chunking and embedding happen in the background. Poll the document until it settles:

curl -s "$KOVAC_BASE/api/manage/collections/$COLLECTION_ID/documents/$DOCUMENT_ID" \
  -H "Authorization: Bearer $KOVAC_API_KEY"

status moves from parsing to done, or to error — in which case the error field carries the reason. A document is not searchable until it reaches done, so an agent asked about a freshly uploaded file will honestly report finding nothing.

One case looks like an error and is not: uploading a document whose content the collection already holds returns 200 rather than 202, with status already done. Documents are identified by a hash of their content, so re-uploading an unchanged file is a no-op instead of a duplicate. Re-uploading a changed file with the same name does ingest, because the content differs.

GET .../documents lists a collection’s documents, with limit (1–200, default 50) and offset.

How retrieval actually works

When an agent searches, several things happen that are worth knowing about, because they explain results that would otherwise look surprising.

The query is decomposed into up to three focused sub-queries, so a compound question does not have to match a single passage. Each is answered by two retrieval strategies at once — vector similarity for meaning, keyword matching for exact terms like part numbers and proper nouns — and the two result sets are fused. That combined candidate set is then re-scored by a reranking model, and the best rerank_top_n passages are what the agent sees. If the best of those still looks weak, the query is broadened and retried once.

None of this is configured per agent. It is the same pipeline for every collection, tuned by the two rerank_* settings above.

Documents in more than one language

Every chunk is labelled with the language it is written in when it is ingested, and every document with its main language. A short fragment — a heading such as “Член 1” — takes its document’s language rather than a guess of its own, and a table or section in another language keeps its own label. Text that is mostly numbers is left undetected. Serbian, Croatian and Bosnian share one label.

Keyword matching respects the language: English and German words are matched with their other forms (Kündigungsfrist finds Kündigungsfristen), and Serbian likewise; languages the database has no stemmer for, such as Macedonian and Albanian, are matched on whole words.

A collection’s languages are the ones that make up at least 5% of its chunks, up to four. When a question is in a different language, the query planner also translates each sub-query into the collection’s other languages, and keyword matching runs on every translation, so a Macedonian question finds a passage that only exists in an English document, and the reverse. Vector similarity already works across languages and is not repeated per translation. A collection in one language, asked in that language, is searched exactly as before.

You can pin the list under Settings → Languages, or with PATCH .../collections/{id} and "languages": ["mk", "en"] (null returns to automatic). Pin a language the collection does not contain yet when people will ask in it — English questions over Macedonian documents, for example. A collection’s GET returns the effective list as languages, what you pinned as languages_pinned, and the measured split as language_shares.

A document’s detected language can be corrected on its page, or with PATCH .../documents/{id}/language and {"language": "en"}. Its chunks are relabelled straight away; nothing is re-parsed or re-embedded.

The agent answers in the language the question was asked in, and each result and citation shows its source’s language. The language metadata key holds the chunk’s language, so {"language": "en"} is a valid filter. In a trace, the retrieval.query_plan observation shows the question’s detected language and every translation, and retrieval.pre_rerank counts keyword matches per language as lexical_by_config.

What the agent is allowed to search

The tenant filter is applied server-side, from the credential, on every query. It is not a parameter an agent can set, and not something a question can persuade it to change.

Within the tenant, an agent can only search the collections named in its own collections configuration. Naming one in a search narrows the scope; it can never widen it. An agent with no collections configured has nothing to search at all — see Attach knowledge to an agent.

Next: Memory.