One appliance, the machines you own, one endpoint
AiGrid has two parts. The appliance is a container image that runs on a single-node Kubernetes or a Kubernetes cluster. It is the coordinator and holds the index. The pools are the machines you lend, which do the heavy document conversion. Your documents never move out of your own storage.
The path of a document
Scan
Your documents sit in S3-compatible storage that you own. The appliance scans it on a schedule, with read-only access, and queues anything new or changed.
Claim
A machine inside its allowed hours asks the appliance for work. The appliance never pushes work to a machine, and it only offers a document to a machine large enough to handle it.
Convert
The machine downloads one document, works out the reading order, headings, columns, and tables, OCRs any scanned pages, and cuts the text into short passages. This is the expensive part, and it is what the pool does. It handles PDF, Word, Excel, PowerPoint, RTF, HTML, and pictures that contain text.
Index
The machine sends the passages to the appliance, which indexes them centrally. The whole index is produced by one version of one model.
Retrieve
An AI assistant sends a question to the retrieval endpoint and gets back the best-matching passages, each with its filename and page number. The assistant writes the answer and cites the sources.
Pools and hours
You group machines into pools, and an administrator sets the schedule for each one. The agent runs at low priority whatever the schedule, so it does not compete with other work on the machine.
- Workstations
- You install the AiGrid agent with Microsoft Intune, Group Policy, or whichever software deployment tool you already use. Windows 11 desktops run the worker as a container, using the container support built into Windows under WSL. The default hours are weekday evenings and overnight. Each desktop follows its own local clock, so offices in different time zones each keep their own evening.
- Servers and clusters
- You deploy the agent as a manifest with the tool you prefer: Portainer, Argo CD, Flux, or kubectl. On a Kubernetes cluster the worker runs as low-priority pods that scale up from zero when there is work and are evicted when the cluster needs the room. A server pool, such as a DR site, can run around the clock.
- No GPU required
- The models that do layout analysis, OCR, and embedding are small enough to run on ordinary processors. If a machine in a pool has a GPU, the worker uses it, and otherwise it runs on the CPU.
- Unreliable machines are expected
- Machines report their own capacity, so nobody enters a specification. If a machine sleeps, crashes, or loses its network mid-document, its claim lapses and the document returns to the queue.
A good guest on borrowed machines
The people whose desktops you lend should not have to think about AiGrid. It is designed to stay out of their way.
It keeps to your schedule
An administrator sets the schedule for each pool. A workstation pool can run out of hours only, and a pool of unused compute, such as a DR site, can run around the clock.
It takes what it is given
The worker runs under a memory ceiling set on the machine, at the lowest priority for processor time, and is the first thing stopped if memory runs short.
It stays in its own lane
On a desktop the agent runs under its own service account, not the user's account. It uses temporary storage only while it converts a document.
What your assistant gets back
- Passages with citations
- Question in, ranked passages out. Each passage carries its text, filename, and page number. Passages never span a page boundary, which is what makes page-level citation possible.
- Meaning and keywords together
- Search by meaning finds the paragraph that answers a question in different words. Keyword search finds the part number, the surname, and the acronym. AiGrid runs both and combines them.
- Nothing skipped for speed
- Every passage is scored against the question instead of using an approximate index.
- Honest about unreadable documents
- A result from a document that produced very little text, such as a photograph or an unlabeled diagram, is flagged so the assistant does not treat it as evidence.
- Any assistant
- The dashboard provides a small download with connection instructions and the appliance's certificate. Any assistant or agent that can call an HTTPS endpoint can use it.
Use all of it, or only the part you need
The appliance lets you move parts of the pipeline outside it.
- Everything built in
- The default. Documents are converted on your pools, indexed in the appliance's own vector database, and served through its retrieval endpoint.
- Your own vector database
- If you already run a vector database for other services, the appliance can write to it and search it in place of the built-in one. Tested with Elasticsearch 9.5. Other Elastic-compatible databases should work.
- Your own RAG layer
- You can move the whole RAG layer outside the appliance and keep AiGrid purely as a text extraction tool. Your pools do the conversion, and AiGrid hands each document to your pipeline as markdown, with page markers.
What you need
- Kubernetes to run the appliance, which ships as a container image. A single node is enough, and a cluster works too. Allow at least 8 CPUs, 32 GB of memory, and 100 GB of disk, or 4 CPUs and 8 GB when it runs for text extraction only.
- At least 4 GiB of memory made available to the worker on each machine you lend.
- S3-compatible object storage holding your documents, with read-only access for the appliance.
- Optionally, an OpenID Connect identity provider for operator sign-in.