What the AI Act asks of companies using AI on their own documents

The deadlines for high-risk systems are slipping. Buyers and auditors are still putting the same four questions to your document system today.

Many of the companies we talk to have a working system that reads invoices, sorts incoming post or finds answers in internal procedures. Since the political agreement on the digital omnibus of 7 May 2026, they know the EU wants to simplify the AI Act and push back the deadlines for high-risk systems. The natural reflex is to wait until everything is settled. We think that is the more expensive choice.

What follows is how we read the rules when building document systems. It is not legal advice. For your own situation you need a lawyer who can put the text and your use case side by side.

Provider or deployer

The AI Act distinguishes between whoever develops a model and places it on the market, the provider, and whoever uses a model within their own operations, the deployer. The rules for general-purpose models now apply, but those obligations fall mostly on the people who build the models: documentation, information about training data, arrangements around copyright.

If you use an existing language model through an API to read delivery notes, you are usually a deployer, not a provider. That position still comes with conditions. A company that substantially modifies a model, or sells a system to others under its own name, can end up in the provider role itself. Better to settle that question early than once the product is already running at customers.

Why most document systems are not high risk

Whether a system counts as high risk depends on what it is used for, not on the technology behind it. Pulling data from purchase orders, classifying contracts by type or searching an internal knowledge base with RAG: none of these are on the list. Most of what we build falls outside it.

The same technology can drift across the line, though, often without anyone touching the code. A classifier for incoming post gains a category for job applications and starts helping decide who gets an interview. An extraction pipeline for payslips starts feeding a credit decision. A search system can end up helping decide who gets access to a benefit or an insurance policy. Recruitment, creditworthiness and access to essential services are exactly the areas where the Act gets stricter. On every project we have learnt to ask not only what the system does, but where its output goes next.

Four things you need regardless

Even outside the high-risk categories, we keep hearing the same questions from procurement teams, internal audit and data protection officers. They want to know what the system is allowed to read, what it records, who approves the output and how a decision can be traced. That leads to four things we build in by default:

• scoped access to data, per user and per source

• logging of input, model version, prompt and output

• human approval on output with consequences

• traceability of every answer to its source document

Scoped access means a search system over HR files does not share an index with the general knowledge base, and the user's permissions also govern what the model gets to see. With logging, our early mistake was to record only errors. When someone asks months later why a document was classified a certain way, it is the ordinary cases you need, along with the model version that was running at the time.

Human approval is not needed everywhere. A suggested tag on an internal memo can go through automatically; an amount heading for the accounts, or a file being rejected, cannot. Traceability means every extracted field and every answer points to the page and passage it came from. Without that reference nobody can check the output, and approval becomes a formality.

Why waiting costs more

Adding these four things afterwards costs more than building them in from the start. Logs you never kept cannot be recovered, and a pipeline that did not store source references often has to reprocess everything. A system that already has these answers ready gets through a procurement review or an audit meeting faster. And if the rules do tighten later, or your use case moves into a sensitive area, the foundations are already in place.