AI security in 2026: governing systems that act on their own

Two years ago the board's question was whether staff should be allowed to use chatbots. That question is settled and largely irrelevant. AI is now inside the products: retrieval pipelines reading internal document stores, copilots wired into ticketing and CRM, and agents that call APIs, open records, and in some deployments move money.

That shift changes the control problem entirely. You are no longer securing a tool your staff use. You are securing a system that takes actions on your behalf, using credentials you issued, based on instructions it may have read from a document an attacker controls.

What actually changed

Four shifts account for most of the new risk, and none of them are about the model getting cleverer.

• From chat to systems. Almost no serious deployment is a bare model any more. It is a pipeline: a retriever pulling from your document stores, a prompt template, a model, a set of callable tools, and an output rendered into an application. Each of those is a component with its own failure mode.
• From model risk to system risk. In the assessments we run, the model is rarely the weak point. The plumbing is — an over-permissioned service account, a vector store with no access control, a retriever that happily returns HR records to anyone who asks the right question.
• Non-determinism as a control problem. The same input can produce a different output. Test evidence built for deterministic software — “we ran the test, it passed, here is the screenshot” — does not establish very much. Assurance has to shift toward distributions of behaviour, not single passes.
• Autonomy. An agent that can call tools has, in effect, been granted permissions. Most organisations grant them far more broadly than they would grant a junior employee doing the same job.

The AI assurance stack

It helps to stop talking about “AI risk” as one thing. Five layers, each with a distinct failure mode and a distinct owner:

Layer	What goes wrong	The control that actually helps
1. Data	Personal data enters training or retrieval with no lawful basis. The vector store inherits none of the source system's access controls, so retrieval quietly becomes a bypass.	Classify before you ingest. Enforce document-level permissions at retrieval time, not just at the UI.
2. Model	Undocumented model changes alter behaviour overnight. Hosted models move data across borders. Fine-tuning bakes in data you later have to delete.	Version and pin models. Contract for change notice. Treat hosted inference as a cross-border transfer.
3. Application	Model output is rendered or executed without validation, turning a language model into an injection vector for the surrounding app.	Treat every output as untrusted input. Validate, encode, and constrain before it reaches anything else.
4. Agent & tools	Excessive agency: standing credentials, broad API scopes, and no approval step before irreversible actions.	Least privilege per tool, short-lived scoped tokens, and a human in the loop for anything you cannot undo.
5. Oversight	No inventory, no named owner, no review cadence. The system drifts and nobody notices until it is a finding.	An inventory with an accountable owner per system, and a scheduled review with recorded outcomes.

Where AI systems actually get attacked

The OWASP Top 10 for LLM Applications is the most useful public reference here. Condensed to what we see in practice in the Kingdom:

• Prompt injection. Direct injection is a user trying to talk the system out of its instructions. Indirect injection is the serious one: a payload sitting in a document, a web page, or a support ticket that the retriever pulls in and the model obeys. The attacker never touches your interface.
• Data and index poisoning. If anyone can write to a source the retriever indexes — a wiki, a shared drive, a ticket queue — they can influence answers for everyone.
• Sensitive information disclosure. Usually not the model leaking training data. Usually the retriever returning a document the asker was never entitled to see.
• Excessive agency. The agent can do more than the task requires, so a successful injection inherits all of it.
• Supply chain. Models, plugins, extensions, embedding services, and vector databases pulled in with less scrutiny than a library would get.
• Model and API exposure. Inference endpoints without authentication, rate limits, or logging — sometimes standing up quietly during a proof of concept and never taken down.

The regulatory picture in the Kingdom

There is no single “AI law” you can point at and be finished. What applies is a stack of existing obligations plus AI-specific guidance:

• SDAIA — the Saudi Data & AI Authority sets national direction, has issued AI ethics principles and guidance on generative AI, and is also the regulator for personal data.
• PDPL — the Personal Data Protection Law applies to AI processing exactly as it applies to anything else. Lawful basis, purpose limitation, retention, data subject rights, and cross-border transfer rules all bind your training data, your retrieval corpus, and your prompts.
• NCA ECC — an AI system is still an IT system. Asset inventory, access control, logging and monitoring, third-party requirements, and cloud controls all apply, and this is where most AI deployments first fail an assessment. Our NCA ECC checklist covers that ground.
• Sector regulators — financial institutions in particular should expect model governance, explainability, and outsourcing expectations to be applied to AI-driven decisions.
• International anchors — ISO/IEC 42001 (AI management systems) and the NIST AI Risk Management Framework give auditors a recognised structure. The EU AI Act matters if your system's output is used in the EU, regardless of where you are.

A control checklist that survives an audit

The order matters. Each step makes the next one cheaper.

1. Inventory every AI system, including the ones nobody told you about. Start with expense reports, SSO logs, and browser extensions. Shadow AI is the norm, not the exception. You cannot govern what is not on a list.
2. Classify by impact and autonomy. Two axes: what happens if it is wrong, and can it act without a human. A drafting assistant and an agent with payment API access are not the same risk and should not get the same controls.
3. Fix the data layer first. Decide what may be ingested, establish the lawful basis under PDPL, set retention, and — critically — enforce source-system permissions at retrieval time. Most disclosure incidents die here.
4. Constrain the agent. One scoped credential per tool, short-lived, least privilege. Explicit human approval for anything irreversible: payments, deletions, external communication, production changes.
5. Test adversarially, not functionally. Red-team the prompts. Plant an indirect injection in a document the retriever indexes and see whether it fires. A functional test that the assistant answers questions correctly proves nothing about this.
6. Log the whole interaction. Prompt, retrieved context, tool calls, output, and who approved what. Logging only the final answer makes incidents unreconstructable — and this is the evidence an assessor will ask for.
7. Assign an owner and a review cadence. A named person, a scheduled review, and a written record of the outcome. “The AI team” is not an owner.
8. Contract for it. Training-data use, model change notice, data location, deletion rights, and audit access. Renegotiating after deployment costs considerably more than asking before.

The five mistakes that cost the most time

1. Treating the model as the risk. It is usually the integration. Time spent evaluating model safety benchmarks is time not spent on the over-permissioned service account.
2. Policy without inventory. An acceptable-use policy circulated to staff while eleven undocumented systems run in production.
3. Logging the answer but not the context. Without the retrieved documents and tool calls, you cannot explain why the system did what it did.
4. Standing credentials for agents. Convenient during the pilot, and the single largest amplifier of every other weakness.
5. Assuming the vendor's certification covers you. Their certification covers their platform. Your configuration, your data, and your agent's permissions are yours.

Where to start this quarter

If you do nothing else: build the inventory, classify by autonomy, and fix retrieval-time permissions on the highest-impact system. Those three take weeks rather than quarters, and they remove the failure modes that turn into disclosure incidents. Everything else in this article is easier once they are done.

Frequently asked questions

Does Saudi Arabia's PDPL apply to data used to train or prompt an AI system?

Yes. The PDPL applies to personal data regardless of the technology processing it. If personal data is used for training, fine-tuning, retrieval, or pasted into a prompt, the same obligations apply — lawful basis, purpose limitation, retention limits, data subject rights, and controls on transfers outside the Kingdom. Using a third-party model hosted abroad is a cross-border transfer and should be treated as one.

Do we need ISO/IEC 42001 certification?

It is not mandatory here. It is useful when you have to demonstrate AI governance to a customer, regulator, or board, because it gives auditors a recognised structure. Many organisations adopt the structure without pursuing certification and capture most of the practical benefit.

Does the EU AI Act affect a Saudi company?

It can. The Act reaches providers and deployers outside the EU when their system's output is used within the EU. If you serve EU customers or embed a model in a product sold there, classify your system before assuming you are out of scope.

What is prompt injection, and can it be fixed?

It is instructions hidden in content the model reads, which the model then follows. Indirect injection — arriving through retrieved content rather than the user — is the harder variant. No filter reliably eliminates it. The defence is architectural: treat model output as untrusted, scope the tools an agent can call, and require human approval for anything irreversible.

Who should own AI risk?

A named business owner per system, supported by security, data protection, and legal. AI risk spans procurement, data protection, and the business decision the system influences, so it cannot sit with the security team alone.