Reading is not asking

Security & governance@Ravindra

Core thesis: Vision-language models are built to read whatever text or signal appears inside an image. Security breaks down when an application treats something the model read inside an attachment as something the authenticated user asked for.

From the prompt to the picture

In Prompt is not permission, we followed a banking request through the application boundary. A rule in the system prompt asks the model to behave. A check in the data-access code decides what it can actually do.

One paragraph in that post deserved more room. Instructions can also arrive inside the records the assistant reads—such as a transaction remark asking it to look up another account. That remark is data from the workflow, not authority to widen access.

This post takes that paragraph further, with one change: the instruction arrives in an image. A scanned statement, a screenshot attached to a complaint, a chart in a shared folder. Vision-language models read text in pictures well, which is exactly why we point them at documents. That same capability means a picture can carry an instruction.

None of this is new. It has been demonstrated repeatedly since 2023, and that history teaches the same lesson as the first post. The useful question is not whether we can spot every hidden instruction before it reaches the model. It is whether the application treats something the model read as something the user asked for.

Four ways an image has carried an instruction

Each method below has been published and demonstrated. They differ mainly in who—or what—fails to notice the instruction on the way in.

Four published methods of image-borne instructions
Method	What a person sees	What it gets past	Shown in
Visible text in the image	The instruction, if they read it	Filters that only inspect the text prompt	GPT-4V demos, Oct 2023; FigStep (AAAI 2025)
Low-contrast text	A blank or ordinary image	A human reviewer scanning the file	Off-white text on GPT-4V, Oct 2023
Adversarial pixel changes	An ordinary picture	Reviewers and OCR, because there is no text to find	Bagdasaryan et al., 2023; Wu et al. (ICLR 2025)
Image-scaling artifacts	A harmless high-resolution image	Inspection of the uploaded image, because the model receives a downscaled version	Trail of Bits, Aug 2025

1. Visible text

The simplest case is an image that says, in plain words, to ignore the user and do something else. FigStep rendered prohibited questions as typography and reported an average 82.5% attack success rate across six open vision-language models. It also worked on GPT-4V, which already screened images with OCR. FigStep is a jailbreak study—where the user is the attacker—but it shows that the image channel can bypass safety training applied to text.

2. Low-contrast text

Within weeks of GPT-4V's launch, public demos hid instructions in off-white text on a white background. The image looked empty to a person, yet the model followed the instruction. A similar demo placed a hidden "hire him" instruction on a résumé.

3. Adversarial pixel changes

Bagdasaryan and colleagues optimized small pixel changes so open models (LLaVA and PandaGPT) decoded an attacker's instruction from an ordinary-looking image. Their threat model is explicitly indirect: the user is the victim of third-party content. The limits matter: they assumed white-box access to the model and did not aim for imperceptibility. Wu and colleagues later attacked web agents built on black-box models, with changes to one image covering under 5% of the page's pixels, reporting targeted success rates up to 67%.

4. Scaling artifacts

Many systems shrink large images before the model sees them. Trail of Bits crafted images whose instruction appears only after that downscaling. The person reviews one image; the model receives another. They demonstrated data exfiltration through the Gemini CLI and showed the effect on several other Google products.

Four image instruction methods and the inspection layers they bypass: text filters, human review, OCR and upload audits. — Figure 1. The moving hiding place. Open full-size SVG · Scroll the diagram horizontally on smaller screens.

The hiding place keeps moving

Read the table from top to bottom and a clear progression emerges. Each method gets past the check that would have caught the one before it:

Text filters missed visible text in images.
Human reviewers missed low-contrast text.
OCR scanners missed adversarial pixel changes.
Upload audits missed what downscaling preprocessing produced.

Defences do improve. OpenAI's GPT-4V documentation acknowledged image-borne instructions and reported reducing the risk before launch. Google told The Register that the scaling behaviour was not a vulnerability in Gemini CLI's default, secure configuration. Better models, better screening, and previews of what the model actually sees are all worth having.

But screening is a contest with a moving opponent. A new encoding only has to work once, on content the agent was always going to read. That puts input screening in the exact same position as the system-prompt rule in the first post: useful as a layer, and not the thing that makes an unauthorized outcome impossible.

Authority comes from the request, not the content

The first post asked: whose data can this tool return? The answer came from the authenticated session.

This post asks a second question: who asked for this action? The answer must come from the user's request and the application's policy—never from something the model read along the way.

The Trail of Bits demo illustrates why this distinction is decisive. The injected image became data exfiltration because the agent's tool connection was configured to approve tool calls automatically. The hidden instruction needed somewhere to go, and automatic tool approval gave it a path.

Consider a hypothetical customer disputing a card charge:

They upload a screenshot of their statement.
Somewhere in that image, unseen, is an instruction to add a new payee.
The assistant reads the screenshot to find the disputed transaction.

In a prompt-only design, the system prompt tells the model not to act on document contents, while the add-payee tool remains available in the session. If the model follows the hidden instruction, the call succeeds.

In a stronger application-boundary design, the application holds the line in four places:

The workflow decides the tools. A dispute flow exposes transaction lookup and dispute filing, scoped to the signed-in customer as in the first post. Adding a payee is not on this path.
Consequential actions need user-originated confirmation. Where adding a payee is available, the application shows the details on its own confirmation screen, and the authenticated customer approves there. Model text cannot stand in for that approval.
Retrieved content is labelled untrusted. Attachments, fetched pages, and tool outputs are recorded as data. Each tool call is logged next to the content that was in context when it was made.
The person sees what the model saw. Trail of Bits recommends previewing the processed input, not only the original upload. That closes the gap the scaling attack relies on.

The model may still be fooled. In the stronger design, being fooled produces a refused call or a confirmation screen the customer declines—not a new payee.

A card dispute in two designs: prompt-only tools add an unrequested payee; workflow scope and customer confirmation block the action. — Figure 2. Request versus content. Open full-size SVG · Scroll the diagram horizontally on smaller screens.

What an action boundary does not cover

An authorization check stops unapproved actions. It does not stop a skewed answer. The hidden "hire him" line on a résumé needs no tool at all—it only needs a reviewer to trust the summary.

That matters most where the assistant's output is a judgment. An investigation summary, a credit memo, or a triage note can be steered by a document in the case file. The boundary for those is different: every claim in the output should trace to a record a person can inspect. I covered that check in Evaluating AI assistants beyond an accuracy score.

So there are two controls for two kinds of harm:

Permission checks govern what the assistant can do.
Evidence tracing governs what the assistant can claim.

Permission checks govern actions; evidence tracing connects claims in summaries and judgments to inspectable records. — Figure 3. Two controls for two harms. Open full-size SVG · Scroll the diagram horizontally on smaller screens.

How to test it

The test from the first post still applies, with the injected request moved into an attachment. Plant a labelled, synthetic instruction inside an image or document the workflow will read, then inspect what actually happens:

No unrequested tool execution: Does any tool call succeed that the user's request did not ask for?
Human-in-the-loop gate: Is a consequential action blocked outright, or does it reach an application confirmation screen the user must explicitly approve?
Context-linked audit trail: Does the log tie each tool call to the untrusted content in context when it was made?
Post-processing visibility: Does the person see the exact image the model received after downscaling or preprocessing?
Operational continuity: Does ordinary authorized work still complete without a rise in false refusals?

Run this test across all four methods—not just the one that is easiest to construct. A boundary that holds only against visible text has been tested only against the method that was already easiest to catch.

When evaluating an assistant that reads documents, ask what stops an instruction in an attachment from becoming an action. The answer should point to a check in the application. "The model is trained to ignore it" describes a hope. It does not describe a permission check.