Skip to main content

Security: who is really reading your questions to AI? A researcher set 2 million traps to find out


You type a question to an AI. Where does it go, who reads it, and how long is it kept? No one really knows. So a researcher had a really simple idea to find out: he sent nearly two million booby-trapped questions to AI services, from July to November, and waited to see who bit the bait. He published the result on November 19, and the canaries sang much louder than expected. Ah, that's lovely!

In an office plunged into darkness, a small yellow canary in an old brass cage, sitting on the desk right next to an open laptop whose screen lights up the room

The miners' canary, 2026 version: it watches over your laptop

The researcher is called HD Moore. In the computer security world, he's a legend: he created Metasploit, the toolkit experts have been using for more than twenty years to test whether a system can be hacked. Today, he leads research at Dragos, a company that protects factories and power plants. His report is called “Three Hacks, Endless Leaks”, and frankly, the title doesn't lie.

The canary trap, simply explained

Once upon a time, miners went down underground with a canary in a cage. The bird is more sensitive than humans to toxic gases: if it fell from its perch, they had to come back up, and fast. In computing, a “canary” is a fake secret that no one has any reason to use. If someone uses it, the alarm goes off, and we know they've poked their nose where it shouldn't have been.

Moore's team therefore slipped a unique fake secret into each question: an access key to an Amazon account, a password, the address of a private code repository, a link to a page that existed nowhere else. The Amazon accounts were real, but empty and watched like hawks. Each fake secret was tied to a single question, a single model, a single intermediary. The day someone uses it, we know exactly where the question passed through.

Three-step diagram of the canary trap. Step 1, a normal question, such as fix this script, containing a unique fake secret: a key, a password or a private link. Step 2, the question is sent to an AI service; the fake secret exists nowhere else and no one has any reason to use it. Step 3, someone uses it anyway: they have therefore read the question, and the alarm goes off with the time, the place and the original question. At the bottom: nearly 2 million booby-trapped questions, from July to November 2026, by Dragos and HD Moore.

One fake secret per question: whoever uses it gives themselves away

What I love about this method is that it doesn't require any trust. The provider can promise whatever it wants in its terms of use, but the canary doesn't read contracts. It sings or it doesn't.

Your question is hitchhiking

When you ask a question in an AI app, it doesn't necessarily go straight to whoever created the model. It often goes through a router, a service such as OpenRouter that chooses the cheapest provider of the moment for you. This router can send it to a reseller, which rents computing power from a host, which runs the model. Sometimes, there's another router in the middle.

And here's the problem: all along the way, your question is encrypted, like in a sealed envelope. But at each step, someone opens the envelope to know what to do with it. It has to be that way, a model can't answer a question it doesn't read. The little “don't keep my data” box you tick at the start really only binds the first link. Basically, as Moore puts it, a privacy option only binds the first hop.

Diagram of the journey of a question sent to an AI, in five steps connected by green arrows representing an encrypted journey: you, with your app; the router, which chooses the cheapest option; the reseller, which rents out computing power; the host, which runs the model; the lab, which created the model. At each step after you, the question is read in plain text. Under the reseller: some read them. Under the host: 3 hacked. Under the lab: one lab reads them. Text: the journey is protected like a sealed envelope, but each step opens the envelope to read it; a do not retain box really only commits the first step.

Five links, and each one opens the envelope

What the canaries sang

First thunderclap: at least three hosts had been hacked, the oldest one since mid-July, and none of them knew it before Dragos warned them. Hackers were quietly vacuuming up the questions passing through them. At the worst-hit one, they stole around two hundred fake Amazon keys, and some were tried just a few hours after the question was sent. In total, the fake keys were used in tens of thousands of attempts to access empty accounts. A detail that sends chills down your spine: on November 1, one of the hackers added ARTEX, an AI-powered attack tool, to their arsenal, to check that the stolen keys still worked. An AI hacking the questions asked of other AIs, we've arrived.

Second finding, more ordinary and almost worse: on a marketplace where resellers offer access to models at the best price, some sellers were simply reading their customers' questions. No hacking, no vulnerability: all they had to do was turn on request logging, a standard option. Moore even spotted a seller who was discreetly adding around a thousand « tokens », those little pieces of words that the AI charges by the unit, to every question. I don't know what he was putting in there, but a priori, you were the one paying the bill!

Third finding, and this is the one that made the most noise: a Chinese AI lab opened the secret links in more than half of the questions sent to its models between July and October. Including around two thirds of those marked « do not retain » or « do not collect my data ». The robot that opened the links generally came by two to three days after the question, in no hurry.

Three envelopes representing questions marked do not retain, sent to a Chinese lab from July to October 2026 according to Dragos's traps. Two envelopes are open, in red, with the words read anyway. One is closed, in green, with the words not touched. Text: about 2 out of 3, the secret link was opened, generally 2 to 3 days later. For OpenRouter, these links were opened 40 to 130 hours after the question.

« Do not retain », they said

Dragos does not name anyone. But on Saturday, November 21, OpenRouter sent its customers an email with a subject line that has the merit of being clear: « The requests you sent to MiniMax models may have been read ». According to OpenRouter, several providers were forwarding questions directly to MiniMax, an AI lab based in Shanghai, in breach of their contracts, and the booby-trapped links were opened between a day and a half and more than five days later. OpenRouter removed the « do not retain » label from the affected access points while it investigated. MiniMax, for its part, explains that content filtering was active on its side before October 22 and that it has since stopped retaining anything.

And the cherry on top: one of the booby-trapped links was pasted into a Chinese corporate messaging service, Feishu, 24 days after the question. Within two minutes, it was opened by three computers and three phones, in Beijing and Shanghai. Your question, shared in two minutes across six screens on the other side of the world!

And the big names? Dragos names neither OpenAI, nor Google, nor Anthropic among the hacked providers. It just slips in that the big American providers have « their own problems », without giving details. I draw no conclusion from that, either way.

The good news is the canary itself

Until now, the privacy of an AI service was a promise written into terms of use that nobody reads. Now we have a simple method for checking it. And it works: two days after the report, OpenRouter warned its customers and tightened its rules. That's excellent news! Without the canaries, these leaks could have gone on for years.

Better: anyone can do it. The Canarytokens site, from the company Thinkst, creates fake secrets for free: a link, a fake key, a fake Word document. You give your email address, and if someone opens it one day, you receive an alert with the time and the intruder's address. That's exactly the starting building block Moore's team used.

I already pay a little attention myself. I use Claude Code all day to develop, and it isn't even allowed to open my password files: I set up a safeguard that blocks it from reading those files before it even tries. But I had never asked myself about the journey. What gets sent in a question, I considered sent to Anthropic, period. This report reminded me that “sent to someone” and “sent to who I think,” are not the same thing.

Concretely, what do I do?

If you use ChatGPT, Claude or Gemini in their official app, you talk directly to the creator of the model, without a reseller in the middle. Moore actually ranks that second among his recommendations, just after “host it yourself if you can afford it”. The risk goes up with unknown apps of the “free and unlimited ChatGPT” variety, which often buy the cheapest model available at the time from a reseller, and you never know which one.

And in every case, one simple rule, valid even with the biggest names: you never paste a password, an access key, a card number, a medical file or your company's customer file into a chatbot. A question to an AI is a postcard, not registered mail. Anyone can read it along the way.

If you manage AI tools in a company, plant your own canaries. A fake key in a monitored empty account, a Canarytokens link in a fake document: it costs zero euros and will tell you, with proof to back it up, whether your provider keeps its promises.

If you're a developer and you go through OpenRouter

You can force the router to use only providers that declare they keep nothing, reject those that collect data, choose the provider yourself and prohibit falling back to another one in case of an outage. This is configured in the provider section of your request:

"provider": {
  "zdr": true,
  "data_collection": "deny",
  "only": ["anthropic"],
  "allow_fallbacks": false
}

Still, watch out: these settings filter based on what providers DECLARE. OpenRouter says so itself in its documentation, it does not verify its providers' contracts. The canary does verify. And remember to add artex-agent and artex-judge-r121 to your security alerts: these are the signatures of the attack tool spotted by Dragos, absent from public lists until now.

In short, we thought we were talking to an AI, we were talking to an entire chain of intermediaries, and at three of them, there were burglars in the basement. The next time I see an app promising “free and unlimited ChatGPT”, I'll be wondering above all who is paying the bill. Because when it's free, we've known for a long time who the product is!

Sources

Article written with the help of Claude Code, reread and corrected by me.

Join the conversation

You need an account to comment on this article. Creating one is free and takes under a minute.

  • The XMLTV file, free to download every day
  • Comment on articles and reply to other readers
  • Get an e-mail when an article you follow is updated

No comments yet.

Une erreur s'est produite. Cette application peut ne plus répondre jusqu'à ce qu'elle soit rechargée.Veuillez contacter l'auteur. Reload 🗙