What is the ‘second invoice’ paid by AI users, the silent cost most consumers do not even notice
Earlier this month, Palantir's Alex Karp said in an interview that 'something has gone completely wrong' in how AI is sold and the chief executives he meets are livid watching their competitive edge migrate to model companies. Days later, Microsoft's Satya Nadella gave the fear a name: the ‘Reverse Information Paradox’.
Two rival AI company CEOs arrived at the same fear within a fortnight. On July 1, Palantir's Alex Karp told CNBC that "something has gone completely wrong" in how AI is sold and the chief executives he meets are privately livid watching their competitive edge migrate to the model companies. Eleven days later, Microsoft's Satya Nadella published an essay on X, viewed more than eleven...
Two rival AI company CEOs arrived at the same fear within a fortnight. On July 1, Palantir's Alex Karp told CNBC that "something has gone completely wrong" in how AI is sold and the chief executives he meets are privately livid watching their competitive edge migrate to the model companies. Eleven days later, Microsoft's Satya Nadella published an essay on X, viewed more than eleven million times, giving the fear a name: the ‘Reverse Information Paradox’. His claim: with AI, you "pay for intelligence twice" — once in money, once in the proprietary knowledge you must reveal to make the model useful for you.
Think of a cook you pay a monthly salary to. She memorises your family recipes with each meal she makes for you and then uses or incorporates them in every other house she works in. The salary you pay her is her first payment; your home recipes are the second.
Interestingly, Nadella, in his essay, quoted Karp. This was not coincidence; it was convergence.
Economist Kenneth Arrow, in 1962, described the original information paradox: a seller of information cannot prove its value without revealing it and once revealed, the buyer has it for free. AI, however, flips the trap onto the buyer. To get useful answers, you must feed the machine your context, judgement and evaluations. The better you want it to perform, the more you must reveal (and lose to it).
Arrow had a remarkable 1962. That same year he gave economics "learning by doing": capability compounds through practice. He was right and AI genuinely accelerates it. But your doing so now produces two learners. You practise and the AI system observes. With AI, a rented intelligence, the compounding (in this case, information) also accrues to the vendor who owns the model.
Also read: Why, despite growing market, IVF continues to pose a challenge for many
Most users think the leak is the file they upload. Not really. The prompt itself leaks.
Ask a chatbot to redraft a settlement letter for a defaulting SME borrower in Coimbatore and you have revealed your process and problem without sharing a single document.
Corrections leak even more. Explain to the bot why the model's answer is wrong or question its logic and you hand over the concentrated form of your judgement — the knowledge a competitor could never have bought.
Memory leaks silently, assembled across months of ordinary use. Connected apps leak your emails and documents without your typing a word. Even with training (the system to make the AI chatbot more effective) option switched off, the product still observes what you ask, how you ask, what you accept, how you refine, why you retry. The off-switch closes a tap. It does not end the relationship.
Memory leaks silently, assembled across months of ordinary use. Connected apps leak your emails and documents without your typing a word. Photo: iStock
I tested this. Learning loops switched off, I asked the three AI assistants I use daily a simple question: what do you know about me? One returned a dossier naming senior executives at my client firms, then volunteered something I never asked for — a judgement on where my strategy is weak.
Another, I use three times as much, answered thinly at first; when challenged, it surfaced information far deeper than its first admission. By design perhaps?
The third confidently described a company I do not run, stitched from the one workstream it had seen. Nothing was breached. This is just use.
The machine intelligence certainly gathered and registered much about me.
I wondered how much the people around me knew about the machines. We put three questions to a cross-section I trust: CEOs, senior professionals across diverse industries, risk heads, medical leaders, my auditor, students of varying seniority. Nearly all knew that consumer AI plans train on chats by default. The exceptions were telling.
A global pharma leader, a senior risk professional at a global bank, and my auditor's office underestimated the propensity of the leak. The Class 12 student knew; his school taught him that models must keep training to improve. The generation entering the workforce knows what the generation running it does not.
Their worries mapped the terrain of my experiment.
A sales leader at a Gulf telecom major on AI said there was "no problem unless they put a confidential file or data into it". The prompt, as we have seen, leaks long before the file does. A freelance PhD student, answering what worries her, said, "nobody wants every rabbit-hole they go down to be remembered and fed into a pattern of themselves". A vice-president at a pharma manufacturing major, meanwhile, responded, "I don't want to be under surveillance by my AI assistant, connecting dots when there might be nothing to connect". The head of AI at a financial services major in Mumbai said, "If your bank RM uses what he knows about you in conversation, you appreciate it. If you see the diary page with every detail written down, it is scary." To a hospital managing director, "It feels less like a privacy concern and more like the AI overstepping". And my auditor, whose firm has served small and medium enterprises for decades, speaking for the herd default on anything worrisome in the AI remembering his details, responded, "Absolutely no. On the other hand, it would be extremely helpful".
The off-switches tell their own story, beginning with their names. One is called "Improve the model for everyone". Another says "Help improve our AI models". A third files its memory under "Personal Intelligence". Not one contains the word training. The defaults finish the story: on a hundred-dollar-a-month premium plan, the toggle came pre-selected to on, alongside a location setting most users discover only because there is a switch to turn it off. And the third AI assistant, preinstalled on Android phones that are nearly all of India's smartphones, offers "Manage and delete" as a link to a help article, not a control. The most accessible AI has the least accessible exit.
Neither Nadella nor Karp pretends to be a neutral witness; each man's remedies point back to his own product stack: keep your company's edge at home (with us in the loop).
With Karp, the irony runs deeper. Palantir counted the CIA's venture arm among its earliest investors and built its fortune fusing data for intelligence agencies, militaries and immigration enforcement: the business of extracting meaning from citizens, migrants and adversaries. The loudest recent voice on business data sovereignty also runs one of the world's largest data-mining engines.
But mark the distinction, because it is the whole point. Palantir's extraction is contracted: governments sign for it, pay for it, and answer for it to congressional oversight, courts and voters. Many find that work objectionable, but the government consented to this on the record.
The leak this article describes has no contract, no signature and no line item. It happens through the prompt, quietly, at a scale no procurement office ever reviewed.
Another paradox sits in the industry's own paperwork: the laboratories claim fair use to train on the public web while barring customers from using their model outputs to train anything that competes, open source included. A one-way learning loop towards the model, benefitting the vendor. That is not a sales pitch; it is the business model.
India has seen this movie before. The brain drain of the 1990s took our engineers abroad and their learning compounded in someone else's economy. The AI-era version is quieter: engineers stay in Chennai and Pune and the judgement emigrates one prompt at a time.
Infosys, TCS and Wipro crossed three lakh Microsoft Copilot seats in June. And who holds the institutional knowledge of the world's largest enterprises in trust? The same three: paradox again? Nandan Nilekani calls sovereign, localised AI data strategic infrastructure; Forrester's Dario Maisto counsels "minimum viable sovereignty", protection matched to the workload. Between those poles sits every Indian enterprise's real decision.
Regulators move where the law lets them: RBI's June draft wants to kill switches in bank AI and the Digital Data Protection (DPDP) Act, 2023, carries penalties to protect personal data. A bank's settlement procedure or a retailer's pricing heuristic is not personal data and it can walk out through the AI exhaust with no law watching.
Enterprises will fight this with lawyers and trust boundaries in contracts. You and I negotiate nothing; we take the herd default, paid in a currency that appears on an AI valuation.
As entrepreneur and content creator Ankur Warikoo puts it, enterprises will get boundaries; individuals will not. The second invoice arrives every day we prompt. The least we can do is read it.
