Google researchers have described a new way to learn what people use AI chatbots for while protecting the people in the data. Their paper, “CALLIOPE: Private Insights into AI Use, without Embeddings”, is dated 20 October 2026 on arXiv. It reports better results than Urania, an earlier private method, on public test datasets. On the largest dataset, it puts the cost of one run at approximately $175, against approximately $40 for Urania: about 4.4 times as much, by our arithmetic.
The 14 authors list Google Research, Google and Google DeepMind as affiliations. In plain words: AI labs want a map of the topics people bring to chatbots, without anyone being able to tell from the map whether a given conversation was in the data. The abstract says the method “provides better utility over existing (embedding-based) methods in the DP setting, especially in the high privacy regime”.
A vertical news Short narrated from the published headlines and opening lines of this piece and related FourWeekMBA pieces.
Business Pill · Embeddings Explained
An evergreen Business Engineer explainer on embeddings, the method Calliope avoids, and how a machine compares meaning. It explains the general idea only and does not discuss this paper.
The key insight: As we read it, the paper puts a price on privacy: it reports a more useful private map of AI use than Urania, at approximately $175 against $40 for one run on its largest test dataset.
The usual approach turns each conversation into a vector embedding, a list of numbers, and groups nearby vectors into clusters. Clio, the system Anthropic described in 2024, and Urania, a private method published in 2025, both work this way after using an AI model to summarise each conversation. The paper calls this an “arguably convoluted and lossy pipeline”.
Calliope drops the embeddings and works only in text. An AI model reduces each document to a few keywords. A privacy step then keeps only keywords common enough across the data to be safe to show. The model proposes topics from those keywords and sorts every document into one, repeating the process to build a hierarchy of topics and sub-topics.
The privacy standard is differential privacy (DP). In the paper’s words, it guarantees that the output “does not significantly reveal the inclusion of any specific document in the corpus”. A setting called epsilon controls how strict that guarantee is; smaller values mean stricter privacy. The paper tested epsilon at 0.1, 1 and 10.
Tokens used by Calliope as a multiple of the tokens used by Urania, per test dataset, in the non-private setting at the deepest topic tree tested, with gemini-3.5-flash. Multiples are our arithmetic from the paper’s token counts. Source: Cohen et al., CALLIOPE, arXiv 2610.28797, 20 Oct 2026 (Google), Table 1.
What Was Posted on X
New Google DeepMind paper finds that having an LLM build topic trees from keywords, not embeddings, gives clearer topic maps under strict privacy.
Teams want to know what users ask chatbots without exposing anyone. The usual private approach clusters embeddings with added noise, and under strict privacy it returns too few clusters with weak names.
CALLIOPE has an LLM tag each document with a few keywords and adds noise so only common keywords survive. The LLM then proposes topics from them and sorts documents into those topics.
At the strictest privacy setting tested, human raters preferred CALLIOPE over the embedding-based URANIA on all 3 datasets, by up to 76% versus 18%.
The method uses 2 to 5 times more tokens. For privacy-safe usage reports, start with LLM-built topics and budget for that cost.
Rohan Paul posted this summary of the paper on X on 28 October 2026. His post says that at the strictest privacy setting tested, human raters preferred CALLIOPE over the embedding-based URANIA on all 3 datasets, by up to 76% versus 18%, and that the method uses 2 to 5 times more tokens. This piece quotes his closing advice on budgeting for that cost.
What the Tests Showed
The team ran Calliope against its own implementations of Clio and Urania on six public datasets, using Google’s gemini-3.5-flash as the language model. According to the paper, Urania’s quality “degrades rapidly in the high privacy regime”, where it “is unable to produce a reasonable number of clusters and cannot produce good descriptions for them”.
Overall. “Even when privacy parameters are less restrictive Calliope overall still outperforms Urania”, the paper says.
Labels and names. Calliope outperformed Urania on two measures that compare clusters with known labels (NMI and Adjusted Rand Index) and gave “much better results” on a test in which a judge must pick, from several cluster descriptions, the one that fits a sample of documents.
Where they tie. On measures of cluster shape alone, the two “perform similarly”, and on spotting the odd document out they were on par in private settings.
Human raters. In side-by-side comparisons, the paper reports that “Calliope outperforms Urania”.
Rohan Paul, who shared the paper on X on 28 October, summarised the trade-off: “For privacy-safe usage reports, start with LLM-built topics and budget for that cost.”
The Price of Privacy, in Tokens
Approximate cost of one run on MASSIVE, a dataset of over 1 million utterances, with Gemini 3.5 Flash: about $40 for Urania and about $175 for Calliope, per the paper. 4.4x is our arithmetic. Source: Cohen et al., CALLIOPE, arXiv 2610.28797, 20 Oct 2026 (Google).
The paper names the main downside itself: “the requirement to run LLM on each data point multiple times”. Its token table, for the non-private setting at the deepest topic tree tested, shows Calliope using more tokens than Urania on every dataset. By our arithmetic, the ratio runs from 2.3 times on IMDB movie reviews to 4.7 times on MASSIVE, 576 million tokens against 123 million.
The authors argue the extra cost is modest. They write that “this increased price is not that steep even for large dataset”, citing approximately $40 for a Urania run on MASSIVE with Gemini 3.5 Flash and approximately $175 for Calliope. They suggest sharing cached prompt prefixes, or using a smaller model for the sorting step, which they call the most expensive part.
Connecting the Coverage
The paper lands on a day of news about who gets to see AI usage. Anthropic launched smart reports for Claude Enterprise that analyse a sample of a team’s transcripts and group the work into workstreams, as we reported on 28 October. OpenAI’s Signals data showed questions about prices rising to 0.69% of US work-related ChatGPT messages, as we reported on 27 October.
Token prices bear on the cost question: a Ramp index put them roughly 35% lower since the end of June, as we reported on 28 October. The Calliope paper cites both Anthropic’s Clio and OpenAI’s “How people use ChatGPT” study as examples of research on chatbot conversations.
The Structural Read
As we read it, the method matters to any company that reports what people do with a chatbot; Anthropic’s new smart reports, which analyse a sample of a team’s transcripts, are one such product.
The paper says private clustering of embeddings tends to “suffer from sharp utility cliffs, where performance degrades precipitously as the privacy budget tightens”.
The authors say the extra cost is not steep, and they list ways to cut it, such as using a smaller model for the step that sorts documents into topics.
Cohen et al., arXiv 2610.28797
“our framework leverages LLM capabilities directly for the entire pipeline”
Three Implications
FOR AI LABS The paper reports that Calliope still produced useful topic maps at strict privacy settings, where Urania could not produce a reasonable number of clusters.
FOR ENTERPRISE BUYERS The paper says its privacy protection applies per document, not per person, and names person-level privacy as future work.
FOR THE COMPUTE BILL The paper’s token table shows Calliope using 2.3 to 4.7 times Urania’s tokens, by our arithmetic, in the non-private setting.
The framework describes the governance layer of the AI stack: “It is where decisions are made about who gets access to frontier intelligence, under what conditions, and for which purposes.”
As we read it, usage data raises the mirror question: who gets to see how people use AI, and under what conditions. Calliope is one answer from Google researchers. The framework was published in June 2026 and does not discuss this paper or usage data.
What Is Not Established
The paper is a preprint. Its tests used public benchmark datasets, such as online banking customer-service queries, short multilingual requests labelled by intent, 1987 Reuters financial news and IMDB movie reviews, rather than real chatbot conversation logs. The paper does not say whether Google uses Calliope on Gemini conversations. Clio and Urania were run as the team’s own implementations, not the original systems.
The authors list limits of their own. Privacy is protected per document, not per person, though one user may have many conversations. The method works on static batches, while “real-world conversation logs are dynamic streams”. And it “does not inherently solve upstream risks such as data centralization, vulnerabilities during model inference, or third-party processing”.
This Week In The AI Supercycle
Execution Fault Lines, Concentrated Demand, and Geopolitical Firewalls
This week’s issue: the AI build is not short of demand for its parts. It is short of cheap money, and its end revenue rests on a narrow base of large, often venture-funded buyers.
A Business Engineer analysis of the governance layer of the AI stack: who gets access to frontier intelligence, under what conditions, and for which purposes.
A Google paper dated 20 October 2026 describes Calliope, a method that builds a private topic map of AI use with a language model instead of embeddings. On public datasets it reports better results than Urania, especially under strict privacy, at a higher compute cost: approximately $175 against $40 on the largest dataset, and up to 4.7 times the tokens, by our arithmetic.
A note on sourcing. Results, figures and quotations come from the paper “CALLIOPE: Private Insights into AI Use, without Embeddings” (arXiv 2610.28797, dated 20 October 2026), except a quotation from Rohan Paul’s post on X of 28 October 2026. Token and cost multiples are our arithmetic from the paper’s figures. We did not contact the authors, Google, Anthropic or OpenAI.
Gennaro Cuofano is a CRO and tech executive who has worked in AI since late 2015, starting with bringing NLP, natural-language generation, voice and chatbot products to the marketing industry. His background is in law and finance: he holds a Master's degree in Law and an International MBA with an emphasis on corporate finance (LUISS Business School and the University of San Diego, 2012), and worked as a financial analyst at a real-estate investment firm in San Diego and as an assistant controller. His work focuses on business model strategy, business engineering and, more broadly, structural analysis: how companies make money, read from their own filings. He created FourWeekMBA and leads research there and at The Business Engineer, his newsletter on AI and business strategy, with over 95,000 subscribers and more than 1,000 published analyses. His writing has also appeared on Entrepreneur (for example, How to Digitize a Family Business in Europe) and HackerNoon. Find him on LinkedIn and Substack. See how we source.
