Microsoft reveals 'Whisper Leak' attack that identifies AI conversation topics in encrypted traffic

Microsoft has revealed details of a new side-channel attack targeting remote language models that could allow a passive attacker capable of monitoring network traffic to gather details about the topics of conversation with the model, despite encryption protection under certain circumstances.

This leak of data exchanged between humans and streaming-mode language models could pose a serious risk to the privacy of consumer and corporate communications, the company noted. The attack is codenamed Whisper Leak.

„"Cyberattackers who have the ability to observe encrypted traffic (for example, a government entity at the ISP level, someone on the local network, or someone connected to the same Wi-Fi router) could use this cyberattack to infer whether a user's prompt is about a specific topic," said security researchers Jonathan Bar Orr and Jeff McDonald, with the Microsoft Defender Security Research Team.

Put another way, the attack allows an attacker to observe encrypted Transport Layer Security (TLS) traffic between a user and an LLM service, extract sequences from the size and timing of the packets, and use trained classifiers to infer whether the conversation topic matches a sensitive target category.

The Whisper Leak Attack and Data Streaming

Model streaming in large language models (LLMs) is a technique that allows data to be gradually ingested while the model generates responses, rather than having to wait for the entire output to be computed. This is a key feedback mechanism, as some responses may take time, depending on the complexity of the prompt or task.

The latest technique demonstrated by Microsoft is significant, not least because it works despite the fact that communications with artificial intelligence (AI) chatbots are encrypted with HTTPS, ensuring that the content of the exchange remains secure and cannot be manipulated.

Many side-channel attacks against LLMs have been developed in recent years, including the ability to infer the length of individual tokens in cleartext from the size of encrypted packets in stream model responses, or by exploiting timing differences caused by caching of LLM outputs to perform input data theft (known as InputSnatch).

Whisper Leak builds on these findings to explore the possibility that "the sequence of encrypted packet sizes and the arrival times between them during a response from a streaming language model contains enough information to classify the topic of the initial prompt, even in cases where responses are streamed in groups of tokens," according to Microsoft.

Inspection results and consequences

To test this hypothesis, the Windows maker said it trained a proof-of-concept binary classifier that was capable of distinguishing a prompt on a specific topic from the rest (i.e., "noise") using three different machine learning models: LightGBM, Bi-LSTM, and BERT.

The result is that many models from Alibaba, DeepSeek, Mistral, Microsoft, OpenAI, and xAI were found to achieve scores above 98%, making it possible for an attacker observing random chatbot conversations to reliably tag that particular topic. The models from Google and Amazon, in comparison, showed greater resilience, likely due to token batching, although they are not completely immune to the attack.

„"If a government agency or internet service provider were monitoring traffic to a popular AI chatbot, they could reliably identify users asking questions about specific sensitive topics – whether it's money laundering, political dissent or other monitored topics – even though all the traffic is encrypted," Microsoft said.

Improvements, measures and new assessments

The researchers found that the effectiveness of Whisper Leak could improve over time as the attacker collects more training samples, making it a practical threat. Following responsible disclosure, OpenAI, Mistral, Microsoft, and xAI have already implemented mitigations.

„"Combined with more sophisticated attack models and the richer patterns available in multi-turn conversations or multiple conversations from the same user, this means that a cyberattacker with patience and resources can achieve higher levels of success than our initial results suggest," the text added.

One effective countermeasure, developed by OpenAI, Microsoft, and Mistral, involves adding a „random variable-length text sequence“ to each response, which in turn masks the length of each token to make the side channel meaningless.

Microsoft also recommends that users who are concerned about their privacy when interacting with AI chatbots avoid discussing highly sensitive topics when using untrusted networks like public Wi-Fi, use a VPN for an extra layer of protection, use LLM models that are not in streaming mode, and switch to providers that have mitigation measures in place.

The disclosure comes amid a new assessment of eight open-weight LLM models from Alibaba (Qwen3-32B), DeepSeek (v3.1), Google (Gemma 3-1B-IT), Meta (Llama 3.3-70B-Instruct), Microsoft (Phi-4), Mistral (Large-2 aka Large-Instruct-2047), OpenAI (GPT-OSS-20b), and Zhipu AI (GLM 4.5-Air), which were found to be highly susceptible to adversarial manipulation, especially when it comes to multi-turn attacks.

„These results highlight the systemic inability of current open-weight models to maintain safety precautions over prolonged interactions,“ say Cisco AI Defense researchers Amy Chang, Nicholas Conley, Harish Santanalakshmi Ganesan, and Adam Suanda in an accompanying paper.

„"We estimate that alignment strategies and lab priorities significantly influence resilience: capability-focused designs, such as Llama 3.3 and Qwen 3, demonstrate higher susceptibility to multi-reply attacks, while safety-oriented designs, such as Google Gemma 3, show more balanced performance."“

These findings indicate that organizations adopting open source models may face operational risks in the absence of additional security safeguards, adding to a growing body of research exposing fundamental security weaknesses in LLMs and AI chatbots following the public debut of OpenAI ChatGPT in November 2022.

This makes it critical for developers to implement adequate security controls when integrating such capabilities into their workflows, fine-tune open-weight models to be more resilient to breaches and other attacks, conduct periodic AI „red team“ assessments, and implement strict system prompts that are aligned with defined use cases.

Source: https://thehackernews.com/