Some companies whose employees pasted passwords or API keys into Notion AI have a cleanup job. Notion is telling the owners of fewer than 1% of its workspaces to treat certain AI chat transcripts as exposed and to rotate any credentials in them, after attackers reached data held by Fireworks AI, an inference provider Notion used behind the scenes.
Notion’s own systems weren’t breached, the company said. The data sat with a vendor that ran behind Notion’s product. A breach at an AI provider “can reach businesses that use its technology through someone else’s software,” VentureBeat wrote. Notion has disabled the affected feature and stopped sending traffic to Fireworks.
Notion used Fireworks for two jobs, it said in its security advisory. One was classification work for Insights, a feature that shows customers how their teams use AI. The other was general inference to serve open-weight models inside Notion. The exposed data was Notion AI chat transcripts used for Insights, and for most affected workspaces those chats were “unattributable,” meaning they carried no label for who wrote them or which workspace they came from, and they did not include tool call results. “Beyond what’s in the chat transcript itself, there’s no information tying the chat back to workspaces, users, or companies,” Notion said. The chats themselves, though, can contain identifying details, The Register noted.
Fireworks first told Notion on the evening of Nov. 2 that a small set of data had been accessed, and that set turned out to contain Notion AI synthetic testing data and no customer data. By the next morning Notion had disconnected its systems from Fireworks and deleted all of its data there. On Nov. 6, Fireworks warned that customer data might be involved, and two days later it confirmed the final impacted dataset. Notion began contacting affected customers on Nov. 9, and on Wednesday it revoked exposed Notion API keys that its investigators found in affected user messages.
Fireworks, which The Record says has more than 10,000 customers, has disclosed two October incidents. The first, reported Oct. 9, involved stored secrets such as those used in evaluation jobs. Then a third-party cloud provider that hosted Fireworks’ R&D machines was hit on Oct. 23, and the next day the intruder used it to reach Fireworks development machines and “run an unauthorized workload that led to additional customer data impact,” the company said in a Nov. 9 bulletin. The data involved includes resource metadata, billing details such as addresses and invoice histories, and, for some customers, incomplete model weights and datasets. Fireworks said it doesn’t store payment card numbers and has not detected any impact to its Inference, Training SDK and Nexus products, which are zero data retention by default.
Lin Qiao, Fireworks’ co-founder and chief executive, said the intruders did not appear to be hunting for customers’ models. “From what we see attackers are not after weights,” she said, as quoted by The Register. “Most downloads were partial and stopped.” The company has brought in Palo Alto Networks’ Unit 42, notified law enforcement and disabled self-serve dataset uploads and downloads for Batch Inference and dataset-based training while it investigates. The widened disclosure came days after Fireworks hired Meta’s former chief financial officer Dave Wehner as its chief financial officer.
Fireworks said the evidence points to a financially motivated group that sells tokens from unauthorized access to large language models through illicit routing gateways, a practice known as LLMJacking. Google Threat Intelligence Group and Mandiant described the trend in a Nov. 2 report, which cited fraudulent charges above $100,000 in a single day and cumulative bills that reached $1 million at some organizations before access was revoked. Wiz counted at least nine AI developer platforms that disclosed incidents in October, according to VentureBeat. WebProNews covered a related pattern last month, when Symantec’s honeypot caught an attack agent built to run on other people’s exposed AI models.
For Notion, the episode follows an October push to deepen its AI features, when it rebuilt its agents and moved Notion AI to a new credit system. Owners of affected workspaces can review the exposed data through a security incident review in Notion’s desktop web interface, BleepingComputer reported. Both companies say their investigations are still open, and Fireworks warned that its work “may reveal additional information.”
WebProNews is an iEntry Publication