top of page

The AI Did Nothing Wrong: What the Uber and Fireworks Breach Teaches About Privacy

2 days ago
5 min read
Uber drivers' license information was obtained through stolen credentials at an AI vendor, not through Uber's own systems. (OSRS illustration)
Uber drivers' license information was obtained through stolen credentials at an AI vendor, not through Uber's own systems. (OSRS illustration)

Bottom Line Up Front: Uber is notifying thousands of drivers that their driver's license information was stolen in October. The intruders never broke into Uber. They stole credentials at Fireworks AI, a company that runs artificial intelligence models for other businesses, and used them to open Uber's account there. No AI system misbehaved. The failure was older and simpler: sensitive data sat in a place it may never have needed to be. That is the privacy lesson for every organization now connecting AI vendors to its operations.


What Happened

Fireworks AI is an inference provider. In plain terms, it rents out computing power and open AI models so that companies such as Uber can build AI features without running the machines themselves.

The public record comes from Uber's notice to the California Attorney General and from reporting by Reuters and TechCrunch:

  • October 9. The vendor told Uber that an unauthorized party had compromised the vendor's internal credentials and used them to enter Uber's account. Uber says it cut its data connections immediately.

  • October 13. Uber's investigation determined that personal information had been accessed.

  • November 10. Uber dated its notices to affected people. The notice names neither the vendor nor the data involved.

  • November 13. TechCrunch identified the vendor as Fireworks AI and reported that the data included driver's license information. Uber confirmed the vendor's name.


State filings so far count 7,870 people: 2,567 in Massachusetts, 3,430 in South Carolina, and 1,873 in Rhode Island. Uber says it has found no misuse to date and is offering free credit monitoring.


Fireworks has disclosed two intrusions in October. It attributes the activity to a financially motivated group that sells stolen access to AI models, a practice some researchers call LLMjacking. No group has claimed the theft.


Sorting Confirmed From Unknown

Statement

Standing

Stolen vendor credentials were used to reach Uber's data

Confirmed. Uber's notice.

The vendor is Fireworks AI

Confirmed by Uber to TechCrunch. The notice itself does not name it.

Driver's license information was taken

Reported by TechCrunch from state filings. The public California notice does not list data types.

7,870 people in three states

Confirmed in filings. Illinois is pending and California is undisclosed, so the total will rise.

The data has been misused

No misuse identified, according to Uber.

Why license data was held at an AI vendor

Unknown. Neither company has explained.

Whether the data was license numbers, images, or both

Unknown.

Who the attackers are

Unnamed. Fireworks describes a profit-driven group. No claim of responsibility.


The Skeptic Is Mostly Right

Before calling this an AI privacy crisis, hear the strongest objection. A seasoned privacy officer would say this is not an AI story at all. No model leaked, guessed, or was tricked. A thief used a stolen key to open a vendor account. That is how vendor breaches have worked for more than a decade.


The same officer would add that the label "AI risk" does Uber a favor. It moves attention away from two principles every privacy student learns in the first week:


  • Data minimization. Collect and share only what the task requires.

  • Purpose limitation. Use data only for the reason it was collected.


A driver hands over a license to prove identity and the right to drive. No public source explains why that information ended up with a company that hosts AI models. Until Uber answers, that remains the central question.

The model did not leak the data. The decision to send it there did.

What Is Different About AI Vendors

However, three features of the AI supply chain make old failures more likely.

AI projects are hungry for data. Teams that test or tune models copy real records into vendor environments for evaluation and analytics. Those copies often sit outside the vendor inventory that the privacy office maintains.


Contract language may not match practice. Notion, the workspace software company, disclosed that the Fireworks breach exposed AI chat transcripts from fewer than 1 percent of its workspaces. Notion says Fireworks had contractually committed to zero data retention for its data. Fireworks' answer to that specific point does not appear in the reporting we reviewed.


AI credentials are now merchandise. If Fireworks is right about motive, the intruders wanted model access to resell. Driver records may have been swept up along the way. Your data can be taken by someone who was not looking for it.

A promise of zero retention is a claim until someone tests it.

Why a Driver's License Is Not a Password

A password can be changed in a minute. A driver's license number is hard to replace, and it is a building block for identity fraud: opening accounts, passing verification checks, and filing false claims. Many of the people affected are gig workers. They carry the cost of a decision in which they had no part.


For policymakers in Washington, Brussels, London, and Abuja, the case tests a question regulators everywhere are asking. When personal data passes from a company to an AI vendor, and from that vendor to its cloud provider, who answers to the individual? Under U.S. state breach laws, the duty to notify affected people generally rests with the business that collected the data. Responsibility does not transfer with the files.


Five Questions for Every AI Vendor

  1. What personal data do we send, and does the task require it? A yes-or-no verification result will often do where a license number is being sent today.

  2. Where are the copies? Include testing, evaluation, analytics, and backup environments.

  3. What does the contract say about retention, and how have we verified it?

  4. Who at the vendor can reach our data, and how are those credentials protected?

  5. How fast must the vendor tell us about a breach, and can we cut the connection the same day?


An Analyst's View

The honest reading is that the important facts are still missing. We do not know why the data was there, how much was taken, or whether it has been sold. Nevertheless, the pattern is clear enough to act on. Artificial intelligence did not weaken privacy in this case. The rush to adopt it did. Organizations are adding AI vendors faster than they are updating the inventories, contracts, and access rules that protect individuals. Therefore, the right response is not fear of the technology. It is discipline about what data travels with it.


How OSRS Can Help

OGUN Security Research and Strategic Consulting LLC provides intelligence analysis, security assessments, and strategic advisory services to government agencies, corporations, and institutions across the United States, Europe, and Africa. Our team helps organizations map the personal data flowing to AI vendors, conduct due diligence on third-party providers, review contracts and retention commitments, and prepare decision-makers for breach response. Contact us at contact@ogunsecurity.com to schedule a confidential consultation.


Enjoyed this article? Stay informed by following us on Google News, Twitter, and LinkedIn for more exclusive cybersecurity insights and expert analyses. Share this article with your network and subscribe to our email list for weekly intelligence briefings.


About the Author

Dr. Sunday Oludare Ogunlana is the Founder and CEO of OGUN Security Research and Strategic Consulting LLC, Professor of Cybersecurity, national security scholar, and intelligence and digital forensics practitioner. He advises public and private sector leaders on cyber risk, privacy, intelligence, and digital investigations.


Comments

Rated 0 out of 5 stars.
No ratings yet

Add a rating
bottom of page