Running an AI model on data the server can’t read has mostly been a job for cryptographers. Google wants to make it a job for ordinary developers. The company is showcasing HEIR, an open-source compiler that converts pretrained models built for plaintext data so they run on encrypted inputs instead.
HEIR, short for Homomorphic Encryption Intermediate Representation, is now part of Google’s Private Computing Toolkit. “Our vision is to make HEIR a one-click solution to enable non-experts to incorporate encrypted inference into production applications,” Google wrote.
The problem it targets
Homomorphic encryption lets a server compute on ciphertext and send back an encrypted result without ever seeing the underlying data. That solves a real bind. End-to-end encryption protects user data, but then a provider can’t offer features that depend on that data, such as spam or malware detection. Running the model on the device instead is limited by the device’s hardware and puts a proprietary model in users’ hands, where it can leak.
The catch has always been cost and effort. Homomorphic encryption carries significant compute overhead, and Google notes that converting an existing program to use it efficiently by hand “requires a team of cryptographers.” HEIR is meant to automate that conversion. Google’s argument is that the technique turns the privacy question into a cost question, and that the cost is falling quickly.
Four demos
Google compiled four private inference applications with HEIR and published the source in its GitHub repository. It reported latency on a single-threaded CPU.
- Recommendations: a Deep Learning Recommendation Model that serves content suggestions without the service seeing the user’s features, built with Belfort Labs, LG and New York University.
- Fraud detection: a credit card fraud detector, with Niobium and hardshell.ai.
- Network anomaly detection: the Kitsune system, compiled with Niobium, which can flag anomalies in encrypted traffic without exposing packet contents.
- Hotword detection: a model, built with Belfort Labs, that could let an audio-triggered agent recognize a wake word while keeping the recordings private.
Google has also worked with homomorphic encryption hardware accelerator makers Belfort, Niobium, Cornami and Optalysys, and says it plans to show those accelerators’ latency gains soon.
Using it today
HEIR is built on MLIR, the compiler framework from the LLVM project. According to its GitHub README, there are three ways to use it right now: through Bazel and rules_heir with the OpenFHE or Lattigo backends; by installing OpenFHE and the heir_py Python package; or by building from source and calling the heir-opt and heir-translate tools directly.
It has picked up academic traction, too. Google says researchers at Georgia Tech, Carnegie Mellon, UC Santa Barbara, Purdue, the University of Edinburgh, Tsinghua and others have built on it, and four peer-reviewed papers so far rely on HEIR.
Where it fits
Google isn’t alone in shipping this technology. Apple’s open-source Swift Homomorphic Encryption library already powers features such as Live Caller ID Lookup, which uses private information retrieval to check numbers without revealing them to the server. Google’s pitch with HEIR is broader: a general compiler for arbitrary models, not a library for a few specific lookups.
For developers in regulated fields like healthcare and finance, where sharing data across institutions is tightly restricted, the demos are worth a look. The practical limit is still performance, so the near-term candidates are small, high-value models such as fraud scoring or anomaly detection rather than large language models.
WebProNews is an iEntry Publication