Where your data actually goes: Haven's three-tier inference design

A standard IMAP fetch or a Google Drive API call typically results in a binary outcome: the data is either on your machine or it is on the provider’s server. When we began building Haven, we realized that applying modern machine learning to sensitive data security breaks this binary. If you run everything in the cloud, you compromise privacy by design; if you run everything on the device, you often compromise the recall necessary to find a deeply buried tax return or a stray password in a decade of archives.

We settled on a three-tier inference architecture. It is designed to be local-first, but it is not local-only. We believe the most honest way to build security software is to document exactly where the data goes, why it goes there, and how you can audit that movement yourself. This note details the mechanics of those three tiers and the ‘tiebreaker’ logic that governs the ~0.8% of messages that require a second opinion.

Tier 1: Deterministic patterns and the first pass

Every message or document Haven encounters first passes through Tier 1. This layer does not use neural networks. It consists of high-precision regular expressions and checksum-based patterns designed to catch ‘low-hanging’ sensitive data. If a string matches the structural format of a Social Security Number or a known private key header (e.g., -----BEGIN RSA PRIVATE KEY-----), Tier 1 flags it immediately.

This layer runs entirely on your CPU. It is computationally inexpensive and serves as a high-velocity filter. Because Tier 1 is deterministic, it has zero false positives for specific structural formats, but it is blind to context. It cannot tell the difference between a real credential and a formatted example in a technical newsletter. To solve for context, we move to the second tier.

Tier 2: The on-device classifier

Tier 2 is a bundled, small-parameter model that runs locally on every single message. This is an on-device classifier optimized for high recall. Its job is to look at the linguistic context surrounding the data. While Tier 1 might find a 16-digit number, Tier 2 determines if that number is likely a credit card or just a shipment tracking ID based on the words appearing in the preceding 50 tokens.

In our testing across diverse real-world inboxes, Tier 2 is decisive for the vast majority of content. It provides a high-confidence ‘Ignore’ or ‘Flag’ verdict for over 99% of processed items. Because this happens locally, the raw content of these messages never touches a network interface during the classification process. However, machine learning is probabilistic. There exists a thin margin of edge cases where the local model’s confidence score falls below our required threshold—situations where the language is ambiguous or the document structure is highly irregular.

Tier 3: The second-opinion tiebreaker

When Tier 2 cannot reach a confident conclusion, the system invokes Tier 3. This is our most capable model, designed to act as a tiebreaker. Based on our measurements, this occurs for approximately 0.8% of messages in a typical user’s archive. At install time, Haven runs a 30-second hardware benchmark to determine where Tier 3 should execute.

If your local GPU passes the benchmark, Tier 3 runs entirely on your machine. In this configuration, nothing leaves your machine at any tier, because your hardware is capable of handling the larger model weights without degrading the user experience. If the hardware does not pass the benchmark, Tier 3 defaults to Haven-operated cloud infrastructure.

This is the honest exception to the local-processing rule. If Tier 3 runs in the cloud, Haven sends a bounded excerpt of the message—never more than 2 KB—to our servers for a single round-trip inference call.

The architecture of the Tier 3 cloud path

We have engineered the cloud Tier 3 path to be as ephemeral and transparent as possible. We do not use third-party AI APIs or hosted model services. Instead, we operate our own GPU infrastructure end-to-end. We run open-weights models on hardware we control, which allows us to enforce strict data-handling guarantees that are structurally impossible when using a third-party provider.

When an excerpt is sent to our cloud Tier 3:

1. No Retention: The content is held in volatile memory only for the duration of the inference (typically milliseconds).
2. No Logging: Our inference logs record the fact that a request occurred and the latency of that request, but never the content of the excerpt itself.
3. No Training: We do not, and cannot, use these excerpts to train or fine-tune our models. The data is discarded immediately after the verdict is returned to your device.

Every single cloud call is visible to you. We include a Network Activity log within the application that shows when a Tier 3 request is made. This ensures that the ‘black box’ of cloud processing is actually a glass box.

What never leaves the device

While Tier 3 excerpts are the only content that might leave the device, it is equally important to define what is never sent. Our telemetry and sync systems are strictly separated from our inference logic.

Aggregate telemetry (used to understand app stability), signed-channel fetches (used to update detection patterns), and crash reports never carry message content or specific finding details. Your OAuth tokens, which grant Haven access to your providers, are stored in your operating system’s credential store — the Keychain on macOS, Credential Manager on Windows, and the Secret Service on Linux and are used only to fetch data directly to your local machine. This is critical because malicious forwarding rules or compromised credentials can often go unnoticed without local auditing.

When Haven displays a finding to you, it paraphrases the nature of the data — it will report that it found “a tax return” or “a plaintext password,” never the value itself. Where a snippet of surrounding context helps you identify the source, the sensitive portion is redacted with block characters (████). The raw value is never rendered, so it cannot leak into screenshots, UI logs, or system snapshots. Understanding what ‘anyone with the link’ sharing really exposes in your cloud drive is the first step toward securing these sensitive files.

The state of the system

By splitting inference into these three tiers, we avoid the binary trap of ‘fast but blind’ or ‘smart but invasive.’ Today, this architecture allows Haven to process hundreds of thousands of messages with the precision of a large language model while keeping 99.2% of the heavy lifting on your local silicon.

After the initial scan and any necessary Tier 3 tiebreakers are complete, the truth of the system is this: your sensitive data remains where it has always been—in your email or cloud drive—but you finally have a local map of where the vulnerabilities are. The cloud didn’t need to learn your secrets to help you find them.