AI notes that never leave the laptop are a claim you can test in five seconds, not one you have to trust.

Every AI product now says some version of the same sentence: your data is private, your data stays yours. The specific promise that matters most on confidential client work, AI notes that never leave the laptop, is also the one almost nobody reading it can actually check. A privacy policy only proves a company wrote a privacy policy; it says nothing about what the software in front of you actually did. There is one test that does, and it takes five seconds: turn on aeroplane mode, ask the AI feature something it has never seen before, and watch whether a coherent answer still comes back. If it does, nothing was sent anywhere to produce it. If the feature stalls or quietly waits for a connection, the claim was never really about the device at all.

The industry's privacy claims are not all empty

Much of the effort behind these claims is genuine. SOC 2 audits, encryption in transit, documented data-retention windows and named subprocessor lists are real engineering and compliance work, done by people trying honestly to earn a user's trust. A vendor publishing a security page at all beats the alternative, and treating every privacy claim as cynical marketing does a disservice to the companies doing that work.

The effort is why the gap matters. A well-written, honestly maintained privacy policy is still a description of intent, audited at a point in time, by a process the end user never sees happen. It is categorically different from something the user can verify themselves, on their own machine, in the moment they actually care about the answer.

Why a policy document can't be the proof

None of the usual reassurances close the actual gap between what a vendor says and what a user can confirm.

A privacy policy is not evidence.

It is a legal document describing what a company intends to do and is contractually bound to. It says nothing about what the software running on a specific laptop, on a specific Tuesday, did with a specific piece of text.

Encrypted in transit still means processed somewhere else.

Encryption protects data on its way to a server. It says nothing about whether the data needed to leave the device in the first place. A perfectly secure connection to a cloud model is still a connection to a cloud model, and the privacy question was never really about the transport layer.

"Trust us" does not survive an acquisition, a bad quarter, or a subpoena.

Terms change. Companies get bought. Retention policies get quietly rewritten in a changelog nobody reads. A promise that was true and well-intentioned at signup carries no guarantee about the vendor's incentives eighteen months later, and none of that risk is visible from inside the product itself.

A claim that can only be checked by reading a document is a claim you have to trust. A claim you can check by turning off your own network is a claim you can verify yourself, and those are not the same kind of promise.

A verifiable claim is worth more than a reassuring one

Outside software, the trust that lasts tends to rest on something a customer can check personally. A glass wall onto a factory floor, a serial number that can be looked up, an ingredient list precise enough to be wrong. A company that shows its working earns a different kind of belief, because the belief doesn't depend on the company's honesty being assumed in advance. It depends on the customer's own eyes.

AI privacy claims have mostly skipped this step, for an understandable reason. Verifying that a cloud request wasn't made requires either technical skill most users don't have, like inspecting network traffic, or a claim about architecture that sounds identical whether or not it's true. "Your data is processed securely" and "your data never leaves your device" read the same to a non-technical user, right up until one of them can be tested and the other can't.

The aeroplane test, defined

The test itself is almost embarrassingly simple, which is part of why it works as a public demonstration rather than a technical audit only an engineer could run. Put the device in aeroplane mode. Open the AI feature in question. Ask it something genuinely new, not a cached or previously seen query, something that requires actual processing to answer. If a coherent, specific answer comes back with no network connection available, nothing was sent anywhere to generate it, because there was no path for anything to travel over. If the feature spins, times out, or silently produces nothing, the "on-device" language in the marketing copy was describing something other than where the actual inference happens.

AI notes that never leave the laptop, tested with the network off

Inside Callisto this runs on the same routing logic as every other day. No demo mode. The local backends execute a LiteRT-LM model inside a dedicated browser worker via WebGPU, entirely on-device, and Callisto's own routing treats an active network connection as a hard precondition for reaching any cloud provider at all. The moment a device goes offline, that precondition fails, and every path to a cloud model closes, regardless of what the user's general priority setting says. A feature that's genuinely local, like Spotlight's "Ask AI", the same kind of quick-question surface a consultant reaches for instead of opening yet another app to find one answer, keeps answering under a local-priority routing setting. A feature that has no local implementation at all, like receipt scanning, correctly fails rather than pretending to work, which is itself an honest result: it proves the product isn't hiding a network call behind a fake local UI.

There's also a browser-native path worth mentioning, separate from Callisto's own bundled model. On browsers that support it, some requests route to Chrome's built-in AI Prompt API instead, which runs inside the browser itself rather than Callisto's worker. Whichever of the two answers a given request, the offline behaviour is identical, because both are genuinely local implementations rather than a thin client pointed at someone else's server.

Why a coherent answer is proof, not a coincidence

The code path is what makes this a real test rather than a lucky demo. Callisto's cloud readiness check is one boolean: cloud AI allowed for this caller, an API key present, and the device online. Remove any of the three and the cloud branch becomes unreachable rather than unlikely. No cached response is replayed, and no background sync waits for the connection to return. The local model either has what it needs or it doesn't. Aeroplane mode forces that into view instead of leaving it a claim on a settings page nobody reads twice.

What the test cannot prove

The aeroplane test proves that a specific feature, at that moment, had no network path available and still produced a result. It proves nothing about accuracy, and nothing about how the rest of the product behaves. Callisto has exactly three features that are allowed to reach a cloud provider at all, each with a specific disclosure and a confirmation step, and none of them will work offline by design, because they were never claimed to be local in the first place. Run the aeroplane test against one of those three and it fails. That is the test working, separating the features that are local from the ones never marketed that way.

It is also not a test of a product's honesty in general. A vendor could build one genuinely local feature for the demo everyone runs and route everything else through the cloud. So test the feature you rely on. The easiest one to demonstrate proves little.

Where AI notes that never leave the laptop still need a network

The test is specific to AI processing, not to the product as a whole. Callisto still needs a network connection to load a workspace, sync a calendar feed, or save most ordinary edits, and none of that is what the aeroplane test checks. Point the same test at other tools. A notetaker advertising "on-device processing" or an app claiming local-first storage can be checked the identical way, and a reader who tries it on three tools and gets three different results has learned something no comparison article could have told them as reliably.

This is, deliberately, a harder standard than most vendors are used to being held to. A tool that fails the test isn't necessarily lying; plenty of genuinely good products are honest that they require the cloud and never claimed otherwise. The test only becomes a problem for a vendor when the marketing copy says one thing and the behaviour with the network off says another, and that gap is precisely the one worth finding before, not after, a client's confidential information depends on the answer.

Try it on the feature that actually worries you

Run this on the feature that made you nervous in the first place, not on a generic chat box in the settings menu. Callisto's person-profile summary, the one AI feature that generates itself without a button press, is a good target because it touches the most sensitive record in the product: the accumulated history of 1-on-1 conversations with a direct report or a client contact. Add a note to someone's discussion thread, switch to aeroplane mode, and refresh their profile. The summary either regenerates from the local model or it visibly declines to update, because the automatic trigger is hard-coded to refuse running on anything but the local backend. There is no third outcome where it quietly reaches a server anyway.

Compare that to a feature honestly built to require the cloud, like scanning a receipt. Try the same test there and it correctly fails offline, with no attempt to fake a local result. Seeing both outcomes side by side is more convincing than either alone. The boundary was drawn on purpose.

Getting started

Before trusting any AI feature with real client work, not a throwaway test query, especially the kind that touches something an NDA covers, run the test once. Turn off the network, open the feature, and ask it something specific that requires real processing rather than a cached lookup. Watch what actually happens rather than reading what the settings page claims will happen, since the two are not always the same thing. It takes less time than reading the privacy policy. And unlike the policy, it's a result you saw with your own eyes rather than one you were asked to take on faith.