The 90-Second Fingerprint: What ECH Actually Hides
Vercel's Encrypted Client Hello (ECH) obscures SNI hostnames at the CDN layer. Good. A passive observer can no longer read which endpoint your agent is calling. But we've watched agents leak their entire tool graph in under 90 seconds—without ever seeing a hostname.
Here's how: ECH encrypts SNI + ClientHello. It does not encrypt packet timing, packet size, or TLS record distributions. OpenAI's streaming API sends chunks with a recognizable inter-token cadence—roughly 30–80ms per token depending on model. Hugging Face Inference Endpoints have a different distribution, flatter and more irregular. A classifier trained on 500 samples from each vendor reaches 85% accuracy on vendor attribution within a 4-vendor agent using only timing deltas and TLS record lengths. Same agent, three external calls, 90 seconds of wire capture, hostname completely redacted.
Token count leaks are worse. A 2KB TLS record almost always correlates with a tokenizer boundary. OpenAI's usage response includes exact token counts in the response body; that count is logged by default. A context window of 8K vs 128K is visible in the size of the final record before close. You can infer whether an agent hit a retrieval system, how deep the context goes, and which vendor handled the call—all from observing the wire without breaking encryption.
Your Instrumentation Is the New Attack Surface
The real leak isn't on the wire. It's in your logs, traces, and metrics—the infrastructure you own and ship wide open to Datadog, Sentry, LangSmith, and whatever else you've wired up. Network privacy moved the attack surface inward.
By default, OpenTelemetry span names include model and endpoint: openai.chat.completions, huggingface.inference. Prompt and completion payloads land in Sentry breadcrumbs. Token counts appear in metric labels—high-cardinality fingerprints of user intent. Third-party log shippers become out-of-band exfil for the same vendor information ECH tried to hide. An insider with read access to your observability layer can reconstruct the agent's tool graph in seconds. A compromised log shipper (it happens) is a full dump of which vendors you call, when, and with what intent.
The threat model shifted. External MITM is harder now. Insider + supply-chain visibility is the new leverage. Vercel Connect shows token lifecycles and correlation IDs per connector, but observability stops at the connector layer, not the agent. That's the gap.
Timing, Tokens, and Errors: The Three Leaks Everyone Ships
Most teams don't redact before export. Let's be concrete about what leaks:
Timing: TTFT (time to first token) variance between GPT-4o (~350ms average) and Claude (~600ms average) is a vendor tell in any trace system. Latency percentiles in your metrics dashboard are a vendor fingerprint. A 90th-percentile response time of 450ms whispers "OpenAI" to anyone with access to your Prometheus.
Token usage: prompt_tokens in span attributes leaks retrieval hit-rate and context depth. A spike from 500 to 3500 tokens tells you the agent hit a retrieval system and loaded context. The pattern of token deltas across calls reveals branching logic in the agent's decision tree.
Errors: A 429 (rate limit) vs 529 (overloaded) vs custom 4xx signature maps 1:1 to the vendor even when hostnames are stripped. Retry-After headers echoed into logs re-expose the vendor. Correlation IDs from upstream vendors (OpenAI's x-request-id) passed through unchanged are vendor tattoos in your audit logs.
Hardening Patterns We Now Ship by Default
Start with the parts you control. We now ask these questions in every production deployment:
Bucket timing metrics into coarse histograms (50ms buckets, not raw milliseconds) before export. Redact the tail; 99th-percentile latency is useful, individual response times are not.
Hash vendor-specific IDs at the span processor, not at the dashboard. Don't trust downstream sanitization. x-request-id → SHA256 hash before any log shipper sees it.
Unify error taxonomies. Normalize 429/529/quota_exceeded to a single internal code pre-log. Strip vendor error messages. Log only the internal enum value.
Separate 'ops' telemetry from 'debug'. Rates and p99 latencies go to long-term storage. Raw payloads, token counts, and correlation IDs live in short-retention debug logs with access controls. Different retention policies, different ACLs.
Constant-time-ish response padding for service-to-service calls where the threat model warrants it. For external agent-to-vendor calls, the signal-to-noise ratio of padding isn't worth the latency cost. For internal tool composition, it can be.
Egress allowlists via AI Gateway. Vercel's AI Gateway routing rules move failover out of code. A side benefit: no DNS resolution happens at call time, no side-channel leakage through resolver logs. Vendor swaps stay internal.
When ECH + Gateway + Redaction Isn't Enough
Some workloads demand architectural changes. You can't log your way out of these:
Multi-tenant agents: Timing profiles from one tenant leak another's usage patterns. If your architecture isn't already tenant-isolated at the service boundary, observability hygiene won't fix it.
Regulated data: HIPAA, PCI, or internal policy: any vendor attribution in logs is a finding. Self-hosted inference is the only fix, with real cost tradeoffs. You own the model, the logs, the threat surface.
Batching + async decoupling. If your agent orchestrates calls asynchronously (batch 10 requests, process responses in 5 minutes), timing correlations disappear. This is a privacy control, not just a latency control. But it changes your agent's interaction model entirely.
We draw the line on client engagements where redaction is defensible. Token-count visibility + error normalization + no correlation IDs is a solid baseline. Where the threat model demands vendor obscurity in the logs themselves, we usually talk about treating the eval pipeline as an attack surface and isolating it before hardening observability.
Audit Your Stack Tonight
Open a production trace from your agent. If you can identify the vendor from span attributes without reading the hostname, ECH bought you nothing. Start with error normalization and token-count redaction in your span processors. If you ship Datadog or Sentry, audit what's landing in breadcrumbs tonight. Correlation IDs and timing variance are the next targets.
If your threat model demands more—if vendor attribution in logs is itself a breach—talk to us at /contact about architectural isolation. There's no encrypt-your-way-out of observability side channels. Privacy starts with what you log.