What Actually Leaves Your Machine When You Dictate: Packet Captures and Memory Buffers

Dissecting the Cloud Audio Payload: Opus Frames, API Tokens, and Telemetry

We ran Wireshark while speaking into three popular cloud dictation apps. Over a 20-minute dictation session, our network interface captured 42 megabytes of raw audio streams, TLS handshakes, device fingerprint headers, and background telemetry heading to remote infrastructure in northern Virginia.

When you use a cloud voice tool, "private" is a legal promise written by marketing departments. When you run local inference, "private" is an architectural guarantee enforced by your operating system kernel. Here is the network-level breakdown of both pipelines, and what it takes to eliminate network egress entirely.

[Client Microphone]
       │
       ▼ (16kHz 16-bit PCM Audio)
[CoreAudio / WASAPI Capture Buffer]
       │
       ▼ (Opus compression: 32 kbps to 64 kbps)
[WebSocket Frame / HTTPS POST] ──► [TLS 1.3 Handshake] ──► [Public Internet Routing]
                                                                  │
                                                                  ▼
                                                      [Cloud Load Balancer / API Gateway]
                                                                  │
                                                                  ▼
                                                      [Remote GPU Host (ASR Decode)]
                                                                  │
                                                                  ▼
                                                      [Cloud LLM Post-Processor]

What Wireshark captured over the wire:

Even when a cloud vendor pledges never to sell user data or train models on customer recordings, the audio stream still traverses intermediate BGP routes, CDN edge nodes, and cloud provider disks. If an API key leaks, an employee machine is compromised, or a cloud bucket is misconfigured, your voice recordings are exposed.

The Local-First Pipeline: Ring Buffers, Silero VAD, and Metal Tensors

To eliminate the network attack surface, you must decouple speech recognition from network sockets entirely. Here is the architecture we implemented in Murmur:

[Microphone Hardware]
       │
       ▼
[OS Audio Capture (WASAPI / CoreAudio)]
       │
       ▼ (Zero-copy 16kHz float32 ring buffer in volatile RAM)
[Silero VAD (Voice Activity Detector)]
       │
       ├── Speech detected? ──► Accumulate in RAM buffer
       └── Silence detected? ──► Trigger Whisper inference pass
                                       │
                                       ▼
                       [whisper.cpp Engine (Metal / DirectML)]
                                       │
                                       ▼ (Greedy token decode + regex normalization)
                       [Accessibility / SendInput API]
                                       │
                                       ▼
                       [Pasted at Active Cursor] ──► Buffer in RAM zeroed immediately

1. Volatile RAM Ring Buffers

Audio frames enter a fixed-size circular buffer in volatile RAM. No temporary .wav or .mp3 files are ever written to the SSD or disk cache.

2. Silero Voice Activity Detection (VAD)

Instead of streaming silence over a socket, a tiny (1.8MB) on-device ONNX model processes audio frames in 30ms slices. It flags precisely when human speech begins and ends, rejecting keyboard clicks and ambient office noise.

3. Native C++ Tensor Ops

Audio tensors pass directly to whisper.cpp, executing in parallel across GPU execution units (DirectML on Windows, Metal on Apple Silicon).

4. Immediate Buffer Zeroing

As soon as the greedy token decoder outputs the final text string, the underlying audio buffer in memory is overwritten with zeros:

// Zero out sensitive audio buffers immediately after inference
pub fn purge_audio_buffer(buffer: &mut Vec<f32>) {
    buffer.fill(0.0);
    buffer.clear();
    buffer.shrink_to_fit();
}

The Latency Breakdown: 172ms Local vs 480ms Cloud

Marketers claim that massive cloud server clusters are faster than laptops. When we profiled actual end-to-end wall-clock latency, the math told the opposite story:

Cloud Dictation Wall-Clock Pipeline:
Audio recording complete (t = 0ms)
├── Client Opus compression: +25ms
├── TLS packet dispatch: +15ms
├── Public internet transit (RTT): +65ms
├── Cloud API Gateway queue: +40ms
├── Cloud GPU Whisper decode: +180ms
├── Cloud LLM clean-up pass: +110ms
└── Response transit + paste: +45ms
Total p99 latency: 480ms

Murmur On-Device Metal Pipeline:
Audio recording complete (t = 0ms)
├── Silero VAD silence confirmation: +30ms
├── whisper.cpp quantized Metal decode: +128ms
├── Local regex clean-up & casing: +2ms
└── Native OS accessibility text insertion: +12ms
Total p99 latency: 172ms (2.8× faster)

The cloud system's raw GPU might calculate the matrix multiply 40ms faster, but it pays a 300ms penalty in network transit, TLS handshakes, and serialization. Local hardware wins because moving data across the PCIe bus takes microseconds; moving data across the internet takes tenths of a second.

What We Broke Along the Way: Circular Audio Buffer Overflows

Building a reliable local audio pipeline is not trivial. Our earliest prototype suffered from a nasty bug: if a user held down the push-to-talk key for more than 90 seconds while dictating a complex technical design doc, the audio capture thread dropped 300ms chunks of speech.

The root cause:

CoreAudio's realtime thread demands zero allocations. Our initial buffer implementation used a standard Rust Vec<f32> that reallocated dynamically when speech exceeded 60 seconds. Reallocating on a high-priority audio callback thread introduced a 4ms lock contention that caused CoreAudio to drop incoming frames:

// BAD: Dynamically growing vector on the audio callback thread
// Triggers memory allocation and frame drops under load
fn audio_callback(data: &[f32], storage: &mut Vec<f32>) {
    storage.extend_from_slice(data); // Allocates!
}

// FIXED: Pre-allocated circular ring buffer with atomic write pointers
// Zero heap allocations in the realtime audio path
struct AudioRingBuffer {
    buffer: Box<[f32; 16000 * 120]>, // Fixed 120-second capacity
    write_head: AtomicUsize,
}

By switching to a pre-allocated fixed-size ring buffer with atomic write heads, we eliminated allocation lag, keeping real-time audio capture rock-solid across 10-minute continuous dictation marathons.

How to Verify Zero Egress on Your Own Machine

You do not have to trust our code. You can verify the network isolation of your dictation tools yourself using standard OS utilities.

On Windows (via built-in Packet Monitor):

# 1. Start a packet monitor trace filtered to non-loopback traffic
pktmon filter add MurmurFilter -p 443
pktmon start --etw

# 2. Dictate for 30 seconds into your app

# 3. Stop packet monitor and inspect results
pktmon stop
pktmon format PktMon.etl -o packets.txt
Select-String -Path packets.txt -Pattern "murmur.exe"
# Expected result: 0 matching lines

On macOS (via tcpdump):

# Listen on all network interfaces for any traffic originating from the local app
sudo tcpdump -i any -nn -s0 -v "tcp and port 443" | grep -i "murmur"
# Expected result: silence — zero network packets emitted

When an application physically contains no network socket initialization code, zero packets leave your machine. That is privacy you can prove.