Why Offline Dictation on Windows Sucked for a Decade (And How DirectML Fixed It)

The Graveyard of Windows Speech Recognition

For the past ten years, offline voice dictation on Windows was a choice between two bad options: pay $500 for Dragon NaturallySpeaking's bloated legacy installer, or run Windows Speech Recognition, an acoustic relic that still struggles to punctuate basic English sentences. When Microsoft introduced Windows Voice Typing (Win + H), they locked it behind mandatory cloud telemetry.

We ported whisper.cpp to native Windows using DirectML acceleration. It runs speech-to-text entirely on your local GPU (Intel, AMD, or NVIDIA) with sub-180ms latency and zero internet connection. Here is how the graphics pipeline works, what broke along the way, and how the top Windows options compare.

[Windows Audio] ──► [SAPI 5.0 (2001)] ──► Hidden Markov Models (HMM) ──► 28% Error Rate
[Windows 11]    ──► [Win + H Hotkey]   ──► Azure Cloud Speech API    ──► Cloud Telemetry Required

How DirectML Executes Whisper Across AMD, Intel, and NVIDIA Silicon

On macOS, Metal gives Apple developers a uniform GPU target. On Windows, hardware is fragmented across NVIDIA RTX, AMD Radeon, and Intel Arc / Iris Xe GPUs.

If you build on CUDA, you lock out millions of AMD and Intel laptop users. If you build on pure CPU, transcription takes 3× longer than real time, draining your battery and causing text to lag seconds behind your voice.

DirectML solves this by providing a unified DirectX 12 compute abstraction for machine learning primitives:

                  ┌──────────────────────────────┐
                  │   whisper.cpp Tensor Model   │
                  └──────────────┬───────────────┘
                                 │
                  ┌──────────────▼──────────────┐
                  │    DirectML Execution API   │
                  └──────────────┬───────────────┘
                                 │
      ┌──────────────────────────┼──────────────────────────┐
      ▼                          ▼                          ▼
[NVIDIA Tensor Cores]    [AMD RDNA Execution Units]  [Intel Xe Cores]

Benchmarks Across Windows Silicon:

We benchmarked 30 seconds of spoken prose across four common Windows hardware setups using whisper-small-q5_1:

On any modern integrated or discrete GPU, DirectML executes inference faster than you can blink, allowing real-time text insertion without spinning up noisy laptop cooling fans.

The Top Offline Dictation Tools for Windows 11 Compared

The Windows Bugs We Had to Solve: Keyboard Hooks and DPI Scaling

Porting a global dictation utility to Windows uncovered several platform-specific pitfalls:

1. The Low-Level Keyboard Hook Freeze (WH_KEYBOARD_LL)

To listen for global hotkeys like Alt + Space, Windows applications register a low-level keyboard hook via SetWindowsHookExW. If your hook callback blocks for more than a few milliseconds, the Windows OS watchdog silently kills your hook:

// BAD: Doing work inside the low-level hook callback freezes input
unsafe extern "system" fn low_level_keyboard_proc(code: i32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
    if code >= 0 && wparam == WM_KEYDOWN as usize {
        expensive_audio_state_check();
    }
    CallNextHookEx(std::ptr::null_mut(), code, wparam, lparam)
}

// FIXED: Immediately dispatch key events to an asynchronous channel
unsafe extern "system" fn low_level_keyboard_proc(code: i32, wparam: WPARAM, lparam: LPARAM) -> LRESULT {
    if code >= 0 {
        let vk_code = (*(lparam as *const KBDLLHOOKSTRUCT)).vkCode;
        EVENT_SENDER.try_send(vk_code).ok();
    }
    CallNextHookEx(std::ptr::null_mut(), code, wparam, lparam)
}

2. Unicode Injection via SendInput

Synthesizing keystrokes across Windows applications (Notepad, VS Code, Slack, WSL terminals) often mangles special characters like quotes, em-dashes, and code symbols. By dispatching KEYEVENTF_UNICODE packets rather than virtual scan codes, Murmur inserts UTF-16 code units directly into target windows without clipboard side effects.