The problem

Two things break the flow of working at a desk. Typing out something you could just say, badly and slowly — and having to describe what is on your screen to something that cannot see it. Dictation tools solve the first and then mangle every proper noun and piece of jargon you use. Assistants solve the second only if you paste in the context by hand, which costs more than it saves.

Constraints

• Always-listening, so wake-word detection has to be accurate enough not to trigger on ordinary conversation and not miss the real thing
• Dictation is worthless if it mangles the words you use most — the names, the jargon, the shorthand nobody else says
• Reading the screen means capturing and understanding it inside a spoken exchange's latency budget, not a page-load's
• Desktop and cross-process: it has to reach the operating system and other applications, not live in a browser tab
• Solo project built alongside full-time work, so complexity I could not maintain was not worth having

What I did

• Built the assistant on ElectronJS and Azure Functions, with speech handled through Deepgram and Whisper
• Built the dictation surface around personalisation: custom phrases and a personal vocabulary, so the words you actually use come out right instead of being guessed at phonetically
• Wired “Hey Voxa” as the activation path — the wake word brings the assistant up so it can read what is on screen and answer about it, rather than making you describe your own screen to it
• Split the Node.js backend into separate services for command processing, intent classification and entity extraction, so the assistant carries state across turns instead of treating every utterance as new
• Put a confirmation step in front of anything irreversible: the drafted action is shown and has to be approved before it runs

Outcome

Voxa runs as a working desktop app. Dictation learns the phrases and vocabulary you give it, “Hey Voxa” brings up an assistant that can read the screen and answer about it, and anything destructive or outbound goes through a confirmation step first. Wake-word accuracy sits above 95% in my own testing.

Stack

• ElectronJS
• Node.js
• TypeScript
• Microsoft Azure
• Azure Functions
• LLMs
• Deepgram
• Whisper

Screens