Getting text into the application already in use
The user places the cursor in an editor or message field, activates the microphone and speaks. The text appears in that same application, without a manual copy-and-paste step. Once the models have been downloaded, recognition runs on the computer.
I built the workflow around Whisper, through faster-whisper, and the Silero speech detector. Keyboard shortcuts suspend listening, transcribe the remaining fragment or put the tool to sleep. Automatic pasting under X11 restores the previous clipboard contents afterwards.
Recognising a pause without ending the sentence too early
Transcribing words is only part of the task: the application must decide when to send a recording to the model. Silence can end a sentence or mark a hesitation. I combined its duration with falling energy and voice pitch to adjust the waiting time.
The thresholds are staggered. Silence alone waits longer than several agreeing cues, and a test checks their order so simpler conditions cannot hide the others. When background noise makes automatic detection awkward, a manual mode lets the user choose when to transcribe.
I also separated short patterns from longer phrases in the filter for unwanted outputs. Exact matching for short words avoids discarding a legitimate sentence merely because it contains the same word.
Protecting capture and managing resources
The PySide6 visualiser runs in a separate process from audio processing. It receives status and text through an SSE event stream, which fits the mainly one-way communication. This separation keeps interface work away from the capture loop.
Sleep reduces activity between uses and unloads the models after a longer absence. Continuous preview is disabled by default: its second Whisper pass costs resources for limited benefit in this workflow.
Checking changes to audio processing
I replaced the high-pass filter’s Python loop with its vectorised SciPy equivalent. For the measured 30-second buffer, preprocessing fell from 445 to 10 ms, with a maximum numerical difference of 1.2 × 10⁻⁷.
That measurement covers the filter rather than the entire dictation latency. An audio-file demonstration and detection tests also make the components inspectable without a microphone, alongside ordinary desktop use.
Architecture and decisions · Filter tests · Sentence-end tests