steinbro.github.io

An unobtrusive text-to-speech workflow in GNOME

2026-06-06T00:00:00+00:00

When text-to-speech is not a convenience but a necessity, it needs to be available instantly, for any text on the screen. The operating system, or more specifically the desktop environment, can step in to provide it system-wide, across all applications. For Linux, this functionality (as with much else on Linux) is possible, with some tinkering.</p>

On-demand speech on other operating systems</h3>
macOS has had “Speak Selection” functionality</a> for ages. When actively speaking, the system shows a floating controller for playback and speech rate adjustment. </p>
Microsoft similarly introduced speech features into Magnifier</a> in Windows 10. Clicking the “Read from here” button starts reading text aloud from wherever the user specifies. </p>
The Reader View in Firefox</a> has text-to-speech built in as well. </p>
Screen readers also have some features that are geared towards partially-sighted users. Both NVDA on Windows and VoiceOver on macOS support a mode that reads text under the mouse pointer. But on Linux, Orca’s Mouse Review feature works inconsistently under Wayland</a> the modern display system for Linux.</p>

Prior art on Linux</h3>
A 2021 blog post</a> demonstrates how to set up a keyboard shortcut to trigger text-to-speech on most any Linux desktop environment. The impressively short shell script simply grabs the highlighted text from the clipboard’s primary selection buffer, and feeds that directly into the espeak synthesizer.</p>
This works great much of the time, but you’ll run into many unhandled edge cases if you use it extensively. For example. text containing Unicode mathematical characters, a common social media trick for adding styling, can turn into a stream of arcane code point names. “𝐉𝐚𝐯𝐚𝐒𝐜𝐫𝐢𝐩𝐭” becomes “letter 1D409, letter 1D41A, …” Emoji fare even worse, often disappearing entirely from the spoken output.</p>
My favorite failure mode is when the script encounters text surrounded by double square brackets. The espeak synthesizer interprets what’s inside as raw phonetic data. The result is text that is a very exotic flavor of garbled. [[ If you happen to be using it now, this example will demonstrate exactly what I mean,. ]]</p>

A native extension for GNOME</h3>
I’ve published a GNOME shell extension</a> that builds on the same basic approach to capturing and speaking selected text, but makes it a bit more robust to text encountered in the wild.</p>
</p>
The most visible improvement is its integration with the GNOME desktop, but the bigger change comes from inserting Speech Dispatcher between your text and the speech engine. This long-standing Linux service handles many of the failure modes we encountered earlier: it translates Unicode characters into something meaningful, announces emoji names instead of silently dropping them, and filters out control sequences that can trip up speech synthesizers.</p>
Because Speech Dispatcher operates as a system-wide service, it also improves the overall listening experience. Other applications won’t try to speak over the extension, and you can change your preferred voice or speaking rate without needing to edit the values hard-coded into the original script.</p>
There are still some limitations. Most significantly, the extension still relies on the primary selection buffer, a legacy feature of the Linux desktop that isn’t supported universally. For example, applications that are based on the Flutter toolkit, including the Ubuntu App Center and FluffyChat, don’t populate this buffer</a> when the user highlights text. That means the extension will ignore what you’ve highlighted in your Flutter-based app, and instead speak the last text highlighted in some other non-Flutter-based application.</p>
Nevertheless, give it a try and let me know if you find it helpful!</p>

Adapting magnifier controls for mouse-centric websites

2026-05-25T00:00:00+00:00

The problem</h3>
Modern websites love using hover effects. Pausing the mouse pointer over various parts of the page can trigger preview cards, floating menus, or something else entirely.</p>
For most users, this is fine and sometimes even helpful. If it’s unwanted, the user can just move the mouse elsewhere. But it poses an accessibility challenge for full-screen magnification users. Their visible area of the page, known as a viewport, will be much smaller than what most other users have.</p>
</p>
Many hover-heavy interfaces are built around a pair of assumptions:</p>

Moving the mouse over something means the user wants to interact with it more deeply.</li>
Users can comfortably look at one part of the screen while manipulating another.</li> </ul>
For full-screen magnification users, neither assumption really holds. Moving the mouse is often just a way to pan the viewport or keep content visible, not an intentional request for more UI. And because only a small portion of the screen is visible at once, users frequently cannot see the area they are interacting with and the area where the resulting information appears at the same time.</p>
Obscured content</h4>
At high zoom levels, the visible portion of the screen is tiny, and the magnified area usually follows the mouse pointer. So when a website decides to reveal extra information on hover, that information can end up covering the very thing the user was trying to look at.</p>
</p>
The obvious solution, moving the mouse elsewhere, doesn’t really work. Moving the pointer also moves the zoomed viewport, which can push the content completely off-screen.</p>
Important information outside the viewport</h4>
In some cases, the problem is spatial rather than transient. An interactive graph, for example, might show numerical values in a fixed corner of the screen while the user moves the mouse across the graph. Under full-screen magnification, the user may be able to see the graph or the readout, but not both at the same time. Moving the mouse to explore the graph also moves the zoomed viewport, making it difficult or impossible to keep the changing values visible.</p> Your browser does not support the video tag. </video>
The root problem is that, for magnification users, mouse movement is doing two completely different jobs at the same time. It’s being used both to indicate what they want to interact with, and to control which part of the screen they can see.</p>
The solution</h3>
Fortunately, these problems don’t require changes to every offending site. We can introduce some small tweaks on the user side to decouple pointer interaction from panning the viewport.</p>
In macOS, this kind of functionality is built-in. One of the “advanced options” of the full-screen magnifier</a> is “Detach zoom view from pointer.” When the user holds down the Control and Command keys, the mouse pointer is effectively frozen in place. Moving the mouse moves the zoomed area, but the focused application doesn’t register any mouse movement.</p>
Freezing the pointer</h4>
With some AI assistance, I’ve replicated this as a “freeze pointer” user script</a>, which can be installed in a Web browser through an extension like Tampermonkey</a>.</p>
Below, as illustrated earlier, the user is moving the mouse across a graph while the exact values appear in a fixed location outside the magnified area. With the frozen-pointer user script, however, the user can hold down the Control key to temporarily freeze the hover position on a single data point. This enables them to pan the magnified viewport until the readout is visible again, without changing which data point is selected under the mouse pointer.</p> Your browser does not support the video tag. </video>
Shielding the page from interaction</h4>
I also created a second user script that temporarily adds an “invisible wall” between the pointer and the web page</a>. This takes a more aggressive, but also more reliable, approach: instead of freezing the current hover state in place, it prevents the page from receiving pointer interactions altogether while the modifier key is held. The downside is that it disables hover behavior entirely rather than preserving the currently displayed item or tooltip, so it’s less helpful in cases like the interactive graph.</p>