Skip to content

About

Speech, braille and screen reader output for .NET through prism, with Native AOT support.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

 
 

Repository files navigation

Refractor

.NET bindings to prism, which puts screen readers and speech engines behind one API, on Windows, macOS, iOS, Linux and Android. On Windows that means NVDA, JAWS, ZoomText, SAPI, OneCore, UI Automation and more. On macOS and iOS it means VoiceOver and AVSpeech, on Linux Speech Dispatcher and Orca, and on Android the running screen reader and Android text to speech. Refractor covers the whole prism C API. It works with Native AOT, and links prism into the executable so that it ships as one file.

The guide and API reference are at trypsynth.github.io/refractor.

Quick start

using Refractor;

using Prism prism = new();
using Backend backend = prism.CreateBest();
backend.Speak("Hello from .NET!");

CreateBest picks the highest priority backend that starts. When a screen reader is running, that is the screen reader, so speech goes through the user's own voice and settings. Otherwise it falls back to a system voice.

Picking a backend

Method State Initialized on return
CreateBest() Yours alone Yes
Create(id) Yours alone No, call Initialize()
AcquireBest() Shared with other callers Yes
Acquire(id) Shared with other callers Maybe, call Initialize()

Use CreateBest unless you need something else. Initialize returns false when the backend was already initialized, which is not an error. Ids for every known backend are on BackendId, for example BackendId.Nvda.

CreateBest and AcquireBest skip backends that report BackendFeatures.RequiresExplicitSelection, so you only get those by id. Two are built in: BackendId.Clipboard puts text on the clipboard, and BackendId.NoSpeech accepts every call and makes no sound, which is handy in tests. BackendFeatures.IsScreenReader and BackendFeatures.IsSpeechEngine tell the two kinds of backend apart.

Not every backend supports every operation. Check GetFeatures() or Supports(...) first. An unsupported operation throws PrismException with PrismError.NotImplemented.

A backend is not thread safe. Use one instance from one thread at a time, or create one per thread.

Knowing when a screen reader starts or stops

using Prism prism = new(new PrismOptions {
	AvailabilityChanged = (id, name, isAvailable) => Console.WriteLine($"{name}: {isAvailable}")
});

The callback runs on prism's poll thread. A backend that is already running when polling starts is not reported.

Custom backends

Derive from CustomBackend, override the operations you support, and name the same operations in the features you register with:

using RegistryBuilder builder = new();
BackendId id = builder.AddBackend("Console", 100, BackendFeatures.Speak, () => new ConsoleBackend());
using Registry registry = builder.Freeze();
using Prism prism = new(new PrismOptions { Registry = registry });

sealed class ConsoleBackend : CustomBackend {
	public override void Speak(string text, bool interrupt) => Console.WriteLine(text);
}

RegistryBuilder.AddLibrary loads prism plugin libraries the same way.

Logging

PrismLog.SetLevel(LogLevel.Warning);
PrismLog.MessageLogged += (_, e) => Console.Error.WriteLine($"{e.Level} {e.Source}: {e.Message}");

Setting the PRISM_LOG environment variable to warn, debug and so on makes prism log to standard error without any code.

F# and Visual Basic

Refractor works the same from F# and Visual Basic, including custom backends. TryFindBackend reads naturally in both:

use prism = new Prism()
match prism.TryFindBackend "NVDA" with
| true, id -> printfn "NVDA is %O" id
| false, _ -> printfn "NVDA is not registered"
Using prism As New Prism(), backend = prism.CreateBest()
    AddHandler PrismLog.MessageLogged, Sub(sender, e) Console.Error.WriteLine(e.Message)
    backend.Speak("Hello from Visual Basic!")
End Using

samples/SpeakFSharp and samples/SpeakVisualBasic are complete programs.

Native AOT

With PublishAot on, Refractor links prism statically. The published executable needs nothing beside it. Set RefractorStaticLink to false to ship prism next to the executable instead, as prism.dll on Windows, libprism.dylib on macOS or libprism.so on Linux.

Without Native AOT, prism loads from the package's runtimes folder.

Platforms

Platform Runtimes Backends Oldest version
Windows win-x64, win-arm64, win-x86 NVDA, JAWS, ZoomText, SAPI, OneCore, UI Automation and more Windows 10
macOS osx-arm64, osx-x64 VoiceOver, AVSpeech macOS 12
iOS ios-arm64, iossimulator-arm64, iossimulator-x64 VoiceOver, AVSpeech iOS 14
Linux linux-x64, linux-arm64 Speech Dispatcher, Orca glibc 2.35, as in Ubuntu 22.04
Android android-arm64, android-x64 The running screen reader, such as TalkBack, and Android text to speech Android 8 (API 26), with net10.0-android

An iOS app cannot load a library of its own, so Refractor always links prism into the app there. There is nothing to set up.

On macOS, VoiceOver speech works without any setup. To use VoiceOver's older automation path, an app bundle must declare NSAppleEventsUsageDescription in its Info.plist. A notarized app also needs the com.apple.security.automation.apple-events entitlement. Sandboxed apps cannot use this path.

On macOS and iOS, VoiceOver and AVSpeech do their work on the main thread, so a call from another thread waits for the main thread to run its run loop, as it always does in an app with a user interface. A console app has no run loop, so it should make every backend call from its main thread. In an async Main, that means before the first await, since the code after it usually continues on another thread. If the main thread is blocked waiting for such a call, the call never returns.

On Linux, prism needs libdbus-1, which every desktop distribution has. Speech goes through Speech Dispatcher, so it has to be installed with a voice, as it is on most desktops. Native AOT links with clang, so a Linux publish needs clang and zlib1g-dev or their equivalents.

On Android, Refractor brings prism's library archive, which merges into your app's manifest and sets prism up when the app starts. Starting Android's speech engine waits for a callback on the main thread, so call CreateBest, AcquireBest and Initialize from another thread, or the app hangs for 10 seconds and the backend fails to start. Android always uses the shared prism, Native AOT or not.

The raw API

Refractor.Interop.PrismNative exposes every prism function, struct and constant one to one, including the plugin ABI types, for anything the safe types do not cover.

Building

Prism is a git submodule, built with CMake. Windows needs Visual Studio with the C++ tools, and macOS and iOS need Xcode. Linux needs GCC 12 or newer, PowerShell, and the development packages for D-Bus, Speech Dispatcher, xcb and Wayland, which on Ubuntu are libdbus-1-dev libspeechd-dev libxcb1-dev libwayland-dev. Linux runtimes are only built on a machine of the same architecture. The build script takes the runtimes to build, and defaults to win-x64:

git clone --recurse-submodules https://github.com/trypsynth/refractor
cd refractor
./scripts/build-native.ps1 -Runtimes osx-arm64
dotnet test -p:RefractorTestRuntime=osx-arm64

dotnet pack needs prism built for every runtime in the package, so CI builds the packages.

License

Refractor is MIT licensed. The package also contains prism, which is under the Mozilla Public License 2.0, and the libraries prism uses. See THIRD-PARTY-NOTICES.md.

About

Speech, braille and screen reader output for .NET through prism, with Native AOT support.

Resources

Code of conduct

Contributing

Security policy

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages