a11y - Building the most accessible terminal

[codeofdusk]

Discord username (optional)

No response

Describe the bug

The app does not read with the [VoiceOver](https://en.wikipedia.org/wiki/VoiceOver) screen reader.

To Reproduce

Steps to reproduce:

1. Press Command+F5 to start VoiceOver.
2. Launch Warp.

Expected behaviour

The app reads its visible controls (terminal input/output, with automatic reading of changes).

Screenshots

No response

Operating System

MacOS

OS Version

12

Additional context

No response

[gagata]

hey! thanks for the feedback - we've done some work on accessibility and while it's not working with custom voice over commands, it should be able to provide some context to the user. we've worked with few engineers gathering their feedback on that, but more is always welcome :)

here's the docs on how warp works with the voice over in its current state: https://docs.warp.dev/features/accessibility

[godbout]

i'm coming here from the feedback form. so as asked there, i'm adding my +1. i build apps that use the macOS AX extensively and Warp is plainly untouchable. thanks!

[gagata]

i'm coming here from the feedback form. so as asked there, i'm adding my +1. i build apps that use the macOS AX extensively and Warp is plainly untouchable. thanks!

hey! in this week's release (that's coming out today) we've done some fixes to the announcements, which should've improved the UX.

thanks for your feedback @godbout do you have more detailed information as to what should be improved?

[godbout]

hey @gagata thanks for keeping in touch. i'm not a Warp user myself. i'm reporting on behalf of users of my apps, that brought to my attention that it's not working with Wrap. my apps are acting like a screen reader to grab Accessibility info. as far as i understand, Warp doesn't provide any standard AX info and has its "own" VoiceOver/announcement system? so, like the author of this thread, i'm talking about the macOS standard VoiceOver and AX system. that would be nice to have that :))

[godbout]

to give you an idea visually: one of my app pings UI elements so that you can reach them through the keyboard. Warp provides no info at all (except the standard traffic lights):

ScreenFlow.mp4

[gagata]

to give you an idea visually: one of my app pings UI elements so that you can reach them through the keyboard. Warp provides no info at all (except the standard traffic lights):

thanks for posting! sorry for the delayed response...

overall Warp has its own UI framework which doesn't expose the element tree to the system (and from what i've seen so far, this is pretty common between rust UI frameworks). This means that those elements are not "pingable".

right now we've added support to warp to "announce" certain events / actions / focused elements using the mac's NSAccessibility notifications (https://developer.apple.com/documentation/appkit/nsaccessibilitynotificationname?language=objc) to open up at least some functionality to blind users. we've tested it with a few users, and while we're aware it's not ideal yet, they found it a pretty cool stop gap that already seemed more accessible than iterm or terminal.app (ie. keyboard access to last commands output etc).

I'm happy to chat more about your app, and whether we can make it work with warp. i know we have a lot to improve in the a11y realm, for sure :) feel free to ping me on cieplik@ warp.dev - maybe we can schedule some zoom call to chat more?

[godbout]

thanks for posting! sorry for the delayed response...

no worries.

overall Warp has its own UI framework which doesn't expose the element tree to the system (and from what i've seen so far, this is pretty common between rust UI frameworks). This means that those elements are not "pingable".

right now we've added support to warp to "announce" certain events / actions / focused elements using the mac's NSAccessibility notifications (https://developer.apple.com/documentation/appkit/nsaccessibilitynotificationname?language=objc) to open up at least some functionality to blind users. we've tested it with a few users, and while we're aware it's not ideal yet, they found it a pretty cool stop gap that already seemed more accessible than iterm or terminal.app (ie. keyboard access to last commands output etc).

yeah, the most important is definitely that you have some sort of system for blind/low vision users. i'm just using myself the technology behind VoiceOver to build productivity apps not related whatsoever do disability actually. so it's on me, not you.

I'm happy to chat more about your app, and whether we can make it work with warp. i know we have a lot to improve in the a11y realm, for sure :) feel free to ping me on cieplik@ warp.dev - maybe we can schedule some zoom call to chat more?

will ping you. thanks!

[elviskahoro]

Some great insight from a member of our community:

1. The ability to "interact" with one or more of the output blocks.
   E.g. like a read-only editor where you could select a line, word or marked selection for copying to the clipboard.
   Imagine trying just to get PID from "ps -aux" for one process only using keyboard navigation.
2. If you have multiple tabs open and keyboard between them using direct goto-tab number or next/prev tab, it would be nice if the title showed some metadata about that particular tab. E.g. current host/dir or something.
   Of course, PS1 support would go a long way to helping this. (For some users)
   What I'd like to see is something like what Safari does with multiple tabs. You can use the kbd to navigate to the tab bar and either activate or close a tab from there. It's not perfect, but quite handy.

[elviskahoro]

Going to convert this issue into a GitHub discussion so that it's easier for us to continually update the community on VoiceOver support and a11y in general (use this GitHub issue as a homebase for accessibility).

[elviskahoro]

We got some great feedback regarding support for voice-control accessibility tools like Talon Voice.

My biggest need is for a terminal which could integrate with cursorless-talon (or something similar) to easily navigate, search, copy-paste (e.g. "bring") using voice commands.