Support for Server Annotation in Win32 list controls?

[skuskusku]

Detailed description of the issue

I am testing my own application against NVDA in order to see if my customers can work with our software and NVDA. Our application uses a standard Win32 list control in report mode and I want to override some accessibility names of list control items, namely empty ones, because Accessibility Insights for Windows flags empty list control items as errors in accessibility tests. I am doing this via Server Annotation and Accessibility Insights for Windows shows the content as provided by the server annotation code of mine, but NVDA does not, it shows the actual content of the list control items as they appear to sighted users. According to a test engineer in my team, JAWS also reads out the content provided by server annotation, if "Speak description of item" is turned on.

Why are the other templates not appropriate in this case?

I am unsure if this is a bug or a feature request or if I am just a plain dumb user who could not find where to turn on this feature in NVDA. It is also quite obscure to determine the correct list control item from Server Annotation and it looks like list controls with server annotation have some undocumented strange behaviour, see also this SO post:

https://stackoverflow.com/questions/40245989/listview-accessibility-issue-how-to-make-windows-narrator-read-the-entire-sel

Have you asked for advice on how to report this issue via a community discussion? If so, please link to the discussion

No.

Steps to reproduce or illustrate the issue (if applicable)

Use an app with a list control and provide the same code as in the SO post to override list control item accessibility names with Server Annotation.

Expected outcome or behaviour (if applicable)

NVDA should read out the text provided via server annotation.

Screenshots, logs, and other attachments (if applicable)

None, let me know what would be useful.

System configuration or software environment (if applicable)

Windows 11 x64 with latest NVDA as of now.

Additional information (if applicable)

None

[josephsl]

Hi, have you tested this with Narrator and see what it says/reports server annotations? Also, when you say latest NVDA, I’m assuming 2024.1 (please be specific). Thanks.

[josephsl]

Hi,

Also, can we get a develoepr info for the list view item? To do so, run NVDA, move to a list item, press Insert+F1, then copy and paste the developer info section. That way we can compare how the control is reported by Windows, the app, accessibility API's such as MSAA and UIA and NVDA itsel.f Also, it would be great to access the ap itself so someone from the project can do some debugging.

Thanks.

[skuskusku]

Dear josephsl,

Narrator speaks out the text I have overridden with Server Annotation, just like Accessibility Insights for Windows. And yes, I am using 2024.1.

You can download the nvda.log from this location:

https://mcblogs.craalse.de/skufiles/2024/may/nvda.log

The little project to reproduce is here:

https://mcblogs.craalse.de/skufiles/2024/may/biglistctrl.zip

Unzip it with directory preservation and build the .sln file with VS2022, maybe older versions work as well. Run the application and let NVDA or AI or whatever do its magic :-)

You will see that AI will report different accessibility names than what you can see in the individual list control items is shown to sighted users, that's what the server annotation code does. But NVDA only shows the item text that is shown to sighted users.

I am still not sure if this is due of a misconfiguration of NVDA on my side, so please bear with me.

I will be offline for a week or so, due to my vacation, so please have some patience the next week with responses expected from my side.

Regards,

--
Stefan

[josephsl]

Hi,

To build biglistctrl project, MFC (Microsoft Foundation Classes) library must be present (I did not install this initialy as I use only the VS build tools and NVDA does not need MFC to build itself). I told msbuild to build the target in x64 debug mode.

Testing results:

Software used: Windows 11 23H2 beta (build 22635.3646), NVDA 2024.3 alpha (alpha-32125,50da9987), JAWS 2024 (2024.2405.83), Narrator

1. Narrator announces server annotation text because it is what UI Automation tells it to say.
2. JAWS announces server annotations if mouse echo is on AND speak description of items is checked (mouse echo is not checked by default).
3. NVDA's behavior depends on which accessibility API is used. With MSAA (sometimes called IAccessible), NVDA says "header" and "row" numbers. With UIA mode, NVDA does announce "index" but it does not report server annotation text.

Based on the initial description, is the JAWS tester using the mouse to test your app with JAWS mouse echo set to on? If yes, this explains the result reported in an earlier comment. I am using keyboard, mouse, and touchscreen interaction to test the app.

As for server annotation, are you trying to set the item name property with this method? While it does work with Narrator, it does not work with NVDA because NVDA does not treat UIA elements with window class name of "SysListView32" as good UIA window, falling back to MSAA implementation. Internally, NVDA uses a combination of MSAA and list view column header/content getters (with some memory allocations involved) to obtain screen content for list view items, thus whatever is shown on screen is announced because that's what the app (more likely the framework) is telling NVDA to say.

To answer the linked Stack Overflow question, Narrator (not NVDA nor JAWS) announces the first column only because that's the name of the list view item reported by UI Automation itself. The rest of the "line" is ignored because UI Automation (and/or perhaps Narrator) does not have anything to report; there could be a property string (such as element description) with the rest of the line text as value, but somehow this is not communicated by the accessibility API. This indicates either something odd going on with UI Automation, Narrator, the program in question, or a combination of them. If Narrator does report the entire line of a list view item for another app, chances are that something odd is going on with this "server" a.k.a. app from the viewpoint of UIA.

Thanks.

[thgcode]

@josephsl Unfortunately the code is correct. I might have missed something, but from what I have discovered, currently it is impossible to override a multicolumn sysListView32 list item name (see the referenced part of the code). NVDA is getting the list item name directly from the list control's API, bypassing server annotations (or IAccessible in this case) and using the control's native API directly.
If the control had a different class name, nvda would work correctly, if this theory is indeed correct, maybe we will need to make an app module to override the behavior for this specific list.

Reference: https://github.com/nvaccess/nvda/blob/00f5ea24a44063b2f5442a91b873078e4ce8d9b6/source/NVDAObjects/IAccessible/sysListView32.py#LL622

[josephsl]

Hi,

For an NVDA add-on example that takes "advantage" of native SysListView32 API, see StationPlaylist (yep, I'm familiar with the linked code too, and thanks for clarifying it much better than I did). @LeonarddeR, is there anything you would like to add since I think you and Lukasz did some investigation in into SysListView32 and UIA before?

Thanks.

[Adriani90]

cc: @ABuffEr do you have any suggestions here maybe?
See also #16604 (comment)

[seanbudd]

it shows the actual content of the list control items as they appear to sighted users

Hi, we are closing this issue as intended as behaviour. Showing actual content which matches the sighted experience is intentional screen reader behaviour.

[thgcode]

Sorry, but I did not understand why the issue was closed, the author has overridden the accessible name property of the list item and NVDA completely ignored it, bypassing the author's code. In this case, that NVDA is speaking the entire column of text, how is the procedure to change the code to speak only the single row list item the user is focused in on the application itself, or the only way is to write an app module for the application or change the control?

[seanbudd]

I think changing the control is the only solution, but an add-on or appmodule for the specific application is also possible.
From what I understand, we cannot handle the generic case without impacting the accessibility of other applications.