Provide access to graphics written to the screen

[nvaccessAuto]

Reported by afsmel on 2011-02-17 23:34
while NVDA PRODUCES SOUNDS TO LET USER IDENTIFY THE LOCATION OF THE POINTER ON A SCREEN, I FIND IT WILL EMPOWER THIS WAY OF DEALING TO LET NVDA PRODUCES ANOTHER SOUND TO INFORM THE USER THAT THE MOUSE IS IS ON A GRAPHIC NOW, SPECIALLY WHEN THE HEAD OF THE ARROW IS LOCATED IN THE CENTRE OF THE GRAPHIC SO THAT ANY CLICK WILL BE FUNCTIONABLE. FURTHER, NVDA MAY HELP WITH INFORMING THE USER WITH THE REQUIRED MOVEMENT, AND HOW IT IS TO BE DONE, THIS IS IN BEGINNER MODE ONLY, THIS WILL GIVE THE USER THE ABILITY TO DEAL WITH GRAPHICAL INTERFACE SOFTWARES, WITHOUT RELYING ON MSAA. PARTICULARLY, IF USER CAN ASSIGN CERTAIN TEXT TO THE GRAPHICS HE NEEDS TO DEAL WITH IN A SOFTWARE, AND STORE THEM TO A USER DEFINED FILE, FOR WHEN LOCATING THEM IN ANOTHER TIME, NVDA INFORMS THE USER WITH THE ASSIGNED TEXT TO THIS GRAPHIC, ALSO, USER MAY BE IN NEED TO STORE THIS FILE ON A STORAGE MEDIA TO INSTALL IT AFTER HAVING ANOTHER VERSION OF THE SOFTWARE, OR ON ANOTHER SYSTEM.

[nvaccessAuto]

Comment 1 by jteh on 2011-02-21 00:22
The scope of this ticket is too large, as it covers multiple major enhancements and doesn't clearly define them. I'm narrowing it to just cover providing access to graphics written to the screen, as this is needed before any of the other ideas can be considered.
Changes:
Changed title from "empowering use of mouse with nvda" to "Provide access to graphics written to the screen"

[LeonarddeR]

@jcsteh: Would it be technically possible to get access to graphics and shapes on the screen for programs which expose output accessible using display model / screen review?

[jcsteh]

@michaelDCurran is the display model expert, so he'd know better. Having said that, I believe it is possible to detect graphics. One problem is calculating a persistent identifier for them. Without that, you can't label them.

P4 because GDI is becoming less and less prevalent, but there are still legacy use cases that need this.

[michaelDCurran]

We could certainly track when an icon is drawn to the screen. We could expose either its file path, or some kind of hash of the data.

[LeonarddeR]

@michaelDCurran commented on 19 jul. 2017 04:13 CEST:

We could certainly track when an icon is drawn to the screen. We could expose either its file path, or some kind of hash of the data.

Would all of this require additional hacking into the C code, or is there already a Pythonic way here to get graphics info? I'm asking because at @BabbageCom, relying on display model info is still quite prevalent, and it would really help if we could at least identify images by hash or file path as you mentioned.

[Adriani90]

Any update on this issue? Does anybody start working on it?

[Adriani90]

This could be a great improvement for the golden cursor addon, at least this would allow for a good test environment before implementing something similar into the core.

cc: @josephsl, @Robert-J-H

[Robert-J-H]

I'm not sure that I understand the proposal.
There are Windows functions that can get the resource ID for icons, cursor and bitmaps. The file path is probably not very useful since a lot of resources are just packed into the executable. Admittedly, it would be advantageous to have at least the areas of the graphic clips and hashes could be done with e.g. md-5 checksums or whatever is fastest. Given the dynamic and animated nature of today's graphics, it won't probably be very reliable.

[cary-rowen]

There are many software developed in China that do not follow the barrier-free standards. MSAA is completely invalid. It would be very good if the proposal can be reached. I am looking forward to it.

[josephsl]

Hi,

I'm no longer maintaining Golden Cursor add-on, so this is beyond my area at this time (sorry).