I create applications, that are divorced from any native framework. All rendering happens in OpenGL, with a context provided by GLFW, all in C, with no framework to rely on supplying compatibility. As such, standard screen readers like NVDA have no chance of picking up information ( excluding OCR ) and my applications are an accessibility black hole.
How can I provide an interface for screen readers to cling unto? I presume this is a per OS thing... How would that be possible on Windows, Linux, BSD or even android? In the *NIX world, I presume this would be Desktop environment dependent...
I'm finding a lot of information on this, with a framework as a starting point, but have a hard time finding resources on how to do it from scratch.
I'm fully aware this is far beyond the capability of a sole developer and know, that writing programs by ignoring native interfaces is a common accessibility hole, which you are advised to avoid.
However, I have a tough time finding resources and jump-in points to explore this topic. Can someone point me in the right direction?
TL;DR: How to provide screen-reader compatibility from scratch. Not in detail - but conceptually.
As you have already well identified, your app is an accessibility blackhole because you are using a rendering engine.
It's basicly the same for OpenGL, SDL, or <canvas> on the web, or any library rendering something without specific accessibility support.
WE can talk about several possibilities:
Become an accessibility server. Under windows, it means doing the necessary so that your app provide accessible components on demand from UIA / IAccessible2 interface.
Use a well known GUI toolkits having accessibility support and their provieded accessibility API to make your app.
Directly talk to screen readers via their respective API in order to make them say something and/or show something on a connected braille display.
Do specific screen reader scripting
However, it doesnt stops there. Supporting screen readers isn't sufficient to make your app really accessible. You must also think about many other things.
1. Accessibility server, UIA, IAccessible2
This option is of course the best, because users of assistive technologies in general (not only screen readers) will feel right at home with a perfectly accessible app if you do your job correctly.
However, it's also by far the hardest since you have to reinvent everything. You must decompose your interface into components, tell which category of component each of them are (more commonly called roles), make callback to fetch values and descriptions, etc.
IF you are making web development, compare that with if you had to use ARIA everywhere because there's no defaults, no titles, no paragraphs, no input fields, no buttons, etc.
That's an huge job ! But if you do it really well, your app will be well accessible.
You may get code and ideas on how to do it by looking at open source GUI toolkits or browsers which all do it.
Of course, the API to use are different for each OS. UIA and IAccessible2 are for windows, but MacOS and several linux desktops also have OS-specific accessibility API that are based on the same root principles.
Note about terminology: the accessibility server or provider is your app or the GUI toolkit you are using, while the accessibility client or consumer is the scren reader (or others assistive tools).
2. Use a GUI toolkit with good accessibility support
By chance, you aren't obliged to reinvent the wheel, of course !
Many people did the job of point 1 above and it resulted in libraries commonly called GUI toolkits.
Some of them are known to generally produce well accessible apps, while others are known to produce totally inaccessible apps.
QT, WXWidgets and Java SWT are three of them with quite good accessibility support.
So you can quite a lot simplify the job by simply using one of them and their associated accessibility API. You will be saved from talking more or less directly to the OS with UIA/IAccessible2 and similar API on other platforms.
Be careful though, it isn't as easy as it seems: all components provided by GUI toolkits aren't necessarily all accessible under all platforms.
Some components may be accessible out of the box, some other need configuration and/or a few specific code on your side, and some are unaccessible no matter what.
Some are accessible under windows but not under MacOS or vice-versa.
For example, GTK is the first choice for linux under GNOME for making accessible apps, but GTK under windows give quite poor results. Another example: wxWidgets's DataView control is known to be good under MacOS, but it is emulated under windows and therefore much less accessible.
In case of doubt, the best is to test yourself under all combinations of OS and screen readers you intent to support.
Sadly, for a game, using a GUI toolkit is perhaps not a viable option, even if there exist OpenGL components capable of displaying a 3D scene.
Here come the third possibility.
3. Talk directly to screen readers
Several screen readers provide an API to make them speak, adjust some settings and/or show something on braille display. If you can't, or don't want to use a GUI toolkit, this might be a solution.
Jaws come with an API called FSAPI, NVDA with NVDA controller client. Apple also alow to control several aspects of VoiceOver programatically.
There are still several disadvantages, though:
You are specificly targetting some screen readers. People using another one, or another assistive tool than a screen reader (a screen magnifier for example), are all out of luc. Or you may multiply support for a big forest of different API for different products on different platforms.
All of these screen reader specific API support different things that may not be supported by others. There is no standards at all here.
Thinking about WCAG and how it would be transposed to desktop apps, in fact you are bypassing most best practices, which all recommand first above anything else to use well known standard component, and only customize when really necessary.
So this third possibility should ideally be used if, and only if, using a good GUI toolkit isn't possible, or if the accessibility of the used GUI toolkit isn't sufficient.
I'm the autohr of UniversalSpeech, a small library trying to unify direct talking with several screen readers.
You may have a look at it if you are interested.
4. Screen reader scripting
If your app isn't accessible alone, you may distribute screen reader specific scripts to users.
These scripts can be instructed to fetch information to give to the user, add additional keyboard shortcuts and several other things.
Jaws has its own scripting language, while NVDA scripts are developed with Python. AS far as I know, there's also scripting capabilities with VoiceOver under MacOS.
I gave you this fourth point for your information, but since you are starting from a completely inaccessible app, I wouldn't advise you to go that way.
In order for scripts to be able to do useful things, you must have a working accessible base. A script can help fixing small accessibility issues, but it's nearly impossible to turn a completly inaccessible app into an accessible one just with a script.
Additionally, you must distribute these scripts separately from your app, and users have to install them. It may be a difficulty for some people, depending on your target audience.
Beyond screen reader support
Screen reader support isn't everything.
This is beyond your question, so I won't enter into details, but you shouldn't forget about the following points if you really want to make an accessible app which isn't only accessible but also comfortable to use for a screen reader user.
This isn't at all an exhaustive list of additional things to watch out.
Keyboard navigation: most blind and many visually impaired aren't comfortable with the mouse and/or a touch screen. You must provide a full and consist way of using your app only with a keyboard, or, on mobile, only by standard touch gestures supported by the screen reader. Navigation should be as simple as possible, and should as much as you can conform to user preferences and general OS conventions (i.e. functions of tab, space, enter, etc.). This in turn implies to have a good structure of components.
Gamepad, motion sensors and other inputs: unless it's absolutely mandatory because it's your core concept, don't force the use of them and always allow a keyboard fallback
Visual appearance: as much as you can, you should use the settings/preferences defined at OS level for disposition, colors, contrasts, fonts, text size, dark mode, high contrast mode, etc. rather than using your own
Audio: don't output anything if the user can't reasonably expect any, make sure the volume can be changed at any time very easily, and if possible if it isn't against your core concept, always allow it to be paused, resumed, stopped and muted. Same reflection can apply to other outputs like vibration which you should always be able to disable.
The question is: Is it a good practice to use SpriteVisualElement instead of UIComponent to create custom components? The reason being I can't add Sprite or FlexSprite as direct children of View and I see that UIComponent is quite heavy then SpriteVisualComponent from code point of view.
I searched extensively to confirm if there are certain examples elsewhere which shows use of this control as the base to create custom controls but could only find this. So actually I was little less confident about weather this is Ok or not.
Below is the supplement information:
Nature of application: Educational game application for children
Target platform: Mobile device (currently only Android platform)
Application environment: Flex SDK4.6 with AIR 3.8. Blank Spark application with spark.components.ViewNavigator. PureMVC framework used. Multiple Views to contain custom components which mostly use graphics package to draw themselves and to draw child controls within them (This is where I actually extend SpriteVisualElement for custom drawing as well as containing and displaying other custom controls).
Please suggest me if what I am doing is a good practice for a mobile app or not.
Thanks in advance and regards,
Sachin.
I use it often as SpriteVisualElement is lighter. I always try to use lighter things if it's fit my needs so I think it's a good practice especially for mobile development.
I want to develop a desktop program, and one of the most important criteria is having a wonderful user interface - both in terms of usability and style. When I say style, I mean the cool designs (and effects) that one normally associates with Android/Apple applications. I wish to design a user interface, similar (or inspired from) to the one designed for Light Table by Chris Granger. Here's the link to Light Table: http://www.kickstarter.com/projects/ibdknox/light-table
Even Autodesk Maya has been designed using Qt.
Also, I wish to design the interface by drag-n-drop methods, so I finish the interface part quickly and move on to other important parts of the program.
I've worked with Java Swing and AWT. But they provide native interfaces, which are good and easy to use, but doesn't fit the style/cool factor. So I tried Qt. Qt is a major improvement over Swing, I felt. But being new to it, I haven't been able to judge it's full capability. Interfaces of VLC and Maya have been designed with Qt, so I know Qt is the way to go.
Now, Qt provides two major tools for UI design - Qt Designer and Qt Quick.
Their docs and site say that Qt Designer should be used for Desktop applications and that it would give you a native style interface. But I don't want a native style interface. Qt Quick on the other hand, promises to give me exactly what I want (stylish looks etc), but their doc says Qt Quick is to be used for Mobile Application.
Is it possible to use Qt Quick for desktop UI design?
Or, is it possible to extract out of Qt Designer, the kind of design I wish to implement?
I also liked the Autodesk Maya interface design and would like to give my program that kind of touch.
Besides, are there other or better cross-platform tools for UI design. Another reason for me choosing Qt is it is cross-platform. I want my program to run on Linux/Windows/Mac all the same.
I know similar questions have been closed for it being "impossible to answer objectively", but stick with me here.
I built a prototype in Adobe Flex, they (customers) liked it. Everything was fine until they later told me that iPads / iOS needs to be supported too.
I checked out Adobe's Packager for iPhone. We're evaluating that and we will know if it works out in a couple of days. (We need to get through Apple's red tape and certificates raj so this angle is delayed by a few days!)
There is a growing voice for using HTML5/Canvas as a technology platform itself. And despite being quite proficient in Flex, I think this makes sense.
I'm in need of a HTML5 library that can:
Render "widgets" i.e. containers with forms and components (this should be easy and possible using POHJC - Plain old HTML, JavaScript and CSS ;-) )
Provide a Tree like control for laying out some data
Provide a Canvas where data structures can be represented as basic shapes
Provide drag and drop capabilities between Trees, Buttons and Canvas
Provide some sort of Tab Navigator container (I guess JQuery works here)
Interact with back-end services (JSON/XML calls will be okay, but mapping directly with back-end entities will be awesome!)
Renders on latest versions of major browsers, Android OS and iOS (WebKit for mobile?)
Am I asking for too much?
I'm ready to give JQuery & JQuery UI a try.
I looked at Sencha / ExtJS but it seems we need to maintain two code bases one for normal browsers and the other for mobiles (is that correct?)
Are there any other JS libraries worth trying out?
My concerns areas are
Single code base, I don't want to suggest to them that multiple code bases for the client need to be maintained. That's a last resort option and would lead to complete ruling out of HTML5 with Flash apps and native apps being developed.
Canvas capabilities - I don't want to work with raw canvas and shape tags. This too is a last resort option. Is there any abstraction available?
Integration with back-end services, obviously I need some capability here!
Help me out. Communitywikify this if required.
Thanks,
Sri
How about trying Vaadin?
http://vaadin.com/comparison
I do construe its irrelevant to your question, but still this framework can help in great deal. I still use Flex and PHP as main base for many application, but actually fell in love with Vaadin and started using it for my new projects.
Grant Skinner the flash guru is working on a html5 libary. It has some features you requested. It is still in development.
The new Canvas element in HTML5 is powerful, but it can be difficult to work with. It has no internal concept of discrete display elements, so you are required to manage updates manually. The Easel Javascript library provides a full, hierarchical display list, a core interaction model, and helper classes to make working with Canvas much easier.
The libary is called easeljs, you can find it here : http://easeljs.com/
For the normal html and css manipulations without html canvas JQuery is very easy to learn.
I am working on a flex application.The application occupies 100% of the browser screen, but it is slow to load.How should i improve the performance, what are the best techniques one should follow for improving the performance of a flex App.
I did try some stuff though, i am not using relative layout instead i am using absolute layout and putting the buttons/images etc at exact places in a Canvas, still i feel it is slow.Can i profile the App or flex code to see where exactly the app is taking the maximum time to execute.
Adobe offers a great article about this topic: Flex Application Performance: Tips and Techniques for Improving Client Application Performance
Here are some great resources on improving performance of not only your Flex app, but of Actionscript in general:
Quick as a Flash Presentation by Grant Skinner - Optimizing Actionscript
Round up of Actionscript and Flex Optimizations
Flex Application Startup Performance
Actionscript Performance Tuning
Overall, some key things to always do:
Keep nesting thin: Don't nest too many VBoxes and Canvases, that will make your app dramatically lag/freeze
Don't Embed: Only embed things that are very small and need to show up immediately (icons in skins for example)
Deferred Instantiation: Only add what the user needs to see. Everything else will wait until the user goes there.
Explicitly size all item renderers. If DataGrid item renderers aren't explicitly sized, your app might freeze for an instance when constructing it.
Optimize Assets for the Web (icons ~< 20k, large images ~< 200k as a rule of thumb).
Refactor to remove excessing curly-bracket Binding (width="{something.other.width/4}"). Causes a bunch of events to be dispatched that you likely don't need. Too many events == too many temporary objects == lag.
People recommend using Modules to decrease swf size and only load what you need, but if your application isn't as large as a CMS, you generally don't need them.
Flex Builder Professional has a built-in profiler that works pretty well for profiling your own code. If the problem is drawing performance and layout, it's not as helpful (it profiles the code just fine, but it's harder to understand the results and apply them in a useful way to changes in your code).