I create applications, that are divorced from any native framework. All rendering happens in OpenGL, with a context provided by GLFW, all in C, with no framework to rely on supplying compatibility. As such, standard screen readers like NVDA have no chance of picking up information ( excluding OCR ) and my applications are an accessibility black hole.
How can I provide an interface for screen readers to cling unto? I presume this is a per OS thing... How would that be possible on Windows, Linux, BSD or even android? In the *NIX world, I presume this would be Desktop environment dependent...
I'm finding a lot of information on this, with a framework as a starting point, but have a hard time finding resources on how to do it from scratch.
I'm fully aware this is far beyond the capability of a sole developer and know, that writing programs by ignoring native interfaces is a common accessibility hole, which you are advised to avoid.
However, I have a tough time finding resources and jump-in points to explore this topic. Can someone point me in the right direction?
TL;DR: How to provide screen-reader compatibility from scratch. Not in detail - but conceptually.
As you have already well identified, your app is an accessibility blackhole because you are using a rendering engine.
It's basicly the same for OpenGL, SDL, or <canvas> on the web, or any library rendering something without specific accessibility support.
WE can talk about several possibilities:
Become an accessibility server. Under windows, it means doing the necessary so that your app provide accessible components on demand from UIA / IAccessible2 interface.
Use a well known GUI toolkits having accessibility support and their provieded accessibility API to make your app.
Directly talk to screen readers via their respective API in order to make them say something and/or show something on a connected braille display.
Do specific screen reader scripting
However, it doesnt stops there. Supporting screen readers isn't sufficient to make your app really accessible. You must also think about many other things.
1. Accessibility server, UIA, IAccessible2
This option is of course the best, because users of assistive technologies in general (not only screen readers) will feel right at home with a perfectly accessible app if you do your job correctly.
However, it's also by far the hardest since you have to reinvent everything. You must decompose your interface into components, tell which category of component each of them are (more commonly called roles), make callback to fetch values and descriptions, etc.
IF you are making web development, compare that with if you had to use ARIA everywhere because there's no defaults, no titles, no paragraphs, no input fields, no buttons, etc.
That's an huge job ! But if you do it really well, your app will be well accessible.
You may get code and ideas on how to do it by looking at open source GUI toolkits or browsers which all do it.
Of course, the API to use are different for each OS. UIA and IAccessible2 are for windows, but MacOS and several linux desktops also have OS-specific accessibility API that are based on the same root principles.
Note about terminology: the accessibility server or provider is your app or the GUI toolkit you are using, while the accessibility client or consumer is the scren reader (or others assistive tools).
2. Use a GUI toolkit with good accessibility support
By chance, you aren't obliged to reinvent the wheel, of course !
Many people did the job of point 1 above and it resulted in libraries commonly called GUI toolkits.
Some of them are known to generally produce well accessible apps, while others are known to produce totally inaccessible apps.
QT, WXWidgets and Java SWT are three of them with quite good accessibility support.
So you can quite a lot simplify the job by simply using one of them and their associated accessibility API. You will be saved from talking more or less directly to the OS with UIA/IAccessible2 and similar API on other platforms.
Be careful though, it isn't as easy as it seems: all components provided by GUI toolkits aren't necessarily all accessible under all platforms.
Some components may be accessible out of the box, some other need configuration and/or a few specific code on your side, and some are unaccessible no matter what.
Some are accessible under windows but not under MacOS or vice-versa.
For example, GTK is the first choice for linux under GNOME for making accessible apps, but GTK under windows give quite poor results. Another example: wxWidgets's DataView control is known to be good under MacOS, but it is emulated under windows and therefore much less accessible.
In case of doubt, the best is to test yourself under all combinations of OS and screen readers you intent to support.
Sadly, for a game, using a GUI toolkit is perhaps not a viable option, even if there exist OpenGL components capable of displaying a 3D scene.
Here come the third possibility.
3. Talk directly to screen readers
Several screen readers provide an API to make them speak, adjust some settings and/or show something on braille display. If you can't, or don't want to use a GUI toolkit, this might be a solution.
Jaws come with an API called FSAPI, NVDA with NVDA controller client. Apple also alow to control several aspects of VoiceOver programatically.
There are still several disadvantages, though:
You are specificly targetting some screen readers. People using another one, or another assistive tool than a screen reader (a screen magnifier for example), are all out of luc. Or you may multiply support for a big forest of different API for different products on different platforms.
All of these screen reader specific API support different things that may not be supported by others. There is no standards at all here.
Thinking about WCAG and how it would be transposed to desktop apps, in fact you are bypassing most best practices, which all recommand first above anything else to use well known standard component, and only customize when really necessary.
So this third possibility should ideally be used if, and only if, using a good GUI toolkit isn't possible, or if the accessibility of the used GUI toolkit isn't sufficient.
I'm the autohr of UniversalSpeech, a small library trying to unify direct talking with several screen readers.
You may have a look at it if you are interested.
4. Screen reader scripting
If your app isn't accessible alone, you may distribute screen reader specific scripts to users.
These scripts can be instructed to fetch information to give to the user, add additional keyboard shortcuts and several other things.
Jaws has its own scripting language, while NVDA scripts are developed with Python. AS far as I know, there's also scripting capabilities with VoiceOver under MacOS.
I gave you this fourth point for your information, but since you are starting from a completely inaccessible app, I wouldn't advise you to go that way.
In order for scripts to be able to do useful things, you must have a working accessible base. A script can help fixing small accessibility issues, but it's nearly impossible to turn a completly inaccessible app into an accessible one just with a script.
Additionally, you must distribute these scripts separately from your app, and users have to install them. It may be a difficulty for some people, depending on your target audience.
Beyond screen reader support
Screen reader support isn't everything.
This is beyond your question, so I won't enter into details, but you shouldn't forget about the following points if you really want to make an accessible app which isn't only accessible but also comfortable to use for a screen reader user.
This isn't at all an exhaustive list of additional things to watch out.
Keyboard navigation: most blind and many visually impaired aren't comfortable with the mouse and/or a touch screen. You must provide a full and consist way of using your app only with a keyboard, or, on mobile, only by standard touch gestures supported by the screen reader. Navigation should be as simple as possible, and should as much as you can conform to user preferences and general OS conventions (i.e. functions of tab, space, enter, etc.). This in turn implies to have a good structure of components.
Gamepad, motion sensors and other inputs: unless it's absolutely mandatory because it's your core concept, don't force the use of them and always allow a keyboard fallback
Visual appearance: as much as you can, you should use the settings/preferences defined at OS level for disposition, colors, contrasts, fonts, text size, dark mode, high contrast mode, etc. rather than using your own
Audio: don't output anything if the user can't reasonably expect any, make sure the volume can be changed at any time very easily, and if possible if it isn't against your core concept, always allow it to be paused, resumed, stopped and muted. Same reflection can apply to other outputs like vibration which you should always be able to disable.
Are there any image analysis or object recognition libraries that would be suitable for use in browser emulation? Eg: automation of user logins, filling forms, extracting data from tables, etc.
I understand that HTML parsers are more commonly used than image analysis algorithms for the purpose of browser emulation. However, I would like to automate tasks on websites that frequently change their html code and appearance. I would like to build an "intelligent" scraper that can recongize these visual browser elements in conjunction with conventional browser emulation techniques.
Are there any existing libraries for this purpose? Ones that would be suitable for screenshots of GUIs? (non-photographic images) such as buttons, tables, checkboxes and text?
"Selenium automates browsers. That's it."
I am more of a server side programmer so bear with me on this. How exactly are user interfaces for websites designed? I mean which tools are usually used? Lets say for example, stackoverflow.com which has lot of dynamic content. How are the various areas designed? I am pretty sure its not in Visual studio. Probably the server side code is in asp.net but what about the actual UI? (layout, images, tables, buttons etc)
What is the usual workflow for an activity like this? Say, I have a design on paper. Where do I go from there? How do you wire in the code after the interface design is complete?
How do you handle the fact that in a page, some of the stuff is static and some areas are dynamic? (like the ask question page I am on now)
As you said, It boils down to the requirement of the webpage.
For a professional (fairly big) website, many teams are involved for example, creative team to do the paper work and design of UI elements and controls, graphics team to actually design images, UI Developers for placing the contents appropriately and CSS, architects to decide on performance for various items (and taking a call on static/dynamic nature of controls)
Generally designers use some external tools for designing HTML pages to provide templates and same can be used later in visual studio to make actual pages. There are many such tools available in the market such as Dreamweaver There are many freeware also available in the market for designing client and CSS rich websites. You can search on Google for these.
If your website requirement is not very client rich, you can still design using visual studio or use new Microsoft product Webmatrix which gives you user friendly tools to make a website look fabulous.
The paper design is the first correct step.
How to continue:
You can get the 960 grid system from http://960.gs/ and start from this one. Its a nice trick that have ready to use css templates that you can build on them your design.
The image effects:
The shadow and borders and other thinks that you ask is usually make on Photoshop, but now the new browsers support many of them using css. For example: http://css-tricks.com/snippets/css/css-box-shadow/
Software that can help on design:
- MS Expression
- DreamWeaver
*Overview:*
Making a multimedia website. Primary focus is to click and drag video clips from any computer file to the website instantly. Has to be very simple and user friendly. Editing videos by creating effects, cut/slash video parts, adding animation, fill effects, caption, different fonts of videos, etc. We do not want a software for the user to download, everything has to be done on the website.
Is this possible in asp.net.
asp.net by itself cannot do this.
You will need help from several sources:
Third party libraries to transform uploaded video from mpg, flash,
mov to whatever you will use.
Anti Virus software to scan the files before they can be transformed
Silverlight to edit video online
But yes, asp.net can support the underline web site.
You can use Silverlight. It runs on client side and can be hosted in asp.net.
Yes it is possible as with most things, but I am not sure you realise what a big piece of coding you are talking about. There are a number of online video editors (YouTube being the most famous) but the issues you will face are huge, such as multiple codecs and conversion to and from and bandwidth issues for uploading videos and downloaded edited content.
Have a look at some of the online video editors:
http://savedelete.com/10-best-and-free-online-video-editing-software.html
and then consider how much budget you have to create your tool. You may wish to use a more clientside approach with silverlight or adobe air but still you face a huge amount of coding to do this right. You will also need to draw upon third party and open source libraries for the actual editing of the videos.
Good luck with the project.
I'd like to set up a CMS that allows clients to generate a PDF brochure from a series of text inputs and text areas (rich text editor). Clients would fill out text inputs and then select (via a check box) which contents should appear in the exported PDF. This would then stitch together a PDF brochure containing pages only applicable to the options chosen by the client.
All the content is in a CMS with web pages and pdfs built on the fly as per the client's choices. A change in the content is then reflected immediately in the published docs meaning no redundant, out of date pdfs and the benefit of all the tools within the CMS (workflow, publishing dates, security, etc).
Does anyone know of any tools that do this?
Appreciate any help.
well your question require a lot of work to be done. I had similar requirement and what i have done i can brief here. I created multiple views which lets me select their columns and i have used my own rules for differentiating columns from test like {view.columnname}. when your save all information i have used itextsharp for pdf generation.
There is plenty of information on itextsharp.
You might be able to do this with design tools and reporting software, plugged into your CMS of choice. For example, you could use Wordpress, and then design your PDF output using iReport, and run it with Jasper. You'd have to work on the bit to translate user inputs to a query passed to the report, but that shouldn't be too much work.
One issue is that, regardless of the work you put into the design, when you attempt to share it, it wont look like a brochure at all, only sequential single page PDFs. As an alternative I found a website where you can upload or design your brochure, it can then be downloaded and shared as a flippable, digital brochure. If you are interested check out https://simplebooklet.com/learn/animated-tri-folds.php