Libraries for gaze tracking with consumer hardware - accessibility

Does anyone know of a good gaze tracking library I can use with consumer hardware (I have a MacBook Pro on hand that I can put Windows or Linux on, and can acquire a selection of consumer to prosumer grade cameras) to build a very simple visual typing interface with?
Essentially so that the user can look at a keyboard (fullscreen) and blink to click the key. Or something like that. It needs to be something users are already familiar with (QWERTY).

Before I get all non-technical, check out Dasher. I used it for a long while on my handheld to type out long messages and it works like a charm. No hunting and pecking, just stare in the direction you want it to move (I used my stylus).
My grandfather was paralyzed from the eyes down from a stroke when I was just a baby. My grandmother worked out a way of talking to him using a piece of cardboard. She had three rows of letters: 1st half, 2nd half, and 3rd half (yes, third half. why, I don't know). The first half had letters A-H, second had I-Q, third had R-Z. She would hold the card up and go through his message letter-by-letter. He would blink when he wanted to choose a letter.
First, she would ask about which half: "First half? Second half?" etc. He would blink when she was on the right one. Then she would start reading off letters and he would blink when she got to the right one. She would right it down.
There was no "space", so you need to do some guess-work to determine some meanings.
Granted, this isn't as "techie" or cool as you might hope for, but it's a hell of a lot more personal than coldly blinking at a computer screen. Also, if the family is going to be using this with her, guess who's going to be touching the computer when you're gone? Not you, and not her. And families aren't great with computers.
Just a suggestion, and it only takes about five minutes to make, requires a sharpie and a pizza box, and only about a minute or two to explain. Sure, it's tedious, but it's also simple and effective.

Look at http://www.cogain.org/eyetrackers/low-cost-eye-trackers
there are free and open source offerings listed there, including one from codeproject (C++)

Windows 10, as of the Fall Creator's Update, has built-in support for Eye Control and the feature also includes a speech keyboard for communication.
The feature works with any of the supported eye tracking devices, one of which is a low cost but high quality consumer focused product, the Tobii Eye Tracker 4C.
For programmers Windows 10 exposes the Gaze APIs and there is also an open source gazeinteraction library that is part of the Windows Community Toolkit to make it very simple to create a UWP application that responds to your eyes.

Related

Micro:bit Bluetooth Low Energy Hacking Persistence (High School Internship Project)

My project is to create an interactive program using the Micro:bit microprocessor I'm building a game which uses a drill motor as a controller of sorts reading the rotation direction and speed as inputs for control
but my mentor also said it would be cool to power the board at the same time as the game is running so now I hit the situation where once I stop turning to change direction or my speed goes below transmitting 3.3 volts to power it then the game restarts and I lose all progress
I had the Idea of using a second micro:bit as a sort of storage place being powered by my computer and the two continuously communicating sending back player position and other objects on the LED's
but i can't figure out how to get the two Micro:bit's to talk to each other
If someone could just point me in the right direction or even set up some sort of communication to nudge me in the right direction as I start moving forward
i'm a high school student who doesn't know as much as I pretend to so I'll probably need a lot of help (i am more advanced then most in my class at this sort of thing so think of me as a tech gifted teenager thrown in with college students losing my undeserved ego day by day LOL) please help me somehow I'm currently completely lost
You won't be able to use Bluetooth for the reasons indicated in the documentation (not enough memory): http://microbit-micropython.readthedocs.io/en/latest/ble.html
However, there is an incoming implementation of the lighter radio module, which would allow you to send simple data: https://github.com/bbcmicrobit/micropython/pull/283
The proposed documentation can be found in: https://github.com/bbcmicrobit/micropython/pull/305
As you can see in GitHub, at the time of writing it has not yet been merged into micropython. So if you'd like to try it you would have to clone the repository, apply the patch and build it from source. Keep in mind there is risk for the API to change, as there are still discussions about it.
Alternatively, as Sean has mentioned, you can use the C++ DAL implementation of the radio module to get something running on the meantime. Or if you prefer, the blocks and touch develop languages also offer radio functionality.
I don't think there is a way to do this in micropython (or at least simply), but the microbit runtime docs describe that, as well as supporting bluetooth, the 2.4 GHz radio:
However, it can also be placed into a much simpler mode of operation based that allows simple, direct micro:bit to micro:bit communication
In order to use this, you might need to write in c++ using the mbed environment (or offline) - but I hope this at least gives you a pointer to start from.
Here's a blog post describing how to do data logging using two microbits in exactly the configuration you describe.
http://www.suppertime.co.uk/blogmywiki/2016/06/microbit-logger
How to get the two micro:bits to talk to each other
As of 2016, you can! First check micropython has the radio module
import radio
If you get the error "No module named 'radio'", use https://codewith.mu/
Then follow the radio tutorial https://microbit-micropython.readthedocs.io/en/latest/tutorials/radio.html
The API is
https://microbit-micropython.readthedocs.io/en/latest/radio.html

Inform 7: Pick up thing based on if specific thing is in inventory

I'm making an adventure with Inform 7. In the adventure, you can pick up a chip. I want to be able to pick up a thing called a pocket computer if you have the chip in your inventory. To put it in a possible inform sentence:
If chip is in the inventory, take pocket computer.
I think you'd be best served by going about it the other way. After all, portable objects normally can be picked up; what you seem to want to do is to prevent taking it if they don't have the chip. I think something like this could work:
Instead of taking the pocket computer when the player does not have the chip:
say "You need the chip first!"
However, I would consider whether it is reasonable to prevent the player from picking it up. Is the pocket computer fixed in place somehow, and the chip is required to unlock it? Or is it simply that the computer isn't useful until they have the chip? If the latter is the case, I would suggest that you still allow the player to pick it up, since in real life there would be nothing preventing them from doing so. It breaks immersion to prevent a reasonable action simply because it isn't useful yet.

Reading and understanding MCU datasheet and codes

Are there any tips for reading source code samples from manufacturer of MCUs'.
I am a newbie for mcu programming, currently I have a MCU, datasheet and sample codes for them. But problem is sample codes are seems written for experienced users. Too many questions about why they initialized RS232, why they set 4th bit of port 1 and etc.,
Do you have a tips for reading or links where can I get info about how to read datasheets and sample codes of MCU?
I guess experience is the only answer I can give. Just like with programming in general, with time you acquire experience as well as learn buzz words and concepts. With microcontrollers you learn to read datasheets, schematics, etc. Learn about open drain, open collector, weak pull ups, etc. And for serial ports for some reason they are always overcomplicated. The hardest part with microcontrollers and the serial port is usually figuring out what to program to get the right clock divisors, some microcontroller serial ports are straight forward, others are overly complicated, some docs are good some docs are bad, etc.
Another answer is datasheets are always wrong. There are always gaps in the information that you have to hack to figure out. Do not write thousands of lines of code in a vaccuum using only a datasheet, write a small amount of code a few lines to a few dozen, test, and move on, you can get more lines written and debugged in a day when programming from a datasheet than the other path. The datasheets are often not written by the engineers that actually designed the hardware, sometimes it is a junior engineer or a non-engineer. Sometimes the information is simply wrong, sometimes the document is for a different but similar part than the one you have. If they provide software that actually does stuff it is sometimes (not always) more accurate than the datasheet (when I say datasheet assume the users manual, programmers reference manual, whatever the vendor calls the doc with the registers, addressses, and bit definitions for the hardware).
With time and experience you may find, if you take a wide enough view, that some vendors tend to do a better job at providing information to users, others do not, some bury the secrets in libraries, sometimes in binary form and not source. Sometimes the secrets are buried in compilers and other tools they provide (well that is back to apis and libraries). I tend to blacklist such companies, but sometimes you cant always. ARM for example does a very good job of providing the information. the problem is they have so many cores with a number of options each, that are very similar in nature (support the same instruction sets) that it can be difficult to sort through what the one processor you are using that moment does and does not from the docs. Atmel, something about atmel that is hard to put a finger on, the docs are generally well above par, but more than that something about atmel makes them popular with the customers. You will never see an arduino like following, culture, pick a word, with a microchip pic for example. There are a lot of pic followers but it is not like the atmel world (which was there well before the arduino thing happened).
Another note, you might not understand with a single example program and single datasheet the history of a product, there might be code that has been used for a number of chip generations, and there might for example be a bit that is required by an older chip or newer chip and to share the same code that bit is manipulated. that bit might make sense looking at one datasheet and no sense looking at another. this is where hacking comes, in try it without, see what happens. maybe study other parts in the family that this code is said to support it might make more sense.
google is your friend or whatever favorite search engine, find as much open source code and other items for the particular device or whatever. At this level hacking is required, I dont use that term in the bad sense, hacking in the sense that you have to try some of the bits documented in the datasheet, see if that actually works, if not then see what it does if possible, look at other source code and see from that if you can figure it out. Just like there is no perfect car that gets infinite miles per gallon, completely safe, lasts forever, and is inexpensive, there is no perfect chip with the perfect datasheet and sample code. If you want to work at this software/hardware level you have to get your hands dirty, have to not be afraid to let some smoke out of the chips (there is a finite amount of smoke in a chip if you let even a little bit out it wont work), etc.
If the reason you wont ask specifically about the mcu or register you are working with is because it is closed source products or behind an NDA then you probably have access to the company that makes that product and you should be able to get support from them. Usually better support than you would get from a company that you dont have to sign an NDA for. Not that open document, open source companies are bad, just that if the company you buy from is interested in you to the point of showing internally protected information they are interested enough to give you better access to the real engineers that made/know the product. If this is not the case and you are able to talk about it, dont be afraid to just post a question to SO about the register and bits you are wondering about.
Sample code and flow charts in the MCU datasheets are good starting point to initialize a specific peripheral (like RS232).
You just start from there, and track the bit information and what it does, in MCU datasheet.

How can you program if you're blind?

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Sight is one of the senses most programmers take for granted. Most programmers would spend hours looking at a computer monitor (especially during times when they are in the zone), but I know there are blind programmers (such as T.V. Raman who currently works for Google).
If you were a blind person (or slowly becoming blind), how would you set up your development environment to assist you in programming?
(One suggestion per answer please. The purpose of this question is to bring the good ideas to the top. In addition, screen readers can read the good ideas earlier.)
I am a totally blind college student who’s had several programming internships so my answer will be based off these. I use windows xp as my operating system and Jaws to read what appears on the screen to me in synthetic speech. For java programming I use eclipse, since it’s a fully featured IDE that is accessible.
In my experience as a general rule java programs that use SWT as the GUI toolkit are more accessible then programs that use Swing which is why I stay away from netbeans. For any .net programming I use visual studio 2005 since it was the standard version used at my internship and is very accessible using Jaws and a set of scripts that were developed to make things such as the form designer more accessible.
For C and C++ programming I use cygwin with gcc as my compiler and emacs or vim as my editor depending on what I need to do. A lot of my internship involved programming for Z/OS. I used an rlogin session through Cygwin to access the USS subsystem on the mainframe and C3270 as my 3270 emulator to access the ISPF portion of the mainframe.
I usually rely on synthetic speech but do have a Braille display. I find I usually work faster with speech but use the Braille display in situations where punctuation matters and gets complicated. Examples of this are if statements with lots of nested parenthesis’s and JCL where punctuation is incredibly important.
Update
I'm playing with Emacspeak under cygwin http://emacspeak.sourceforge.net I'm not sure if this will be usable as a programming editor since it appears to be somewhat unresponsive but I haven't looked at any of the configuration options yet.
I'm blind, and have been programming for about 13 years on Windows, Mac, Linux and DOS, in languages from C/C++, Python, Java, C# and various smaller languages along the way. Though the original question was around configuring the environment, I think it's best answered by looking at how a blind person would use a computer.
Some people use a talking environment, such as T. V. Raman and the Emacspeak environment mentioned in other answers. The more common solution by far is to have a screen reader which runs in the background monitoring OS activity and alerting the user via synthetic speech or a physical braille display (generally showing somewhere from 20 to 80 characters at a time). This then means a blind person can use any accessible application.
So, I personally use Visual Studio 2008 these days, and run it with very few modifications. I turn off certain features like displaying errors as I type since I find this distracting. Prior to joining Microsoft all my development was done in a standard text editor like Notepad, so once again no customisations.
It is possible to configure a screen reader to announce indentation. I personally don't use this, since Visual Studio takes care of this, and C# uses braces. But this would be very important in a language like Python where whitespace matters. Finally, Emacspeak does make use of different voices/pitches to indicate different parts of syntax (keywords, comments, identifiers, etc).
I am blind and have been a programmer for the last 12 years or so. Currently am a senior architect and work with Sapient Corporation (a cambridge-based consulting company creating both Web-based and thick client based enterprise solutions).
I use several screen readers but mostly stick with Jaws for windows and NVDA.
I have mostly worked on the Microsoft platform and visual studio as my environment. I also use tools like the MS Sql enterprise studio and others for DB access, network monitoring etc.
I tried to spend some time with emacspeak but since my work was mostly based on the MS platform, never really spent a lot of time there.
I have also spent a couple of years working on C++ on linux - mostly used notepad or visual studio on windows for all the coding and then samba to share files with the linux environment.
Also used borland C for some experimental stuff. Have recently been playing around with python, which as other people have noted above is particularly unfriendly for a blind user because it is written using indentation as the nesting mechanism. Having said that, NVDA, the most popular open source screen reader is written completely using python and some of the commiters on that project are themself blind.
A particularly interesting question I get frequently asked as an architect is how do I deal with diagrams - UML and visio and rational rose etc. Visio is probably the most accessible diagraming tool out there. I was able to write jaws scripts to read rational rose diagrams for me. I've used a tool called T-dub (technical diagram understanding for the blind) developed by some german university for accessing UML 2.0 diagrams. Have used a java-based ugly tool called magic draw for doing model-driven development and was a commiter on the androMDA project and helped develop the .Net code generator from a UML model.
In general, I find that I thrive most in a team environment where I can work on my strengths. For example, while a diagram is extremely useful to communicate/document a design, the actual design process involves a lot of thinking and brainstorming and when the design has been thought out, one of your team mates can help you quickly put together a neatly drawn picture out of it.
People incorrectly mis-construe the above to be lack of independence or ability while I see this as pure inter-dependence -- as in I am sure that the team mate alone could never have come up with that design on his/her own and in-turn, if I depend on him to document the design, so be it.
Most hurdles I face are tool-based inaccessibility. For example all oracle products have been progressively declining in accessibility over the years (shame on them) and a team environment basically allows me an extra layer of defense against these over and above my screen readers and custom scripts.
I am a blind developer and I work under Windows, GNU Linux and MacOS X. Each of platform has different workflows for blind users. This depends on the screen reader that the blind developer uses.
Development tools are not completely accessible for blind developers. I can type code and use compiling functions in all IDEs but there are many problems if I have to design an interface using designing tools as Interface Builder, XGlade or other. When I was developing with Borland Delphi I could add a control, a Button for example, and I could modify each visual attribute of the control using object inspector window. Many IDEs use object inspector windows to modify visual and non visual attributes but the problem for a blind developer is add new controls because the method to add a new control consists of dragging and dropping a control from the palette to the canvas. Visual studio 200x uses alternative methods to do this but the interface of the IDE changes in each new version and this is a big problem because screen readers for Windows need special support, using scripts, to identify each area of some non standar applications. A blind developer can use Visual studio 2008 with his screen reader but when a new version of this IDE appears he has to wait for a new version of scripts for this version of the IDE.
Xcode with Interface builder has no alternative for dragging and dropping tasks yet. I asked it to Apple many times but they are working in other things. I published 3 apps in the App store (Accessible minesweeper, accessible fruitmachine and Programar a ciegas RSS) and I had to design all the interface by code. It's a hard work but I can manage all features of each control.
Eclipse has an accessible code editor but other development tools as debug console,plugins for designing or documentation area present problems for assistive tools for blind users.
Documentations is a problem for blind developers too. Many samples and demonstrations use images to show the explanation (set the environment settings as you can in the picture)
I think the question is not being blind. The question is the companies and development groups think accessibility affects final software but it doesn't affect development software. They think a blind user should be a client but a blind user can't be a development mate.
Blind associations ask accessibility for products and services but they forgot blind developers. Blind people can work as lawyers, journalists, teachers but a blind developer is a strange concept even for the blind. Many times I feel alone because some blind friends of mine can't understand my work.
You can read my opinion about this issue in this article, in Spanish, in my blog http://www.programaraciegas.net/2010/11/05/la-accesibilidad-en-crisis-para-los-desarrolladores-ciegos/
there is a translation tool in the web page. Sorry but I didn't translate it.
Emacs has a number of extensions to allow blind users to manipulate text files. You'd have to consult an expert on the topic, but emacs has text-to-speech capabilities. And probably more.
In addition, there's BLinux:
http://leb.net/blinux/
Linux for the blind. Been around for a very long time. More than ten years I think, and very mature.
Keep in mind that "blind" is a range of conditions - there are some who are legally blind that could read a really large monitor or with magnification help, and then there are those who have no vision at all. I remember a classmate in college who had a special device to magnify books, and special software she could use to magnify a part of the screen. She was working hard to finish college, because her eyesight was getting worse and was going to go away completely.
Programming also has a spectrum of needs - some people are good at cranking out lots and lots of code, and some people are better at looking at the big picture and architecture. I would imagine that given the difficulty imposed by the screen interface, blindness may enhance your ability to get the big picture...
Hanselman had a really interesting podcast with a blind developer recently.
I worked for the Greater Detroit Society for the Blind for three years running a BBS tailored for blind access and worked with a number of blind users on how to better meet their needs, and with newly blind users to get them acclimated to the available hardware and software offerings that were available at the time. If nothing else, I at least learned to read Braille as a hedge against the case where I ever wound up in the same situation!
The majority of blind computer users and programmers use a screen reader of some sort. Jaws in particular is popular. Fortunately, most major applications these days offer some form of handicapped access. You may have to tune your environment slightly to cut down on the chatter, e.g. consider disabling Intellisense in Visual Studio.
A Braille display is less common and is comparatively much more expensive and can show 40 or 80 columns of text, and can be used when exact positioning/punctuation is important. While a screen reader can be configured to rattle off punctuation, a lot of people find it distracting, and it is easier in many cases to feel your way through it. Jaws can be configured to drive the display, so you're not juggling accessibility applications.
Also, a lot of legally blind users still have some modicum of sight left to them. Using high contrast backgrounds and the magnification functionality can help a lot of these users.
Using ToggleKeys in Windows will let you hear when you accidentally tap one of the modal 'caps lock', 'num lock', 'scroll lock', etc. keys as well.
I know at least one Haskell programmer who uses a screen reader and who explicitly programs without using Haskell's layout rules, and instead opts to use the rather non-idiomatic, but supported {;}'s instead, because it is easier/less distracting for him to get his screen reader to read off punctuation than for him to figure out exact indentation that complies with Haskell's layout rules. On that same note, I've heard some grumbling from a couple of blind programmers about when they have to write Python.
Ultimately, you learn to play on your strengths.
I can't recall the source, but I've heard/read about a form of audible syntax "colouring" - so that instead of a string assignment being read as
foo equals quote this is a string quote
the string part would be read with a different pitch or voice to make the separation of elements clearer.
One place to start is the Blinux project:
http://leb.net/blinux/
That project describes how to get Emacspeak (editor with text-to-speech) and has a lot of other resources.
I worked with one person who's eye sight all but prevented them from using a monitor - they did well with Screen reader software and spent a lot of time using text based applications and the shell.
Wikipedia's list of screen reader packages is another place to start: http://en.wikipedia.org/wiki/List_of_screen_readers
I'm a postgraduate student in Beijing,China. I major in computer science and a lot of my work is programming.
I am born with low sight, I need to use magnifying tools to see fonts on screen clearly. I use microsoft's mgnify tools on windows and use compiz's magnify plug in if on linux. I usally set the tool to magnify as three times many as the original font size.
For me maginify tools is ok, the main problem is the speed,I have to move mouse to keep cursors follow the text I'm looking at, microsoft's magnify provides a option of "auto follow the text edit points",that set me from continuously mouse movement when editting or coding. But it doesn't always works because of the edit software or IDE may not support that.
Magnifying tools on linux are hard to use. The KMag come with KDE has a terrible refresh rate which make my eyes unconfortable, compiz's magnifying plugs which I'm using now is OK,but has no function of auto focus(focus auto following).
iOS provides quite perfect solution for me with full screen magnifying, especially on ipad's 9.7 inches screen. there auto focus is not necessary because I hardly use them to code or do other edit stuff.
Android provides very little accessibility functions, only like shake feedback, which is useless for me.
there is no any kind of good magnifying tools on android , not to mention advance function like full screen magnify on iOS.
I used to study Qt, want to build a useful magnify tools on linux, even on android. But hardly have some progress.
When I was in grad school, we had a member of our research team who was blind. He was a bit older, maybe mid-40s. He told us about how he programmed his first computer (which was well before text-to-speech was common) to output the contents of the screen in Morse Code. To overcome the obvious chicken-and-egg problem, he had to completely rewrite the code each time through from scratch until it was working well enough for him to have it read back to him.
Now he uses text-to-speech, though he plans the code very thoroughly before actually writing any of it, to minimize the debug loop.
He was also pretty good at giving PowerPoint presentations that, despite his lack of sight, were just about as well formatted as any sighted presenter's.
This blog post has some information about how the Visual Studio team is making their product accessible:
Visual Studio Core Team's Accessibility Lab Tour Activity
Many programmers use Emacspeak:
Emacspeak --The Complete Audio Desktop
Back in New Zealand I knew someone who had macular degeneration, so was partially sighted. He's a very talented programmer and wound up using Delphi because he could work by recognizing word shapes This was easier to do with a Pascal-like syntax than a C-ish squiggly bracket one. He has a web site, but doesn't seem to mention macular degeneration at all, so I won't name him.
I'm blind and from some months I'm using VINUX (a linux distro based on Ubuntu) with SODBEANS (a version of netbeans with a plug-in named SAPPY that add a TTS support).
This solution works quite well but sometimes I prefer to launch Win XP and NVDA for launching many pages on FireFox because Vinux doesn't work very well when you try to open more than 3 windows of FireFox...
As many have pointed out, emacspeak has been the enduring solution cross platform for many of the older hackers out there. Since it supports Linux and Mac out of the box, it has become my prefered means of developing Windows egnostic projects.
To the issue of actually getting down syntax through an auditory one as opposed to a visual one, I have found that there exists a variety of techniques to get one close if not on the same playing field.
Auditory icons can stand in place for verbal descriptors for one example. You can, put tones for how far a line is indented. The longer the tone, the further the indent. Since tones can play in parallel with text to speech, the information comes through in the same timeframe and doesn't serialize the communication of something so basic.
Braille can quickly and precisely decode to the user the exact syntax of a line. This is something more useful for people who use braille in daily life; the biggest advantage is random access to the contents of the display. Refreshable units typically have router keys above each character cell which can place the cursor to that cell. No fiddling with arrow keys O(n) op vs O(1) access.
Auditory dimensionality (pitch, rate, volume, inflection, richness, stress, etc) can convey a concept (keyword, class, variable, error, etc). For example, comments can be read in a monotone inflection...suiting, if I might say so :).
Emacs and other editors to lesser extents (Visual Studio) allow a coder to peruse a program symantically (next block, fold block, down defun, jump to def, walk up the parse tree, etc). You can very quickly get the "big" picture of the structure of an entire project doing this; with extensions like Cedet, you can get the goodness of VS/Eclipse/etc cross platform and in a textual editor.
Could probably go on and on, but that in a nutshell, is the basis of why a few of us are out there hacking away in industry, adacdemia, or in our basements :).
A group of students from Southern Illinois University Edwardsville and Washington State University are working on a programming language for the blind:
http://www.youtube.com/watch?v=lC1mOSdmzFc
harald van Breederode is a well-known Dutch Oracle DBA expert, trainer and presenter who is blind. His blog contains some useful tips for visually impaired people.
What in the world would a braille keyboard even be??
There are such things as braille writers but you would never use one as an input device for a computer.
If you're simply talking about a keyboard with the braille symbols on it this would also be a very bad idea. You're going to have a lot more keys to reach while typing and it would still be slower.
Touch typing is NOT a visual skill, a blind person can do it just as well as a sighted person.
I think that this would work well in extreme programming using the pair programming principle. If you're making software for blind people, who better to make it then someone who would literally be in touch with the business requirements, so I don't think it's very far fetched at all.
As for writing code, well unless there was some kind of feedback I think a person may struggle with syntax. Audio feedback may help to a point though.
NVDA is a good open source screen reader for win.
What about inventing some kind of device that you plug in a usb port and that would be basically a "sheet of rubber" that would modify itself to show brail of your code, allowing blind people to read it instead to hear it?
There are a variety of tools to aid blind people or partially sighted including speech feedback and braillie keyboards. http://www.rnib.org.uk/Pages/Home.aspx is a good site for help and advice over these issues.
Once I met Sam Hartman, he is a famous Debian developer since 2000, and blind. On this interview he talks about accessibility for a Linux user. He uses Debian, and gnome-orca as screen reader, it works with Gnome, and "does a relatively good job of speaking Iceweasel/Firefox and Libreoffice".
Specifically speaking about programming he says:
While [gnome-orca] does speak gnome-terminal, it’s not really good enough at
speaking terminal programs that I am comfortable using it. So, I run
Emacs with the Emacspeak package. Within that, I run the Emacs
terminal emulator, and within that, I tend to run Screen. For added
fun, I often run additional instances of Emacs within the inner
screens.

Developing a online exam application, how do I prevent cheaters?

I have the task of developing an online examination software for a small university, I need to implement measures to prevent cheating...
What are your ideas on how to do this?
I would like to possibly disable all IE / firefox tabs, or some how log internet activity so I know if they are googling anwsers...is there any realistic way to do such things from a flex / web application?
Simply put, no there is no realistic way to accomplish this if it is an online exam (assuming they are using their own computers to take the exam).
Is opening the browser window full screen an option? You could possibly also check for the window losing focus and start a timer that stops the test after some small period of time.
#Chuck - a good idea.
If the test was created in Flash/Flex, you could force the user to make the application fullscreen in order to start the test (fullscreen mode has to be user-initiated). Then, you can listen for the Flash events dispatched when flash exits fullscreen mode and take whatever appropriate action you want (end the test, penalize the user, etc.).
Flash/Flex fullscreen event info.
blog.flexexamples.com has an example fo creating a fullscreen-capable app.
Random questions and large banks of questions help. Randomizing even the same question (say changing the numbers, and calculating the result) helps too. None of these will prevent cheating though.
In the first case, if the pool is large enough, so that no two students get the same question, all that means is that students will compile a list of questions over the course of several semesters. (It is also a ton of work for the professors to come up with so many questions, I've had to do it as a TA it is not fun.)
In the second case, all you need is one smart student to solve the general case, and all the rest just take that answer and plug in the values.
Online review systems work well with either of these strategies (no benefit in cheating.) Online tests? They won't work.
Finally, as for preventing googling... good luck. Even if your application could completely lock down the machine. The user could always run a VM or a second machine and do whatever they want.
My school has always had a download link for the Lockdown browser, but I've never taken a course that required it. You can probably force the student to use it with a user agent check, but it could probably be spoofed with some effort.
Proctored tests are the only way to prevent someone from cheating. All the other methods might make it hard enough to not be worth the effort for most, but don't discount the fact that certain types of people will work twice as hard to cheat than it would have taken them to study honestly.
Since you can't block them from using google, you've got to make sure they don't have time to google. Put the questions in images so they can't copy and paste (randomize the image names each time they are displayed).
Make the question longer (100 words or more) and you will find that people would rather answer the question than retype the whole thing in google.
Give them a very short time. like 30-45 seconds. Time to read the question, think for a moment, and click either A, B, C, D, E,
(having just graduated from CSUN I can tell you scantron tests work.)
For essay questions? do a reverse google lookup (meaning put their answer into google as soon as they click submit) and see if you get exact matches. If so, you know what to do.
Will they always take the test on test machines, or will they be able to take the test from any machine on the network? If it will be specific machines, just use the hosts file to prevent them from getting out to the web.
If it is any machine, then I would look at having the testing backend change the firewall rules for the machine the test is running on so the machine cannot get out to the interwebs.
I'd probably implement a simple winforms (or WPF) app that hosts a browser control in it -- which is locked in to your site. Then you can remove links to browsers and lock down the workstations so that all they can open is your app.
This assumes you have control over the workstations on which the students are taking the tests, of course.
As a teacher, I can tell you the single best way would be to have human review of the answers. A person can sense copy/paste or an answer that doesn't make sense given the context of the course, expected knowledge level of the students, content of the textbook, etc, etc, etc.
A computer can do things like check for statistical similarity of answers, but you really need a person for final review (or, alternatively, build a massive statistical-processing, AI stack that will cost 10x the cost of human review and won't be as good ;-))
No, browsers are designed to limit the amount of damage a website or application can do to the system. You might be able to accomplish your goals through Java, an activex control, or a custom plugin, but other than that you aren't going to be able to 'watch' what they're doing on their system, much less control it. (Think if you could! I could put a spy on this webpage, and if you have it open I get to see what other websites you have open?)
Even if you could do this, using a browser inside a VM would give them the ability to use one computer to browse during the test, and if you could fix that they could simply use a library computer with their laptop next to it, or read things from a book.
The reality is that such unmonitored tests either have to be considered "open book" or "honor" tests. You must design the test questions in such a manner that references won't help solve the problems, which also means that each student needs to get a slightly different test so there is no way for them to collude and generate a key.
You have to develop an application that runs on their computer, but even then you can't solve the VM problem easily, and cannot solve the side by side computers or book problem at all.
-Adam
Randomize questions, ask a random set of questions from a large bank... time the answers...
Unless you mean hacking your site, which is a different question.
Short of having the application run completely on the user's machine, I do not believe there is a way to make sure they are not google-ing the answers. Even then it would be difficult to check for all possible loop-holes.
I have taken classes that used web based quiz software and used to work for a small college as well. For basic cheating prevention I would say randomize the questions.
Try adding SMS messages into the mix.
I agree with Adam, that even with the limitations that I suggested, it would still be trivial to cheat. Those were just "best effort" suggestions.
Your only hopes are a strong school honor code and human proctoring of the room where the test is being given.
As many other posters have said, you can't control the student's computer, and you certainly can't keep them from using a second computer or an iPhone along side the one being used for the test -- note that an iPhone (or other cellular device) can bypass any DNS or firewall on the network, since it uses the cellular provider's network, not the college's.
Good luck; you're going to need it.
Ban them from using any wireless device or laptop and keylog the machines?
You could enforce a small time window during which the test is available. This could reduce the chance that a student who knows the answers will be free to help one who doesn't (since they both need to be taking the test at the same time).
If it's math-related, use different numbers for different students. In general, try to have different questions for different copies of the test.
If you get to design the entire course: try to have some online homeworks as well, so that you can build a profile for each student, such as a statistical analysis of how often they use certain common words and punctuations. Some students use semi-colons often; others never, for example. When they take the test you get a good idea of whether or not it's really them typing.
You could also ask a couple questions you know they don't know. For example, list 10 questions and say they must answer any 6 out of the 10. But make 3 of the questions based on materials not taught in class. If they choose 2 or 3 of these, you have good reason to be suspicious.
Finally, use an algorithm to compare for similar answers. Do a simple hash to get rid of small changes. For example, hash an answer to a list of lower-cased 3-grams (3 words in a row), alphabetize it, and then look for many collisions between different users. This may sound like an obvious technique, but as a teacher I can assure you this will catch a surprising number of cheaters.
Sadly, the real trouble is to actually enforce punishment against cheaters. At the colleges where I have taught, if a student objects to your punishment (such as flunking them on the test in question), the administration will usually give the student something back, such as a positive grade change. I guess this is because the student('s parents) have paid the university a lot of money, but it is still very frustrating as a teacher.
The full screen suggestions are quite limited in their effectiveness as the user can always use a second computer or w/ multi monitor a second screen to perform their lookups. In the end it is probably better to just assume the students are going to cheat and then not count online tests for anything important.
If the tests are helpful for the students they will then do better on the final / mid term exams that are proctored in a controlled setting. Otherwise, why have them in the first place...
Make the questions and answers jpeg images so that you cannot copy and paste blocks of text into a search engine or IDE (if it is a coding test). This combined with a tight time limit to answer each question, say three minutes, makes it much harder to cheat.
I second what Guy said. We also created a Flex based examination system which was hosted in a custom browser built in .NET. The custom browser launched fullscreen, all toolbars were hidden and shortcuts were disabled.
Here is tutorial on how to create a custom browser with C# and VB.NET.
This will solve your problem. http://www.neuber.com/usermonitor/index.html
This will allow you to view the student's browser history during and after the test as well as look in on their screen during the test. Any urls visited during test time will be logged, so you can show them the log when you put a big F on their report card. :)
No one can stop people from cheating, but everyone can receive different questions altogether.
I prefer you buy available online scripts in market as starting point for it. This will save you time, cost and testing efforts.
Below is one of the fine scripts that I worked with and it worked like charm. Using this as base I developed a online testing portal of over 1000 users using computer adaptive test.
http://codecanyon.net/item/online-skills-assessment/9379895
It is a good starting point for people looking to develop Online Exam System.
I customized the script with the help of their support.

Resources