Developing a online exam application, how do I prevent cheaters? - apache-flex

I have the task of developing an online examination software for a small university, I need to implement measures to prevent cheating...
What are your ideas on how to do this?
I would like to possibly disable all IE / firefox tabs, or some how log internet activity so I know if they are googling anwsers...is there any realistic way to do such things from a flex / web application?

Simply put, no there is no realistic way to accomplish this if it is an online exam (assuming they are using their own computers to take the exam).

Is opening the browser window full screen an option? You could possibly also check for the window losing focus and start a timer that stops the test after some small period of time.

#Chuck - a good idea.
If the test was created in Flash/Flex, you could force the user to make the application fullscreen in order to start the test (fullscreen mode has to be user-initiated). Then, you can listen for the Flash events dispatched when flash exits fullscreen mode and take whatever appropriate action you want (end the test, penalize the user, etc.).
Flash/Flex fullscreen event info.
blog.flexexamples.com has an example fo creating a fullscreen-capable app.

Random questions and large banks of questions help. Randomizing even the same question (say changing the numbers, and calculating the result) helps too. None of these will prevent cheating though.
In the first case, if the pool is large enough, so that no two students get the same question, all that means is that students will compile a list of questions over the course of several semesters. (It is also a ton of work for the professors to come up with so many questions, I've had to do it as a TA it is not fun.)
In the second case, all you need is one smart student to solve the general case, and all the rest just take that answer and plug in the values.
Online review systems work well with either of these strategies (no benefit in cheating.) Online tests? They won't work.
Finally, as for preventing googling... good luck. Even if your application could completely lock down the machine. The user could always run a VM or a second machine and do whatever they want.

My school has always had a download link for the Lockdown browser, but I've never taken a course that required it. You can probably force the student to use it with a user agent check, but it could probably be spoofed with some effort.
Proctored tests are the only way to prevent someone from cheating. All the other methods might make it hard enough to not be worth the effort for most, but don't discount the fact that certain types of people will work twice as hard to cheat than it would have taken them to study honestly.

Since you can't block them from using google, you've got to make sure they don't have time to google. Put the questions in images so they can't copy and paste (randomize the image names each time they are displayed).
Make the question longer (100 words or more) and you will find that people would rather answer the question than retype the whole thing in google.
Give them a very short time. like 30-45 seconds. Time to read the question, think for a moment, and click either A, B, C, D, E,
(having just graduated from CSUN I can tell you scantron tests work.)
For essay questions? do a reverse google lookup (meaning put their answer into google as soon as they click submit) and see if you get exact matches. If so, you know what to do.

Will they always take the test on test machines, or will they be able to take the test from any machine on the network? If it will be specific machines, just use the hosts file to prevent them from getting out to the web.
If it is any machine, then I would look at having the testing backend change the firewall rules for the machine the test is running on so the machine cannot get out to the interwebs.

I'd probably implement a simple winforms (or WPF) app that hosts a browser control in it -- which is locked in to your site. Then you can remove links to browsers and lock down the workstations so that all they can open is your app.
This assumes you have control over the workstations on which the students are taking the tests, of course.

As a teacher, I can tell you the single best way would be to have human review of the answers. A person can sense copy/paste or an answer that doesn't make sense given the context of the course, expected knowledge level of the students, content of the textbook, etc, etc, etc.
A computer can do things like check for statistical similarity of answers, but you really need a person for final review (or, alternatively, build a massive statistical-processing, AI stack that will cost 10x the cost of human review and won't be as good ;-))

No, browsers are designed to limit the amount of damage a website or application can do to the system. You might be able to accomplish your goals through Java, an activex control, or a custom plugin, but other than that you aren't going to be able to 'watch' what they're doing on their system, much less control it. (Think if you could! I could put a spy on this webpage, and if you have it open I get to see what other websites you have open?)
Even if you could do this, using a browser inside a VM would give them the ability to use one computer to browse during the test, and if you could fix that they could simply use a library computer with their laptop next to it, or read things from a book.
The reality is that such unmonitored tests either have to be considered "open book" or "honor" tests. You must design the test questions in such a manner that references won't help solve the problems, which also means that each student needs to get a slightly different test so there is no way for them to collude and generate a key.
You have to develop an application that runs on their computer, but even then you can't solve the VM problem easily, and cannot solve the side by side computers or book problem at all.
-Adam

Randomize questions, ask a random set of questions from a large bank... time the answers...
Unless you mean hacking your site, which is a different question.

Short of having the application run completely on the user's machine, I do not believe there is a way to make sure they are not google-ing the answers. Even then it would be difficult to check for all possible loop-holes.
I have taken classes that used web based quiz software and used to work for a small college as well. For basic cheating prevention I would say randomize the questions.

Try adding SMS messages into the mix.

I agree with Adam, that even with the limitations that I suggested, it would still be trivial to cheat. Those were just "best effort" suggestions.

Your only hopes are a strong school honor code and human proctoring of the room where the test is being given.
As many other posters have said, you can't control the student's computer, and you certainly can't keep them from using a second computer or an iPhone along side the one being used for the test -- note that an iPhone (or other cellular device) can bypass any DNS or firewall on the network, since it uses the cellular provider's network, not the college's.
Good luck; you're going to need it.

Ban them from using any wireless device or laptop and keylog the machines?

You could enforce a small time window during which the test is available. This could reduce the chance that a student who knows the answers will be free to help one who doesn't (since they both need to be taking the test at the same time).
If it's math-related, use different numbers for different students. In general, try to have different questions for different copies of the test.
If you get to design the entire course: try to have some online homeworks as well, so that you can build a profile for each student, such as a statistical analysis of how often they use certain common words and punctuations. Some students use semi-colons often; others never, for example. When they take the test you get a good idea of whether or not it's really them typing.
You could also ask a couple questions you know they don't know. For example, list 10 questions and say they must answer any 6 out of the 10. But make 3 of the questions based on materials not taught in class. If they choose 2 or 3 of these, you have good reason to be suspicious.
Finally, use an algorithm to compare for similar answers. Do a simple hash to get rid of small changes. For example, hash an answer to a list of lower-cased 3-grams (3 words in a row), alphabetize it, and then look for many collisions between different users. This may sound like an obvious technique, but as a teacher I can assure you this will catch a surprising number of cheaters.
Sadly, the real trouble is to actually enforce punishment against cheaters. At the colleges where I have taught, if a student objects to your punishment (such as flunking them on the test in question), the administration will usually give the student something back, such as a positive grade change. I guess this is because the student('s parents) have paid the university a lot of money, but it is still very frustrating as a teacher.

The full screen suggestions are quite limited in their effectiveness as the user can always use a second computer or w/ multi monitor a second screen to perform their lookups. In the end it is probably better to just assume the students are going to cheat and then not count online tests for anything important.
If the tests are helpful for the students they will then do better on the final / mid term exams that are proctored in a controlled setting. Otherwise, why have them in the first place...

Make the questions and answers jpeg images so that you cannot copy and paste blocks of text into a search engine or IDE (if it is a coding test). This combined with a tight time limit to answer each question, say three minutes, makes it much harder to cheat.

I second what Guy said. We also created a Flex based examination system which was hosted in a custom browser built in .NET. The custom browser launched fullscreen, all toolbars were hidden and shortcuts were disabled.
Here is tutorial on how to create a custom browser with C# and VB.NET.

This will solve your problem. http://www.neuber.com/usermonitor/index.html
This will allow you to view the student's browser history during and after the test as well as look in on their screen during the test. Any urls visited during test time will be logged, so you can show them the log when you put a big F on their report card. :)

No one can stop people from cheating, but everyone can receive different questions altogether.
I prefer you buy available online scripts in market as starting point for it. This will save you time, cost and testing efforts.
Below is one of the fine scripts that I worked with and it worked like charm. Using this as base I developed a online testing portal of over 1000 users using computer adaptive test.
http://codecanyon.net/item/online-skills-assessment/9379895
It is a good starting point for people looking to develop Online Exam System.
I customized the script with the help of their support.

Related

What does a developer need on the front end to ensure a successful project?

I have an idea for a business that requires a well designed web application. I'm not a rocket surgeon, but I'm smart enough to know that you get what you pay for and am willing to pay for talent. However, I want the development process to go as smoothly as possible and would like to know how to make that happen.
So, what information do developers need (or want) initially from the owner to avoid having to make assumptions about business (or other) requirements? Do I need to create state transition diagrams or write use cases?
Essentially, how do I take the concept in my head and package it in a way that allows the developer to do what they do best? (assuming that is creating good software. haha)
Any advice is appreciated.
Shawn
You may need to reword your question, as it is too general to get a good answer, so some vague details would be helpful.
But, the better vision you have of what you want the smoother it will be.
I find UML diagrams too confining, when you aren't going to be doing the work, as you may not come up with the best design.
So, if you start with designing out what each page should look like, as you envision it, then you can write up use cases, which are short scenarios.
So, you may write up:
A user needs to be able to log in using OpenID.
This will tell the developer one function that you want, and who you expect to do that action.
But, don't put in technologies, as you may think that a SOAP service is your best bet, but upon talking about it you may find that there is a better solution.
Use cases are good points to show what you are envisioning, and give text to your page designs.
Talk to the developers. Explain what you want and why you want it. Together you make the flow charts and whatnot. Writing requirements is part of the design process, and it's a good idea to have the developers onboard as soon as possible. Start simple and small, then grow and expand while iterating.
In talking over web services before, I have found the best starting point is drawing on a sheet of paper what you think the site will look like, and add in a few arrows from things you want clickable to the pages that should result. Keep it simple, nothing too fancy, and hopefully you and the developer can come to an understanding of what you want pretty quickly.
Use cases might be best for checking off all the points later in the project about how complete your site is; I haven't really found it to be a helpful starting point, but I'm sure others disagree. (They just seem too tedius to read when actually writing code.)
Same with state transition diagrams; they are too tedious and I think most developers will assume you made mistakes in them anyway. :) Everyone else does... Unless your project hinges very tightly on the correctness of a state machine, I wouldn't really bother.
This book contains some good advice on what constitutes a good statement of requirements from a programmers point of view. It also has the useful guideline of not trying to set the form of your requirements too early, and a substantial piece on describing the problem you are trying to solve.
I like UI mockups based on actual program/site flows e.g registering a customer or placing order. Diagrams/pictures of GUIs with structured, consistent data examples are unambiguous.
I agree that UML and use cases are only really useful if everyone speaks UML and the projects are of sufficient complexity (few are).
You may want to read up on Agile/Scrum techniques. These are becoming a sort of standard and when properly managed can save weeks of development time.
I find that words don't do a good job of communicating how a system is supposed to work. Wireframes, white-board drawings/transition diagrams, and low-fidelity prototypes are great ways to communicate a concrete idea. One example of a low-fidelity prototype is a "clickable" paper prototype that allows a user to touch "buttons" on paper to go from one drawing to another. It costs very little time (cheaper), but goes a long way to communicate an idea between two parties.
Stay away from formal documentation, UML diagrams, or class (technical documentation) diagrams that don't speak to you. This is what large, risk-averse companies move toward to be more "mature". These are also byproducts of an idea that is hashed out, and it sounds like you're in the hashing out stage.

R Quality Assurance Techniques

Could you provide some insight into the techniques that you use to ensure the quality of your solutions. For example, sometimes, I like to test my result using stopifnot() to ensure I'm not receiving ridiculous results. Are there any other techniques or functions that you use in data processing to ensure that you're receiving the solution you meant to?
Note: I realize that this is a broad question and perhaps a candidate for community wiki or even closure, but rather than voting to close, perhaps assist me by adding comments to direct the conversation.
Just a few things that come to mind (in random order)
This page has very interesting link for debugging in R (ok this is during production, but still related to your issue I think)
You can use exceptions, as explained in this discussions (and links therein)
You can write tests with known results (both for success and failure) and see that they actually do what they are supposed to do. Be sure to pass some weird data to the functions and see how they behave in a "not-so-normal" situation.
Don't just rely on automated tests: give your functions to a fairly computer illiterate person at work (not enough that he/she can't use R though!) and let him/her do some beta tests. You'll be amazed at the quantity of errors he/she will come up with!!! :)
Quality in software engineering is quite a massive area, and most of it applies to code written in R as much as code written in Cobol or C#, so my first answer would be 'it depends'.
For me, I come from the Pharmaceutical Industry, where what we do is regulated by government agencies like the FDA and the MHRA. For us, Quality is something we think about throughout the process so I would list the following as visible artifacts of quality;
We have a software development process, that's written down and repeatable (traditionally in this kind of industry this is a waterfall style, but more and more agile / prototyping style methodologies are being used)
We have a system that ensures every person involved knows what they should be doing (job descriptions) and is suitably qualified to do that job (training)
We start by defining what is required in some way, hopefully in some way that can be tested
We have some way of documenting our development process, where we've been and how (a combination of good documentation and Source Control)
We do testing wherever possible, and as early as possible (so, automated if possible)
We have people who are responsible for overseeing Quality, who are separate from people who are doing to prevent conflicts
We control the software environment that is used for development, testing and production (read; change control)
We control and manage software once it is in use, tracking issues and managing them (Issue Tracking)
We keep records, so that even if every person involved went under a bus / won the lottery the new people could still defend and prove everything above to a government inspector.
However, that's a big list, and I imagine their are lots of industries that don't do all of them (finance, education) and probably some who do more (building nuclear reactors, saving lives, NASA).
More specifically to what i assume you're getting at, before you code you should be able to define some specific starting input's and the answers you should get out, and I recommend you use something like RUnit or Testthat to build these in.

Windows standard for mnemonics

Are there any mnemonic standards for Windows?
For example -- the menu bar, actions on the menu bar (e.g. Alt+f to file menu but ctrl+s to do the save under the file menu), and controls.
I'm asking because we have a search screen with many controls and we're trying to decide shortcut keys to get to fields and such described above.
Is there anything official at Microsoft or some RFC document that we could read and present in a meeting to figure out how to properly handle this.
The only thing I've found so far is this -- which helps some, but doesn't go in to great detail.
Further, is there maybe a book I can get to help me with designing such things?
How does the community feel about double mnemonics? (e.g. ctrl a, w)
Thanks!
You want to look at the User Experience Interaction Guidelines.
Particularly, the Access Keys part.
Regarding double mnemonics:
I would consider these only if your users are going to have a high level of expertise. If they use your product day in day out and become very proficient with it, then double mnemonics may pay their way. But for many users they are likely to be hard to discover, remember and use. (I use Visual Studio most of every day, and I only remember and use a very few of the double mnemonics available -- just the ones for commands that I use very frequently.) In which case they would not be worth the effort required to implement them.
If you think your users may have this level of expertise I would still recommend simple usability testing, because this is not a common idiom; though "expert" level features are hard to evaluate in hallway-style usability tests. Nevertheless, as programmers we are notoriously unable to judge usability, so definitely test and measure somehow with real users before putting significant effort into this!

Automating paper forms and process flow in the office

I have been tasked with automating some of the paper forms in HR. This might turn into "automate all forms" eventually, so I want to approach this in a way which will be best for the long term and will be a good framework as this project grows.
The first things that come to mind were:
-InfoPath/SharePoint (We currently don't use SharePoint now, and wouldn't be an option for the next two years.)
-Workflow Foundation (I've looked into this and does not seem too attractive or appropriate)
Option I'm considering at this point:
-Custom ASP.NET (VB.NET) & SQL Server, which is what my team mostly writes their apps with.
-Leverage Infopath for creating the forms electronically. Wondering if there is a good approach to integrating this with a custom built ASP.NET app.
-Considering creating the app as an MVC web app.
My question is this:
-Are there other options I might want to consider?
-Are there any starter kits or VB.NET based open source projects there which would be a starting point or could be used as a good reference. Here I'm mostly concerned with the workflow processing.
-Any comnments from those who have gone down this path?
This is going to sound really dumb, but in my many years of helping companies automate paper form-based processes is to understand the process first. You will most likely find that no single person understands the whole thing. You will need to role-play the many paths thru the process to get your head around it. And once you present your findings, everyone will be shocked because they had no idea it was that complex. Use that as an opportunity to streamline.
Automating a broken process only makes it screw up faster and tell a lot of people.
As far as tools, my experience dates me but try to go with something with these properties:
EASY to change. You WILL be changing it. So don't hard-code anything.
Possible revision control - changes to a process may or may not affect documents already in route?
Visual workflow editing. Everyone wants this but they'll all ask you to drive it. Still, nice tools.
Not sure if this helps or not - but 80% of success in automating processes is not technology.
This is slightly off topic, but related - defect tracking systems generally have workflow engines/state. (In fact, I think Joel or some other FC employee posted something about using FB for managing the initial emails and resume process)
I second the other advice about modeling the workflow before doing any coding or technology choices. You will also want this to be flexible.
as n8owl reminded us, automating a mess yields an automated mess - which is not an improvement. Many paper-forms systems have evolved over decades and can be quite redundant and unruly. Some may view "messing with the forms" as a violation of their personal fiefdoms, so watch your back ;-)
model the workflow in terms of the forms used by whom in what roles for what purposes; this documents the current process as a baseline. Get estimates of how long each step takes, both in terms of man-hours and calendar time
understand the workflow in terms of the information gathered, generated, and transmitted
consolidate the information on the forms into a new set of forms for minimal workflow
be prepared to be told "This is the way we've always done it and we're not going to change", and to gently (a) validate their feelings, (b) explain how less work is more efficient, and (c) show concrete benefits [vs.the baseline from step 1]
soft-code when possible; use processing rules when possible; web services and html forms (esp. w/jquery) will go a long way if you have an intranet
beware of canned packages (including sharepoint) unless you are absolutely certain they encompass your organization's current and future needs
good luck!
--S
I detect here a general tone of caution with regards to a workflow based approach and must agree. Be advised about the caveats of most workflow technologies which sacrifice usability for flexibility.

Requirements Gathering

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
How do you go about the requirements gathering phase? Does anyone have a good set of guidelines or tips to follow? What are some good questions to ask the stakeholders?
I am currently working on a new project and there are a lot of unknowns. I am in the process of coming up with a list of questions to ask the stakeholders. However I cant help but to feel that I am missing something or forgetting to ask a critical question.
You're almost certainly missing something. A lot of things, probably. Don't worry, it's ok. Even if you remembered everything and covered all the bases stakeholders aren't going to be able to give you very good, clear requirements without any point of reference. The best way to do this sort of thing is to get what you can from them now, then take that and give them something to react to. It can be a paper prototype, a mockup, version 0.1 of the software, whatever. Then they can start telling you what they really want.
See obligatory comic below...
In general, I try and get a feel for the business model my customer/client is trying to emulate with the application they want built. Are we building a glorified forms processor? Are we retrieving data from multiple sources in a single application to save time? Are we performing some kind of integration?
Once the general businesss model is established, I then move to the "must" and "must nots" for the application to dictate what data I can retrieve, who can perform what functions, etc.
Usually if you can get the customer to explain their model or workflow, you can move from there and find additional key questions.
The one question I always make sure to ask in some form or another is "What is the trickiest/most annoying thing you have to do when doing X. Typically the answer to that reveals the craziest business/data rule you'll have to implement.
Hope this helps!
Steve Yegge talks fun but there is money to be made in working out what other people's requirements are so i'd take his article with a pinch of salt.
Requirements gathering is incredibly tough because of the manner in which communication works. Its a four step process that is lossy in each step.
I have an idea in my head
I transform this into words and pictures
You interpret the pictures and words
You paint an image in your own mind of what my original idea was like
And humans fail miserably at this with worrying frequency through their adorable imperfections.
Agile does right in promoting iterative development. Getting early versions out to the client is important in identifying what features are most important (what ships in 0.1 - 0.5 ish), helps to keep you both on the right track in terms of how the application will work and quickly identifies the hidden features that you will miss.
The two main problem scenarios are the two ends of the scales:
Not having a freaking clue about what you are doing - get some domain experts
Having too many requirements - feature pit. - Question, cull (prioritise ;) ) features and use iterative development
Yegge does well in pointing out that domain experts are essential to produce good requirements because they know the business and have worked in it. They can help identify the core desire of the client and will help explain how their staff will use the system and what is important to the staff.
Alternatives and additions include trying to do the job yourself to get into the mindset or having a client staff member occasionally on-site, although the latter is unlikely to happen.
The feature pit is the other side, mostly full of failed government IT projects. Too much, too soon, not enough thought or application of realism (but what do you expect they have only about four years to make themselves feel important?). The aim here is to work out what the customer really wants.
As long as you work on getting the core components correct, efficient and bug-free clients usually remain tolerant of missing features that arrive in later shipments, as long as they eventually arrive. This is where iterative development really helps.
Remember to separate the client's ideas of what the program will be like and what they want the program to achieve.
Some clients can create confusion by communicating their requirements in the form of application features which may be poorly thought out or made redundant by much simpler functionality then they think they require. While I'm not advocating calling the client an idiot or not listening to them I feel that it is worth forever asking why they want a particular feature to get to its underlying purpose.
Remember that in either scenario it is of imperative importantance to root out the quickest path to fulfilling the customers core need and put you in a scenario where you are both profiting from the relationship.
Wow, where to start?
First, there is a set of knowledge someone should have to do analysis on some projects, but it really depends on what you are building for who. In other words, it makes a big difference if you are modifying an enterprise application for a Fortune 100 corporation, building an iPhone app, or adding functionality to a personal webpage.
Second, there are different kinds of requirements.
Objectives: What does the user want to accomplish?
Functional: What does the user need to do in order to reach their objective? (think steps to reach the objective/s)
Non-functional: What are the constraints your program needs to perform within? (think 10 vs 10k simultaneous users, growth, back-up, etc.)
Business rules: What dynamic constraints do you have to meet? (think calculations, definitions, legal concerns, etc.)
Third, the way to gather requirements most effectively, and then get feedback on them (which you will do, right?) is to use models. User cases and user stories are a model of what the user needs to do. Process models are another version of what needs to happen. System diagrams are just another model of how different parts of the program(s) interact. Good data modeling will define business concepts and show you the inputs, outputs, and changes that happen within your program. Models (and there are more than I listed) are really the key to the concern you list. A few good models will capture the needs and from models you can determine your requirements.
Fourth, get feedback. I know I mentioned this already, but you will not get everything right the first time, so get responses to what your customer wants.
As much as I appreciate requirements, and the models that drive them, users typically do not understand the ramifications of of all their requests. Constant communication with chances for review and feedback will give users a better understanding of what you are delivering. Further, they will refine their understanding based on what they see. Unless you're working for the government, iterations and / or prototypes are helpful.
First of all gather the requirements before you start coding. You can begin the design while you are gathering them depending on your project life cicle but you shouldn't ever start coding without them.
Requirements are a set of well written documents that protect both the client and yourself. Never forget that. If no requirement is present then it was not paid for (and thus it requires a formal change request), if it's present then it must be implemented and must work correctly.
Requirements must be testable. If a requirement cannot be tested then it isn't a requirement. That means something like, "The system "
Requirements must be concrete. That means stating "The system user interface shall be easy to use" is not a correct requirment.
In order to actually "gather" the requirements you need to first make sure you understand the businness model. The client will tell you what they want with its own words, it is your job to understand it and interpret it in the right context.
Make meetings with the client while you're developing the requirements. Describe them to the client with your own words and make sure you and the client have the same concept in the requirements.
Requirements require concise, testable example, but keep track of every other thing that comes up in the meetings, diagrams, doubts and try to mantain a record of every meeting.
If you can use an incremental life cycle, that will give you the ability to improve some bad gathered requirements.
You can never ask too many or "stupid" questions. The more questions you ask, the more answers you receive.
According to Steve Yegge that's the wrong question to ask. If you're gathering requirement it's already too late, your project is doomed.
High-level discussions about purpose, scope, limitations of operating environment, size, etc
Audition a single paragraph description of the system, hammer it out
Mock up UI
Formalize known requirements
Now iterate between 3 and 4 with more and more functional prototypes and more specs with more details. Write tests as you go. Do this until you have functional software and a complete, objective, testable requirements spec.
That's the dream. The reality is usually after a couple iterations everybody goes head-down and codes until there's a month left to test.
Gathering Business Requirements Are Bullshit - Steve Yegge
read the agile manifesto - working software is the only measurement for the success of a software project
get familiar with agile software practices - study Scrum , lean programming , xp etc - this will save you tremendous amount of time not only for the requirements gathering but also for the entire software development lifecycle
keep regular discussions with Customers and especially the future users and key-users
make sure you talk to the Persons understanding the problem domain - e.g. specialists in the field
Take small notes during the talks
After each CONVERSATION write an official requirement list and present it for approving. Later on it would be difficult to argue against all agreed documentation
make sure your Customers know approximately what are the approximate expenses in time and money for implementing "nice to have" requirements
make sure you label the requirements as "must have" , "should have" and "nice to have" from the very beginning, ensure Customers understand the differences between those types also
integrate all documents into the latest and final requirements analysis (or the current one for the iteration or whatever agile programming cycle you are using ... )
remember that requirements do change over the software life cycle , so gathering is one thing but managing and implementing another
KISS - keep it as simple as possible
study also the environment where the future system will reside - there are more and more technological restraints from legacy or surrounding systems , since the companies do not prefer to throw to the garbage the money they have invested for decades even if in our modern minds 20 years old code is garbage ...
Like most stages of the software development process its iteration works best.
First find out who your users are -- the XYZ dept,
Then find out where they fit into the organisation -- part of Z division,
Then find out what they do in general terms -- manage cash
Then in specific terms -- collect cash from tills, and check for till fraud.
Then you can start talking to them.
Ask what problem they want you want to solve -- you will get an answer like write a bamboozling system using OCR with shark technoligies.
Ignore that answer and ask some more questions to find out what the real problem is -- they cant read the till slips to reconcile the cash.
Agree a real solution with the users -- get a better ink ribbon supplier - or connect the electronic tills to the network and upload the logs to a central server.
Then agree in detail how they will measure the success of the project.
Then and only then propose and agree a detailed set of requirements.
I would suggest you to read Roger-Pressman's Software Engineering: A Practitioner's Approach
Before you go talking to the stakeholders/users/anyone be sure you will be able to put down the gathered information in a usefull and days-lasting way.
Use a sound-recorder if it is OK with the other person and the information is bulky.
If you heard something important and you need some reasonable time to write it down, you have two choices: ask the other person to wait a second, or say goodbye to that precious information. You wont remember it right, ask any neuro-scientist.
If you detect that a point need deeper review or that you need some document you just heard of, make sure you make a commitment with the other person to send that document or schedule another meeting with a more specific purpose. Never say "I'll remember to ask for that xls file" because in most cases you wont.
Not to long after the meeting, summarize all your notes, recordings and fresh thoughts. Just summarize it rigth. Create effective reminders for the commitments.
Again, just after the meeting, is the perfect time to understand why the gathering you just did was not as right as you thought at the end of the meeting. That's when you will be able to put down a lot of meaningful questions for another meeting.
I know the question was in the perspective of the pre-meeting, but please be aware that you can work on this matters before the meeting and end up with a much usefull, complete and quality gathering.
I've been using mind mapping (like a work breakdown structure) to help gather requirements and define the unknowns (the #1 project killer). Start at a high level and work your way down. You need to work with the sponsors, users and development team to ensure you get all the angles and don't miss anything. You can't be expected to know the entire scope of what they want without their involvement...you - as a project manager/BA - need to get them involved (most important part of the job).
There are some great ideas here already. Here are some requirements gathering principles that I always like to keep in mind:
Know the difference between the user and the customer.
The business owners that approve the shiny project are usually the customers. However, a devastating mistake is the tendency to confuse them as the user. The customer is usually the person that recognizes the need for your product, but the user is the person that will actually be using the solution (and will most likely complain later about a requirement your product did not meet).
Go to more than one person
Because we’re all human, and we tend to not remember every excruciating detail. You increase your likelihood of finding missed requirements as you talk to more people and cross-check.
Avoid specials
When a user asks for something very specific, be wary. Always question the biases and see if this will really make your product better.
Prototype
Don’t wait till launch to show what you have to the user. Do frequent prototypes (you can even call them beta versions) and get constant feedback throughout the development process. You’ll probably find more requirements as you do this.
I recently started using the concepts, standards and templates defined by the International Institute of Business Analysts organization (IIBA).
They have a pretty good BOK (Book of Knowledge) that can be downloaded from their website. They do also have a certificate.
Requirements Engineering is a bit of an art, there are lots of different ways to go about it, you really have to tailor it to your project and the stakeholders involved. A good place to start is with Requirements Engineering by Karl Wiegers:
http://www.amazon.com/Software-Requirements-Second-Pro-Best-Practices/dp/0735618798/ref=pd_bbs_sr_2?ie=UTF8&s=books&qid=1234910330&sr=8-2
and a requirements engineering process which may consist of a number of steps e.g.:
Elicitation - for the basis for discussion with the business
Analysis and Description - a technical description for the purpose of the developers
Elaboration, Clarification, Verification and Negotiation - further refinement of the requirements
Also, there are a number of ways of documenting the requirements (Use Cases, Prototypes, Specifications, Modelling Languages). Each have their advantages and disadvantages. For example prototypes are very good for elicitation of ideas from the business and discussion of ideas.
I generally find that writing a set of use cases and including wireframe prototypes works well to identify an initial set of requirements. From that point it's a continual process of working with technical people and business people to further clarify and elaborate on the requirements. Keeping track of what was initially agreed and tracking additional requirements are essential to avoid scope creep. Negotiation plays a bit part here also between the various parties as per the Broken Iron Triangle (http://www.ambysoft.com/essays/brokenTriangle.html).
IMO the most important first step is to set up a dictornary of domain-specific words. When your client says "order", what does he mean? Something he receives from his customers or something he sends to his suppliers? Or maybe both?
Find the keywords in the stakeholders' business, and let them explain those words until you comprehend their meaning in the process. Without that, you will have a hard time trying to understand the requirements.
i wrote a blog article about the approach i use:
http://pm4web.blogspot.com/2008/10/needs-analysis-for-business-websites.html
basically: questions to ask your client before building their website.
i should add this questionnaire sheet is only geared towards basic website builds - like a business web presence. totally different story if you are talking about web-based software. although some of it is still relavant (e.g. questions relating to look and feel).
LM
I prefer to keep my requirements gathering process as simple, direct and thorough as possible. You can download a sample document that I use as a template for my projects at this blog posting: http://allthingscs.blogspot.com/2011/03/documenting-software-architectural.html

Resources