How to make the application intelligent to grade answers? - asp.net

I am adding a feature to an application in which the students answer questions that are more descriptive in nature. I am curious to know if there's a way to make the system "smart" enough to grade these answers. Ofcourse, I can run the answers through a set of keywords to ensure that the student has atleast included the keywords in the answers, but obviously this is not smart enough.
I know there's no fool proof way of grading descriptive answers, but was wondering if there's any technologies out there that I can look into.

You could use mechanical turk which is an API for humans. Which is probably as far as you can get with AI'ing your system. Understanding and grading actual text is one of the last remaining problems where humans are way better than computers (i.e. computers suck)
One notable exception is Watson which is actually really good at Jeopardy, but it runs on a huge computing cluster and includes some serious optimizations and smarts. That's nothing you just turn on. Sorry...

The answer is not so simple. There are "automated grading systems" out there, used, I believe, for example, to grade GRE exams. For example, see this paper and this by ETS.

Related

How can I learn higher-level programming-related math without much formal training? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
I haven't taken any math classes above basic college calculus. However, in the course of my programming work, I've picked up a lot of math and comp sci from blogs and reading, and I genuinely believe I have a decent mathematical mind. I enjoy and have success doing Project Euler, for example.
I want to dive in and really start learning some cool math, particularly discrete mathematics, set theory, graph theory, number theory, combinatorics, category theory, lambda calculus, etc.
My impression so far is that I'm well equipped to take these on at a conceptual level, but I'm having a really hard time with the mathematical language and symbols. I just don't "speak the language" and though I'm trying to learn it, I'm the going is extremely slow. It can take me hours to work through even one formula or terminology heavy paragraph. And yeah, I can look up terms and definitions, but it's a terribly onerous process that very much obscures the theoretical simplicity of what I'm trying to learn.
I'm really afraid I'm going to have to back up to where I left off, get a mid-level math textbook, and invest some serious time in exercises to train myself in that way of thought. This sounds amazingly boring, though, so I wondered if anyone else has any ideas or experience with this.
If you don't want to attend a class, you still need to get what the class would have given you: time in the material and lots of practice.
So, grab that text book and start doing the practice problems. There really isn't any other way (unless you've figured out how osmosis can actually happen...).
There is no knowledge that can only be gained in a classroom.
Check out the MIT Courseware for Mathematics
Also their YouTube site
Project Euler is also a great way to think about math as it relates to programming
Take a class at your local community college. If you're like me you'd need the structure. There's something to be said for the pressure of being graded. I mean there's so much to learn that going solo is really impractical if you want to have more than just a passing nod-your-head-mm-hmm sort of understanding.
Sounds like you're in the same position I am. What I'm finding out about math education is that most of it is taught incorrectly. Whether a cause or result of this, I also find most math texts are written incorrectly. Exceptions are rare, but notable. For instance, anything written by Donald Knuth is a step in the right direction.
Here are a couple of articles that state the problem quite clearly:
A Gentle Introduction To Learning
Calculus
Developing Your Intuition For
Math
And here's an article on a simple study technique that aims at retaining knowledge:
Teaching linear algebra
Consider auditing classes in discrete mathematics and proofs at a local university. The discrete math class will teach you some really useful stuff (graph theory, combinatorics, etc.), and the proofs class will teach you more about the mathematical style of thinking and writing.
I'd agree with #John Kugelman, classes are the way to go to get it done properly but I'd add that if you don't want to take classes, the internet has many resources to help you, including recorded lectures which I find can be more approachable than books and papers.
I'd recommend checking out MIT Open Courseware. There's a Maths for Computer Science module there, and I'm enjoying working through Gilbert Strang's Linear Algebra course of video lectures.
Youtube and videolectures.com are also good resources for video lectures.
Finally, there's a free Maths for CS book at bookboon.
To this list I would now add The Haskel Road to Logic, Maths, and Programming, and Conceptual Mathematics: A First Introduction to Categories.
--- Nov 16 '09 answer for posterity--
Two books. Diestel's Graph Theory, and Knuth's Concrete Mathematics. Once you get the hang of those try CAGES.
Find a good mentor who is an expert in the field who is willing to spend time with you on a regular basis.
There is a sort of trick to learning dense material, like math and mathematical CS. Learning unfamiliar abstract stuff is hard, and the most effective way to do it is to familiarize yourself with it in stages. First, you need to skim it: don't worry if you don't understand everything in the first pass. Then take a break; after you have rested, go through it again in more depth. Lather, rinse, repeat; meditate, and eventually you may become enlightened.
I'm not sure exactly where I'd start, to become familiar with the language of mathematics; I just ended up reading through lots of papers until I got better at it. You might look for introductory textbooks on formal mathematical logic, since a lot of math (especially in language theory) is based off of that; if you learn to hack the formal stuff a bit, the everyday notation might look a bit easier.
You should probably look through books on topics you're personally interested in; the inherent interest should help get you over the hump. Also, make sure you find texts that are actually introductory; I have become wary of slim, undecorated hardbacks labeled Elementary Foobar Theory, which tend to be elementary only to postdocs with a PhD in Foobar.
A word of warning: do not start out with category theory -- it is the most boring math I have ever encountered! Due to its relevance to language design and type theory, I would like to know more about it, but so far I have not been able to deal...
For a nice, scattershot intro to bits of many kinds of CS-ish math, I recommend Godel, Escher, Bach by Hofstadter (if you haven't read it already, of course). It's not a formal math book, though, so it won't help you with the familiarity problem, but it is quite inspirational.
Mathematical notation is is akin to several computer languages:
concise
exacting
based on many idioms
a fair amount of local variations and conventions
As with a computer language, you don't need to "wash the whole elephant at once": take it one part a at time.
A tentative plan for you could be
identify areas of mathematics that are interesting or important to you. (seems you already have a bit of a sense for that, CS has helped you develop quite a culture for it.)
take (or merely audit) a few formal classes in this area. I agree with several answers in this post, an in-person course, at local college is preferable, but, maybe at first, or to be sure to get the most of a particular class, first self-teaching yourself in this area with MIT OCW, similar online resources and associated books is ok/fine.
if an area of math introduces too high of a pre-requisite in terms of fluency with notation or with some underlying concept or (most often mechanical computation and transformation techniques). No problem! Just backtrack a bit, learn these foundations (and just these foundations!) and move forward again.
Find a "guru", someone that has a broad mathematical culture and exposure, not necessarily a mathematician, physics folks are good too, indeed they can often articulate math in a more practical fashion. Use this guru to guide you, as he/she can show you how the big pieces fit together.
Note: There is little gain to be had of learning mathematical notation for its own sake. Rather it should be learned in context, just like say a C# idiom is better memorized when used and when associated with a specific task, rather than learned in vacuo. A related SO posting however provides several resources to decipher and learn mathematical notation
Project Euler takes problems out of context and drops them in for people to solve them. Project Euler cannot teach you anything effectively. I think you should forget about it, if it is popular it does not mean anything. You cannot study Mathematics through Project Euler as it contains only bits and pieces(and some pretty high level pieces) that you're supposed to know in order to solve the problems. Learning mathematics means to consider a subject and a read a book about it and solving exercices or reading solutions, that's how you learn math. If it so happens that through your reading you find something that is close to some project euler thing, your luck , but otherwise Project euler is a complete waste of time. I think the time is much better invested choosing a particular branch of mathematics and studying that. Let me explain why: I solved 3 pretty advanced Projec Euler problems and they were all making appeal to knowledge from Number theory which I happened to have because i studies some part of it. I do not think Iearned anything from Project Euler, it just happened that I already knew some number theory and solved the problems.
For example, if you find out you like number theory, take H. Davenport -> Hardy & Wright -> Kenneth & Rosen's , study those.
If you like Graph Theory take Reinhard Diestel's book which is freely available and study that(or check books.google.com and find whichever is more appropriate to your taste) but don't spread your attention in 999999 directions just because Project Euler has problems ranging from dynamic programming to advanced geometry or to advanced number theory, that is clearly the wrong way to go and it will not bring you closer to your goal.
This sounds amazingly boring
Well ... Mathematics is not boring when you find some problem that you are attached to, which you like and you'd like to find the solution to, and when you have the sufficient time to reflect on it while not behind a computer screen. Mathematics is done with pen and paper mostly(yes you can use computers .. but that's not really the point).
So, if you find a real-world problem, or some programming problem that would benefit from
you knowing some advanced maths, and you know what maths you have to study , it can be motivating to learn in that way.
If you feel you are not motivated it is hard to study properly.
There is also the question of what you actually mean when you say learn. Does the learning process stop after you solved the problems at the end of the chapter of a book ? Well you decide. You can consider you have finished learning that subject, or you can consider you have not finished and read more about it. There are entire books on just one equation and variations of it.
The amount of programming-related math that you can learn without formal training is limited, but it's more than enough. But maybe you can self-teach yourself.
It all boils down to your resources and motivation.
To know mathematics you have to do mathematics not programming(project euler).
For beginning to learn category theory I recommend David Spivak's Category Theory for the Sciences (AKA Category Theory for Scientists) because its relatively comprehensible due to many examples that enable understanding by analogy and which quickly builds a foundation for understanding more abstract concepts.
It requires the ability to reason logically and an intuitive notion of what is a set. It proceeds from sets and functions through basic category theory to adjoint functors, categories of functors, sheaves, monads and an introduction to operads. Two main threads throughout are modeling databases in terms of categories and describing categories with annotated diagrams called ologs. The bibliography provides references to more advanced and specialized topics including recent papers by Dr. Spivak.
An expected outcome from reading this book is the capability of understanding category theory texts and papers written for mathematicians such as Mac Lane's Category Theory for the Working Mathematician.
In PDF format it is available from http://math.mit.edu/~dspivak/teaching/sp13/ (the dynamic version is recommended since its the most recent). The open access HTML version is available from https://mitpress.mit.edu/books/category-theory-sciences (which is recommended since it includes additional content including answers to some exercises).

Can software developing in a large team be interesting and fun? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 2 years ago.
Improve this question
I've been in the business of developing hardware and software for 19 years now. In the earlier days the projects and teams I worked on were smaller, much more effective and more fun.
The effect of the input of one single developer to the final product and to its success was evident to everybody. We had direct contact to and feedback from the customers. This was rewarding for our work and a very effective way to improve the product.
With the years the complexity of hard and software increases and more and more people were needed to get things done on time. The downside of the trend to bigger teams for me is that the contribution of a single developer to the project success gets smaller and smaller. And we lose the contact to real world of the users and customers because of growing QA departments more and more.
I always enjoyed my work and kept in touch with latest technologies like OOP, UML, .NET, and whatever. I already worked a few years as a team leader but I didn't like it very much because I missed developing and coding.
I'm just frustrated about the fact that my piece of the whole "thing" we're working on gets smaller and smaller and I lose the overview about it and the contact to the ground. Please don't understand me wrong, I don't want to cry for the good old days but for me the work on more and more specialized sub modules of a giant system simply gets more and more boring.
I'm wondering if I'm alone feeling like that and maybe if you have some advice how to bring the fun back to my work. And sorry, no, I'm not interested in working on an open source project in my free time. Nine hours a day in front of a computer screen are enough, life is more than coding...
I also require interaction with and feedback from the customer. However, a customer can be many things. As long as I'm satisfying someone (end user, team leader, big boss, etc.) then that's enough for me. The interaction itself is the key factor.
As for the feeling of pride and ownership from having a large impact on the system, again it's a matter of focus. You are still creating something, even if it's a smaller piece of the whole.
I long ago realized that I'm a small fish in a big pond. Learning to feel happy about my place in that pond was the only solution.
IOW, it's all relative!
I guess it all depends, there is a degree of camaraderie that comes with smaller teams and a lesser chance of ego's colliding. I have experienced both and they both have their upsides and downsides. To be honest, while working on a larger team I learned so much from other programmers, you think you know a lot, but someone always knows more.
It all depends on the team and the egos of the individuals.
When working on a team with ego problems, it doesn't matter how cool the technology is or how much interaction you get with the customers. One bad apple can drain all of the fun out of working on an otherwise cool project.
On the other hand, if the team has gelled, it matters very little if the technology is out-of-date, or the business problem is boring. Working on an back-office accounting system using VI and 10-year-old beta C++ compilers can still be invigorating when you feel like your peers are in the same fight and have your back. When you learn from others and are listened to when you have some new approach to try. When the developers control the build/test/deploy process so that it's sane and improves the lives (and sleep patterns) of the support team. When your peers (and you them) are always willing to help with an obscure language issue or work through a maddening bug. That what makes programming fun and interesting regardless of everything else.
You may want to consider changing companies back to a smaller company where you had a broader set of responsiblities, for one idea. Also, what are changes in the process that would help with the points you don't like?
I do have the question of what you mean by large here? Would a team of 50 people in a project be large? Or is it more like 1,000 to be large? On one level I'm asking for scale as there are teams beyond large if one wants to look at all the developers that work on Microsoft's big products like Office and Windows while at the other end of the spectrum are the one person development teams that do it all.
I'd second Kelly's answer that it depends on the team and egos for another big factor in things. What do you consider fun? Is it finding more efficient ways to solve problems that have poor solutions? Is it conquering a Millenium puzzle? Or is seeing someone smile while using your software what makes it fun? Lots of different possible answers and while I can make suggestions, how good or bad they are is totally for you to interpret.
I don't think you're alone in disliking how as a company matures the process can change as new people in various roles are added with increased bureaucracy and losing agility as it may take more signatures to get a change to be allowed or developers lose that touch to the customer of their product. There is a spectrum of various ways to produce software and some places may have less process in place and be focusing on "just make it work" while other places may want the process to be much more formal and organized with 1,001 policies for every little thing. At which end do you want to be working?
To answer the question as it's asked in the title: No!
I feel very similar and talked to many others who think the same. From my experience small teams are much more fun to work with and by that (and some other reasons) they're much more effective.
Thank you all for your interesting and valuable answers (and for correcting grammar and spelling :-)
You gave me some big points to think about:
The missing interaction with custumers (whatever "customer" means)
The interaction and feedback inside the developer team
What means fun for me. I think its more the smile in the face of the user than the use of cutting-edge technology.
How to deal with the sometimes overwhelming processes.
Last but not least to find my comfortable place in the big pond. It may be not the one where I'm staying at the moment...

Can Scrum and Lean principles ruins the life of professionals? [closed]

Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I work with scrum about 2 months and don’t have all the experience I wish, so I would like to hear some inputs about it.
My concern is people never say about drawbacks for the two sides; company and workers.
I know the benefits of a cross-functional team but which are the drawbacks? What is hidden beside the amazing Eden Garden?
I'm confused because as a company benefits of replaceable people, for the team is good because the opportunity of having knowledge and share experience (besides all teamwork benefit).
Again, I know all the benefits but I want explore the drawbacks just because in the middle there are the ordinary people.
Normally these people dedicate heavily to gain knowledge. They buy books, courses, attending seminar and so on.
In every company when someone knows much more than everyone else, people and managers get desperate wishing or even demanding that these ordinary people share all their knowledge.
And that’s strange.. Because these are communism thoughts and we live in capitalism society and since I was born, everything was so competed and now people say about collaborative.
Can Scrum and Lean principles ruins (or making hard) the professionals' life?
Scrum and Lean, in and of themselves, cannot ruin anybody's life. Nor can they, alone, make your life.
The culture of your organization will always be a far more dominant factor than the particular product management or development management method in place. Scrum can be misused. Lean can indeed make workers feel replaceable and pressured to perform all day, all the time.
On the other hand, both tools (they are just tools) can be used to create high-performing teams where all members value each other and each others' contributions. Being on a team that delivers consistently good results at high velocity feels great.
You will also find every result in between. It depends much more on culture than process.
I believe that culture flows from the top. Therefore, look at how the company leaders treat each other, their subordinates, their vendors, and their customers. That will tell you much more about what your life will be like than which methodology the company follows.
I'm only going to address your comments about sharing knowledge reducing your own value. In an ideal team culture, knowledge itself isn't as valued as someone's ability to acquire new knowledge and solve problems they haven't seen before. When I think about the star engineers I have known, it's not because they know this or that, it's because it's obvious they could be on nearly any task, on nearly any team and they would both begin to solve the problem and raise the level of the entire team.
There are a few things I've seen from agile methodologies which I'd put against it when you're weighing it up.
From a developers perspective there are two things:
1) The short sprints often lead to short term decisions - which is as intended but can be frustrating for some developers. While delivering "just enough" is great for the project, asking a developer to do something that they know that they're going to have to very heavily refactor, if not rewrite, two sprints down the line can be demotivating.
2) Where you've got opinionated developers (and is there any other sort) I've seen conflict over prioritisation. Adding not only what should be done but how important it is and therefore when it should be done brings on a whole other level of disagreement. In theory the developers don't have a say here but hard delination never works.
From a management point of view they don't like the uncertainty. "When's it going to be ready?" "No idea, when we get to the point you say you're happy". Essentially for them it's a leap of faith - if they do it once then generally they're sold but getting them to do the first time is hard.
I will assume, that as one commentator suggested, you meant to ask: "What are the drawbacks of Scrum?"
I think that the biggest problem with Scrum is that it is easy to understand - but very difficult to implement properly. Scrum, like XP, like most methodologies is not built on individual atomic practices, each capable of improving an existing process.
Scrum requires a shift in the organizational mindset. It requires a shift from ego-centric to communal behavior. The entire organization should focus on bringing the most value, constantly, and do so over perceived self-interest.
For example, a cross-functional team member may be required to do things out of his comfort zone (the flip-side of being able to experiment with new interesting tasks), because it needs to be done by somebody.
Team Leaders and project managers need to relinquish authority when they are called to take on the role of servant-leaders, and when they are asked to stop telling team-members which tasks to pick, instead relying on the team to manage itself.
Stakeholders are forced to face the reality that they can't eat the cake, and have it whole, when they are forced to choose between having all of the scope they want or having it by the date they want it done (this is always true, but Scrum is really in-your-face about it).
Most of all, the drawback of Scrum, is its tendency to disillusion beginning practitioners. This comes from people expecting something from it that it can't deliver: A solution to their problems!
That's right! Scrum does not solve an organizations problems. It highlights them. It is up to the organization to step up to the bat and do something about them. Incidentally, this is done with what I consider to be the single most important ceremony of Scrum - The Retrospective! If you do nothing else in Scrum - do the retrospective:
Find out what you did well, and continue doing it.
Find out what you need to improve and do something to improve it.
Rinse and repeat!
In a presentation by Ken Schwaber to Google on Scrum, he once said that Scrum isn't necessarily good for the organization. It could tell you early on that your project is doomed to fail. If you avoid Scrum, you may have a few more months of ignorant bliss to prepare you for the day you lose your job. Funny, but true. Think on that.
Hope it helps,
Assaf.
I'm no expert in any particular methodology (Agile, Scrum, etc.) but I empathize with your feelings. One of the biggest issues I've seen is that a team that really isn't interested almost unanimously in the methodology will tend to have problems. A few outliers isn't a problem, but if 1/3 or more of the team isn't interested, it becomes a nightmare. Writing good software is important and a company should hire professionals that help them meet that goal, but if the team is forced to meet that objective without finding the experience rewarding the quality will soon drop off.
No, I don't think it will ruin your professional life, but it can be pretty miserable if a company is pig-headed and doesn't realize that they need an environment where their workers are finding rewarding work.
I'm not totally sure of the question because it was kind of hard to follow.
Basically... no? I fail to see how an agile principle could 'ruin a professionals' life'... if implemented incorrectly it could waste some of their time, meaning a small lack of experience gained. Other than that, if the methodology fits the business and is implemented correctly, then is is a powerful tool that is useful to everybody.
Any methodology only works if the people are competent.
Silly question imo.
I've certainly seen things packaged as Scrum and Lean make the development process more difficult. Usually the result of managers picking and choosing the aspects that support their purpose, without buying in to the underlying spirit. Any process can work if properly applied, and process can fail if applied poorly.
Cross-functionality isn't a means to make people replaceable. It's a means to solve flow bottleneck problems that decrease productivity.
Cross-functionality doesn't mean that everyone can do everyone else's job, it means that people are capable of assisting with the work step that come (immediately) before their work or (immediately) after their work.
A better term for this is "Local Generalization", or "Special Generalization". And again, the goal has nothing to do with making people more removable from the organization. Creating the kid of people who can use Local Generalization to their advantage costs a lot of money in teaching and guidance. Once an organization makes such an investment in a worker, they're even less motivated to remove them. And organizations tend to only make such investments in people that they already want to keep around.

Does the Scrum process ultimately divest team members from their respective skills? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
My organization has been experimenting with the introduction of more "Agile" methods. We've been trying the Scrum approach for a short while, and most of the team has, more or less, adapted to it. I like it as a whole, but I'm concerned about one potentially severe impact of the methodology: as teams are consistently focused on features and backlog items, and testers are more integrated with the overall development process, it seems like skill sets are becoming blurred, and people are sensing less respect for their individual abilities.
Some of our developers are excellent at server-side technologies and optimization of heavy-weight data provisioning. Others have invested a large amount of their careers learning GUI technologies and have developed a fundamental understanding of users and usability in an application. Neither skill set is better than the other, but they are certainly different.
Is this an inevitable result of the Scrum process? Since everyone on the team (as I understand it) contributes to satisfying the next feature/requirement, backlog item, or testing goal at hand, the underlying philosophy seems to be "anyone can do it." This is, in my experience, simply not true. Most engineers (developers, testers, etc.) have a particular skill set they have honed over the years, and the Scrum methodology, in my mind, tends to devalue those very abilities they were previously respected for.
Here's an example for clarification:
If a sudden change of technology occurs on the server-side data provisioning, and every item on the to-do list for the sprint is based on this new change, the GUI developers (who likely haven't had time to become acclimated with the new technology) might not be able to contribute to the sprint. At the very least, they will need to invest time to get ramped up, and then their code will be suspect because of their lack of experience.
I understand the need for rapid development to discourage "role silos" but doesn't this discount one fundamental reality: people develop skills in accordance to necessity, their interests, or their experiences. People seem to be less motivated when they perceive their position is one of "plug-ability" (e.g. we can "plug" anyone in to do this particular task). How does Scrum address this? If it doesn't, has anyone addressed this when adopting the Scrum methodology?
The short answer is an emphatic NO! Scrum does not blur or depreciate the skills required for specialization. Scrum does not promote generalization.
The long answer is that in Scrum, the most important thing is to get the work "Done". The team, as a team (as opposed to a collection of individual "stars") collaborate, as needed, in order to get the job done. Whatever it takes - however they want (Scrum is about self managing, self motivating teams, right?).
What this means is that a scrum team may be composed of several specialists, who primarily do what they specialized in (DBA, Graphic Design, even technical writers). The team, as a whole, should have all of the skills required to fulfill the requirements. This is not the same as saying that each team member has to have all of the skills aforementioned.
That being said, it is often desired - often by the members themselves - that members other than the specialists be at least adequate in skills different from their specialty. Another poster already mentioned Scott Ambler's "General Specialist". This helps the team when there's too much work of one kind, when the specialist is absent, and it helps the member when he really would like to gain experience outside his specialty.
Given that the team is self organizing, if for some reason a specialist finds himself in the middle of the sprint, without any work to do in his specialty, the best way to deal with it, is to simply ask the specialist what he wants to do. Let the team decide. The specialist can decide to help in his other areas of adequacy, do a POC for the next sprint, "shore-up" the defenses by fixing some long forgotten technical debt, or shine the shoes of the members who are working.
Yup. I don't know if this is the long answer. But it definitely was a long answer.
:-)
The point of Scrum is for the developers to self-organize. We use scrum where I am, and jobs get passively sorted by a person's focus. We don't do it on purpose with a chart and list, it just happens. We all know who's best at what, or what their main/secondary focuses are. If the 'main' person needs help, they get the person/people with a secondary focus in it to help. We do get plenty of tasks not necessarily in line with whatever our particular focus is, but you always know who to ask for help then.
For your example - I don't know that if you say had 3 server guys and 5 gui guys, that you'd expect to get all the work done in that sprint (if the server guys + some help from the others wasn't enough). The way the sprint is supposed to work is that from a prioritized list, the developers pick what they think they can get done in that 30-day timeframe. If that meant the GUI guys needed 2 days of server-side training in order to help, that's what it'd mean. Unless there were concurrent things also high up the list that they could do instead. The sprint tasks are not supposed to be dictated by management as a psuedo-deadline.
If you have a Safari account, there's an interesting mostly case-study book by one of the guy/s who invented scrum.
I've been working as a ScrumMaster for about 18 months and have worked with two different teams. I initially expected to experience the potential issues you raise but this has not been the case. What I generally observe is that the team evolves into a mixture of specialists and generalists as people find the appropriate role for themselves - one that they can enjoy and be successful at. This is self-organisation at work. I have never had a case where our specialists were sitting idle.
If this did occur, I would expect it to be raised as an issue in Sprint Retrospective and the team would discuss how to improve the situation. The most obvious (and brutal) conclusion would be to change the team composition.
I am not sure why skill set will get blurred. There is a fair amount of confusion in the agile world. Scrum is a project management process and not a software development process and should not be seen as one. The engineers have to follow their own methodologies like TDD or extreme programming to add their own part to being agile.
Nothing goes away in scrum.
PM's still document as they go
Architects still architect their components. The only thing is they just delay some major decisions to more responsible point in time.
Developers should still follow best practices such as SOLID principles to enable for refactoring in a consistent manner as features change.
I think Scott Ambler addresses this issue very thoroughly in http://www.agilemodeling.com/essays/generalizingSpecialists.htm...
His concept of a Generalizing Specialist is exactly the thing Collective Ownership / Scrum Team calls for, and makes total sense to me.
Its hard to achieve in real life though ;-)
If you find for any reason ('sudden change of technology' or not) that the amount of work required for a system over a sprint is greater than the amount available then there's a problem with your scheduling.
One fix is that, as you suggest, you take programmers from other areas and throw them onto the mix. How well this works depends on the skills of that person and how different the problem domain is, but treating programmers as generic units that can be farmed out as needed is generally not a successful strategy for developing software.
This is still a scheduling problem though.
The best thing about Scrum is exactly the fact that skills do get a bit blurred! The point is to avoid silos at all costs by spreading specialist knowledge across the team and letting people work a bit outside their comfort zone.
Obviously this is not for everybody. Some developers are happy in their own narrow specialist field and such people are more of a hindrance in a Scrum process than an asset, whereas well-rounded and multi-talented people who are determined to get the job done, usually adapt very very well to it and are far more productive.
One of the key benefits of Scrum is to get the whole team actually involved and invested into the project instead of tackling their own special tasks and then riding off to the horizon. I'd claim that for most people, this is a far more rewarding way of working than the conveyor belt -approach of waterfall processes.
So I'd advise to boldly embrace the mixing of skills and having people come together to take down nasty problems instead of relying on specialist silos. The result of teams consisting of motivated people can be surprising.
Sounds like this would lead to more well-rounded developers, and also allow those who are experts in certain areas to continue to contribute their expertise.
I haven't used Scrum much myself (yet), but from your description, these types of teams would lead to a team/organization that is also more well-rounded as a whole - and shouldn't that be the goal of any team?
Handling sudden changes is part of Agile and this may mean that some people have to go off and learn new skills. Course this is more within the general Agile philosophy than anything Scrum-specific. There may be some extreme cases where the customer or business decides to change the world by bringing in something new and thus has to handle the subsequent pain of those people ramping up but if this is what they want and the developers are overruled, then there are only a couple of choices: (Take your lumps and try to handle the major changes) or (quit and get out of there).
While there can be some cases where someone that has specialized in something may be able to do things faster, this doesn't necessarily mean much if that is just one person on the team that is an expert and there is enough work in that area for 10 people for the whole sprint. Should those not an expert simply not do that work and let that one person attempt to get through as much as he or she can? I don't think so but there should be something to be said for those that aren't the best at something still trying to get done what they can get done.

Requirements Gathering

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
How do you go about the requirements gathering phase? Does anyone have a good set of guidelines or tips to follow? What are some good questions to ask the stakeholders?
I am currently working on a new project and there are a lot of unknowns. I am in the process of coming up with a list of questions to ask the stakeholders. However I cant help but to feel that I am missing something or forgetting to ask a critical question.
You're almost certainly missing something. A lot of things, probably. Don't worry, it's ok. Even if you remembered everything and covered all the bases stakeholders aren't going to be able to give you very good, clear requirements without any point of reference. The best way to do this sort of thing is to get what you can from them now, then take that and give them something to react to. It can be a paper prototype, a mockup, version 0.1 of the software, whatever. Then they can start telling you what they really want.
See obligatory comic below...
In general, I try and get a feel for the business model my customer/client is trying to emulate with the application they want built. Are we building a glorified forms processor? Are we retrieving data from multiple sources in a single application to save time? Are we performing some kind of integration?
Once the general businesss model is established, I then move to the "must" and "must nots" for the application to dictate what data I can retrieve, who can perform what functions, etc.
Usually if you can get the customer to explain their model or workflow, you can move from there and find additional key questions.
The one question I always make sure to ask in some form or another is "What is the trickiest/most annoying thing you have to do when doing X. Typically the answer to that reveals the craziest business/data rule you'll have to implement.
Hope this helps!
Steve Yegge talks fun but there is money to be made in working out what other people's requirements are so i'd take his article with a pinch of salt.
Requirements gathering is incredibly tough because of the manner in which communication works. Its a four step process that is lossy in each step.
I have an idea in my head
I transform this into words and pictures
You interpret the pictures and words
You paint an image in your own mind of what my original idea was like
And humans fail miserably at this with worrying frequency through their adorable imperfections.
Agile does right in promoting iterative development. Getting early versions out to the client is important in identifying what features are most important (what ships in 0.1 - 0.5 ish), helps to keep you both on the right track in terms of how the application will work and quickly identifies the hidden features that you will miss.
The two main problem scenarios are the two ends of the scales:
Not having a freaking clue about what you are doing - get some domain experts
Having too many requirements - feature pit. - Question, cull (prioritise ;) ) features and use iterative development
Yegge does well in pointing out that domain experts are essential to produce good requirements because they know the business and have worked in it. They can help identify the core desire of the client and will help explain how their staff will use the system and what is important to the staff.
Alternatives and additions include trying to do the job yourself to get into the mindset or having a client staff member occasionally on-site, although the latter is unlikely to happen.
The feature pit is the other side, mostly full of failed government IT projects. Too much, too soon, not enough thought or application of realism (but what do you expect they have only about four years to make themselves feel important?). The aim here is to work out what the customer really wants.
As long as you work on getting the core components correct, efficient and bug-free clients usually remain tolerant of missing features that arrive in later shipments, as long as they eventually arrive. This is where iterative development really helps.
Remember to separate the client's ideas of what the program will be like and what they want the program to achieve.
Some clients can create confusion by communicating their requirements in the form of application features which may be poorly thought out or made redundant by much simpler functionality then they think they require. While I'm not advocating calling the client an idiot or not listening to them I feel that it is worth forever asking why they want a particular feature to get to its underlying purpose.
Remember that in either scenario it is of imperative importantance to root out the quickest path to fulfilling the customers core need and put you in a scenario where you are both profiting from the relationship.
Wow, where to start?
First, there is a set of knowledge someone should have to do analysis on some projects, but it really depends on what you are building for who. In other words, it makes a big difference if you are modifying an enterprise application for a Fortune 100 corporation, building an iPhone app, or adding functionality to a personal webpage.
Second, there are different kinds of requirements.
Objectives: What does the user want to accomplish?
Functional: What does the user need to do in order to reach their objective? (think steps to reach the objective/s)
Non-functional: What are the constraints your program needs to perform within? (think 10 vs 10k simultaneous users, growth, back-up, etc.)
Business rules: What dynamic constraints do you have to meet? (think calculations, definitions, legal concerns, etc.)
Third, the way to gather requirements most effectively, and then get feedback on them (which you will do, right?) is to use models. User cases and user stories are a model of what the user needs to do. Process models are another version of what needs to happen. System diagrams are just another model of how different parts of the program(s) interact. Good data modeling will define business concepts and show you the inputs, outputs, and changes that happen within your program. Models (and there are more than I listed) are really the key to the concern you list. A few good models will capture the needs and from models you can determine your requirements.
Fourth, get feedback. I know I mentioned this already, but you will not get everything right the first time, so get responses to what your customer wants.
As much as I appreciate requirements, and the models that drive them, users typically do not understand the ramifications of of all their requests. Constant communication with chances for review and feedback will give users a better understanding of what you are delivering. Further, they will refine their understanding based on what they see. Unless you're working for the government, iterations and / or prototypes are helpful.
First of all gather the requirements before you start coding. You can begin the design while you are gathering them depending on your project life cicle but you shouldn't ever start coding without them.
Requirements are a set of well written documents that protect both the client and yourself. Never forget that. If no requirement is present then it was not paid for (and thus it requires a formal change request), if it's present then it must be implemented and must work correctly.
Requirements must be testable. If a requirement cannot be tested then it isn't a requirement. That means something like, "The system "
Requirements must be concrete. That means stating "The system user interface shall be easy to use" is not a correct requirment.
In order to actually "gather" the requirements you need to first make sure you understand the businness model. The client will tell you what they want with its own words, it is your job to understand it and interpret it in the right context.
Make meetings with the client while you're developing the requirements. Describe them to the client with your own words and make sure you and the client have the same concept in the requirements.
Requirements require concise, testable example, but keep track of every other thing that comes up in the meetings, diagrams, doubts and try to mantain a record of every meeting.
If you can use an incremental life cycle, that will give you the ability to improve some bad gathered requirements.
You can never ask too many or "stupid" questions. The more questions you ask, the more answers you receive.
According to Steve Yegge that's the wrong question to ask. If you're gathering requirement it's already too late, your project is doomed.
High-level discussions about purpose, scope, limitations of operating environment, size, etc
Audition a single paragraph description of the system, hammer it out
Mock up UI
Formalize known requirements
Now iterate between 3 and 4 with more and more functional prototypes and more specs with more details. Write tests as you go. Do this until you have functional software and a complete, objective, testable requirements spec.
That's the dream. The reality is usually after a couple iterations everybody goes head-down and codes until there's a month left to test.
Gathering Business Requirements Are Bullshit - Steve Yegge
read the agile manifesto - working software is the only measurement for the success of a software project
get familiar with agile software practices - study Scrum , lean programming , xp etc - this will save you tremendous amount of time not only for the requirements gathering but also for the entire software development lifecycle
keep regular discussions with Customers and especially the future users and key-users
make sure you talk to the Persons understanding the problem domain - e.g. specialists in the field
Take small notes during the talks
After each CONVERSATION write an official requirement list and present it for approving. Later on it would be difficult to argue against all agreed documentation
make sure your Customers know approximately what are the approximate expenses in time and money for implementing "nice to have" requirements
make sure you label the requirements as "must have" , "should have" and "nice to have" from the very beginning, ensure Customers understand the differences between those types also
integrate all documents into the latest and final requirements analysis (or the current one for the iteration or whatever agile programming cycle you are using ... )
remember that requirements do change over the software life cycle , so gathering is one thing but managing and implementing another
KISS - keep it as simple as possible
study also the environment where the future system will reside - there are more and more technological restraints from legacy or surrounding systems , since the companies do not prefer to throw to the garbage the money they have invested for decades even if in our modern minds 20 years old code is garbage ...
Like most stages of the software development process its iteration works best.
First find out who your users are -- the XYZ dept,
Then find out where they fit into the organisation -- part of Z division,
Then find out what they do in general terms -- manage cash
Then in specific terms -- collect cash from tills, and check for till fraud.
Then you can start talking to them.
Ask what problem they want you want to solve -- you will get an answer like write a bamboozling system using OCR with shark technoligies.
Ignore that answer and ask some more questions to find out what the real problem is -- they cant read the till slips to reconcile the cash.
Agree a real solution with the users -- get a better ink ribbon supplier - or connect the electronic tills to the network and upload the logs to a central server.
Then agree in detail how they will measure the success of the project.
Then and only then propose and agree a detailed set of requirements.
I would suggest you to read Roger-Pressman's Software Engineering: A Practitioner's Approach
Before you go talking to the stakeholders/users/anyone be sure you will be able to put down the gathered information in a usefull and days-lasting way.
Use a sound-recorder if it is OK with the other person and the information is bulky.
If you heard something important and you need some reasonable time to write it down, you have two choices: ask the other person to wait a second, or say goodbye to that precious information. You wont remember it right, ask any neuro-scientist.
If you detect that a point need deeper review or that you need some document you just heard of, make sure you make a commitment with the other person to send that document or schedule another meeting with a more specific purpose. Never say "I'll remember to ask for that xls file" because in most cases you wont.
Not to long after the meeting, summarize all your notes, recordings and fresh thoughts. Just summarize it rigth. Create effective reminders for the commitments.
Again, just after the meeting, is the perfect time to understand why the gathering you just did was not as right as you thought at the end of the meeting. That's when you will be able to put down a lot of meaningful questions for another meeting.
I know the question was in the perspective of the pre-meeting, but please be aware that you can work on this matters before the meeting and end up with a much usefull, complete and quality gathering.
I've been using mind mapping (like a work breakdown structure) to help gather requirements and define the unknowns (the #1 project killer). Start at a high level and work your way down. You need to work with the sponsors, users and development team to ensure you get all the angles and don't miss anything. You can't be expected to know the entire scope of what they want without their involvement...you - as a project manager/BA - need to get them involved (most important part of the job).
There are some great ideas here already. Here are some requirements gathering principles that I always like to keep in mind:
Know the difference between the user and the customer.
The business owners that approve the shiny project are usually the customers. However, a devastating mistake is the tendency to confuse them as the user. The customer is usually the person that recognizes the need for your product, but the user is the person that will actually be using the solution (and will most likely complain later about a requirement your product did not meet).
Go to more than one person
Because we’re all human, and we tend to not remember every excruciating detail. You increase your likelihood of finding missed requirements as you talk to more people and cross-check.
Avoid specials
When a user asks for something very specific, be wary. Always question the biases and see if this will really make your product better.
Prototype
Don’t wait till launch to show what you have to the user. Do frequent prototypes (you can even call them beta versions) and get constant feedback throughout the development process. You’ll probably find more requirements as you do this.
I recently started using the concepts, standards and templates defined by the International Institute of Business Analysts organization (IIBA).
They have a pretty good BOK (Book of Knowledge) that can be downloaded from their website. They do also have a certificate.
Requirements Engineering is a bit of an art, there are lots of different ways to go about it, you really have to tailor it to your project and the stakeholders involved. A good place to start is with Requirements Engineering by Karl Wiegers:
http://www.amazon.com/Software-Requirements-Second-Pro-Best-Practices/dp/0735618798/ref=pd_bbs_sr_2?ie=UTF8&s=books&qid=1234910330&sr=8-2
and a requirements engineering process which may consist of a number of steps e.g.:
Elicitation - for the basis for discussion with the business
Analysis and Description - a technical description for the purpose of the developers
Elaboration, Clarification, Verification and Negotiation - further refinement of the requirements
Also, there are a number of ways of documenting the requirements (Use Cases, Prototypes, Specifications, Modelling Languages). Each have their advantages and disadvantages. For example prototypes are very good for elicitation of ideas from the business and discussion of ideas.
I generally find that writing a set of use cases and including wireframe prototypes works well to identify an initial set of requirements. From that point it's a continual process of working with technical people and business people to further clarify and elaborate on the requirements. Keeping track of what was initially agreed and tracking additional requirements are essential to avoid scope creep. Negotiation plays a bit part here also between the various parties as per the Broken Iron Triangle (http://www.ambysoft.com/essays/brokenTriangle.html).
IMO the most important first step is to set up a dictornary of domain-specific words. When your client says "order", what does he mean? Something he receives from his customers or something he sends to his suppliers? Or maybe both?
Find the keywords in the stakeholders' business, and let them explain those words until you comprehend their meaning in the process. Without that, you will have a hard time trying to understand the requirements.
i wrote a blog article about the approach i use:
http://pm4web.blogspot.com/2008/10/needs-analysis-for-business-websites.html
basically: questions to ask your client before building their website.
i should add this questionnaire sheet is only geared towards basic website builds - like a business web presence. totally different story if you are talking about web-based software. although some of it is still relavant (e.g. questions relating to look and feel).
LM
I prefer to keep my requirements gathering process as simple, direct and thorough as possible. You can download a sample document that I use as a template for my projects at this blog posting: http://allthingscs.blogspot.com/2011/03/documenting-software-architectural.html

Resources