Related
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 13 years ago.
I have been working for two years in software industry. Some things that have puzzled me are as follows:
There is lack of application of mathematics in current software industry.
e.g.: When a mechanical engineer designs an electricity pole , he computes the stress on the foundation by using stress analysis techniques(read mathematical equations) to determine exactly what kind and what grade of steel should be used, but when a software developer deploys a web server application he just guesses on the estimated load on his server and leaves the rest on luck and god, there is nothing that he can use to simulate mathematically to answer his problem (my observation).
Great softwares (wind tunnel simulators etc) and computing programs(like matlab etc) are there to simulate real world problems (because they have their mathematical equations) but we in software industry still are clueless about how much actual resources in terms of memory , computing resources, clock speed , RAM etc would be needed when our server side application would actually be deployed. we just keep on guessing about the solution and solve such problem's by more or less 'hit and trial' (my observation).
Programming is done on API's, whether in c, C#, java etc. We are never able to exactly check the complexity of our code and hence efficiency because somewhere we are using an abstraction written by someone else whose source code we either don't have or we didn't have the time to check it.
e.g. If I write a simple client server app in C# or java, I am never able to calculate beforehand how much the efficiency and complexity of this code is going to be or what would be the minimum this whole client server app will require (my observation).
Load balancing and scalability analysis are just too vague and are merely solved by adding more nodes if requests on the server are increasing (my observation).
Please post answers to any of my above puzzling observations.
Please post relevant references also.
I would be happy if someone proves me wrong and shows the right way.
Thanks in advance
Ashish
I think there are a few reasons for this. One is that in many cases, simply getting the job done is more important than making it perform as well as possible. A lot of software that I write is stuff that will only be run on occasion on small data sets, or stuff where the performance implications are pretty trivial (it's a loop that does a fixed computation on each element, so it's trivially O(n)). For most of this software, it would be silly to spend time analyzing the running time in detail.
Another reason is that software is very easy to change later on. Once you've built a bridge, any fixes can be incredibly expensive, so it's good to be very sure of your design before you do it. In software, unless you've made a horrible architectural choice early on, you can generally find and optimize performance hot spots once you have some more real-world data about how it performs. In order to avoid those horrible architectural choices, you can generally do approximate, back-of-the-envelope calculations (make sure you're not using an O(2^n) algorithm on a large data set, and estimate within a factor of 10 or so how many resources you'll need for the heaviest load you expect). These do require some analysis, but usually it can be pretty quick and off the cuff.
And then there are cases in which you really, really do need to squeeze the ultimate performance out of a system. In these case, people frequently do actually sit down, work out the performance characteristics of the systems they are working with, and do very detailed analyses. See, for instance, Ulrich Drepper's very impressive paper What Every Programmer Should Know About Memory (pdf).
Think about the engineering sciences, they all have very well defined laws that are applicable to the design, and building of physical items, things like gravity, strength of materials, etc. Whereas in Computer science, there are not many well defined laws when it comes to building an application against.
I can think of many different ways to write a simple hello world program that would satisfy the requirment. However, if I have to build an electricity pole, I am severely constrained by the physical world, and the requirements of the pole.
Point by point
An electricity pole has to withstand the weather, a load, corrosion etc and these can be quantified and modelled. I can't quantify my website launch success, or how my database will grow.
Premature optimisation? Good enough is exactly that, fix it when needed. If you're a vendor, you've no idea what will be running your code in real life or how it's configured. Again you can't quantify it.
Premature optimisation
See point 1. I can add as needed.
Carrying on... even engineers bollix up. Collapsing bridges, blackout, car safety recalls, "wrong kind of snow" etc etc. Shall we change the question to "why don't engineers use more empirical observations?"
The answer to most of these is in order to have meaningful measurements (and accepted equations, limits, tolerances etc) that you have in real-world engineering you first need a way of measuring what it is that you are looking at.
Most of these things simply can't be measured easily - Software complexity is a classic, what is "complex"? How do you look at source code and decide if it is complex or not? McCabe's Cyclomatic Complexity is the closest standard we have for this but it's still basically just counting branch instructions in methods.
There is little math in software programs because the programs themselves are the equation. It is not possible to figure out the equation before it is actually run. Engineers use simple (and very complex) programs to simulate what happens in the real world. It is very difficult to simulate a simulator. additionally, many problems in computer science don't even have an answer mathematically: see traveling salesman.
Much of the mathematics is also built into languages and libraries. If you use a hash table to store data, you know to find any element can be done in constant time O(1), no matter how many elements are in the hash table. If you store it in a binary tree, it will take longer depending on the number of elements [0(n^2) if i remember correctly].
The problem is that software talks with other software, written by humans. The engineering examples you describe deal with physical phenomenon, which are constant. If I develop an electrical simulator, everyone in the world can use it. If I develop a protocol X simulator for my server, it will help me, but probably won't be worth the work.
No one can design a system from scratch and people that write semi-common libraries generally have plenty of enhancements and extensions to work on rather than writing a simulator for their library.
If you want a network traffic simulator you can find one, but it will tell you little about your server load because the traffic won't be using the protocol your server understands. Every server is going to see completely different sets of traffic.
There is lack of application of mathematics in current software industry.
e.g.: When a mechanical engineer designs an electricity pole , he computes the stress on the foundation by using stress analysis techniques(read mathematical equations) to determine exactly what kind and what grade of steel should be used, but when a software developer deploys a web server application he just guesses on the estimated load on his server and leaves the rest on luck and god, there is nothing that he can use to simulate mathematically to answer his problem (my observation).
I wouldn't say that luck or god are always the basis for load estimation. Often realistic data can be had.
It's also not true that there are no mathematical techniques to answer the question. Operations research and queuing theory can be applied to good advantage.
The real problem is that mechanical engineering is based on laws of physics and a foundation of thousands of years worth of empirical and scientific investigation. Computer science is only as old as me. Computer science will be much further along by the time your children and grandchildren apply the best practices of their day.
An MIT EE grad would not have this problem ;)
My thoughts:
Some people do actually apply math to estimate server load. The equations are very complex for many applications and many people resort to rules of thumb, guess and adjust or similar strategies. Some applications (real time applications with a high penalty for failure... weapons systems, powerplant control applications, avionics) carefully compute the required resources and ensure that they will be available at runtime.
Same as 1.
Engineers also use components provided by others, with a published interface. Think of electrical engineering. You don't usually care about the internals of a transistor, just it's interface and operating specifications. If you wanted to examine every component you use in all of it's complexity, you would be limited to what one single person can accomplish.
I have written fairly complex algorithms that determine what to scale when based on various factors such as memory consumption, CPU load, and IO. However, the most efficient solution is sometimes to measure and adjust. This is especially true if the application is complex and evolves over time. The effort invested in modeling the application mathematically (and updating that model over time) may be more than the cost of lost efficiency by try and correct approaches. Eventually, I could envision a better understanding of the correlation between code and the environment it executes in could lead to systems that predict resource usage ahead of time. Since we don't have that today, many organizations load test code under a wide range of conditions to empirically gather that information.
Software engineering are very different from the typical fields of engineering. Where "normal" engineering are bound to the context of our physical universe and the laws in it we've identified, there's no such boundary in the software world.
Producing software are usually an attempt to mirror a subset of the real-life world into a virtual reality. Here we define the laws ourselves, by only picking the ones we need and by making them just as complex as we need. Because of this fundamental difference, you need to look at the problem-solving from a different perspective. We try to make abstractions to make complex parts less complex, just like we teach kids that yellow + blue = green, when it's really the wavelength of the light that bounces on the paper that changes.
Once in a while we are bound by different laws though. Stuff like Big-O, Test-coverage, complexity-measurements, UI-measurements and the likes are all models of mathematic laws. If you look into digital signal processing, realtime programming and functional programming, you'll often find that the programmers use equations to figure out a way to do what they want. - but these techniques aren't really (to some extend) useful to create a virtual domain, that can solve complex logic, branching and interact with a user.
The reasons why wind tunnels, simulations, etc.. are needed in the engineering world is that it's much cheaper to build a scaled down prototype, than to build the full thing and then test it. Also, a failed test on a full scale bridge is destructive - you have to build a new one for each test.
In software, once you have a prototype that passes the requirements, you have the full-blown solution. there is no need to build the full-scale version. You should be running load simulations against your server apps before going live with them, but since loads are variable and often unpredictable, you're better off building the app to be able to scale to any size by adding more hardware than to target a certain load. Bridge builders have a given target load they need to handle. If they had a predicted usage of 10 cars at any given time, and then a year later the bridge's popularity soared to 1,000,000 cars per day, nobody would be surprised if it failed. But with web applications, that's the kind of scaling that has to happen.
1) Most business logic is usually broken down into decision trees. This is the "equation" that should be proofed with unit tests. If you put in x then you should get y, I don't see any issue there.
2,3) Profiling can provide some insight as to where performance issues lie. For the most part you can't say that software will take x cycles because that will change over time (ie database becomes larger, OS starts going funky, etc). Bridges for instance require constant maintenance, you can't slap one up and expect it to last 50 years without spending time and money on it. Using libraries is like not trying to figure out pi every time you want to find the circumference of a circle. It has already been proven (and is cost effective) so there is no need to reinvent the wheel.
4) For the most part web applications scale well horizontally (multiple machines). Vertical (multithreading/multiprocess) scaling tends to be much more complex. Adding machines is usually relatively easy and cost effective and avoid some bottlenecks that become limited rather easily (disk I/O). Also load balancing can eliminate the possibility of one machine being a central point of failure.
It isn't exactly rocket science as you never know how many consumers will come to the serving line. Generally it is better to have too much capacity then to have errors, pissed of customers and someone (generally your boss) chewing your hide out.
I'll do a demo of my code to slightly non-technical audience, and I need to show them what I've got in my project (about 15K lines of code). I'm trying to convince them that I've spend time on the project and it's in a good state.
These guys planning to invest money into this product. Therefore I should convince them that this app worth the price that they are going to spend and justify the time I've spent, secondly they should see that this is something takes time and I know what I'm doing (basically I need to win their trust) .
What metrics I can use other than "lines of code"? (Maybe lines of comment?)
What are the best tools (preferably free) to generate a report from .NET Projects?
UPDATE :
Also a way to provide "project cost - cocomo" would be cool, like this one :
FOUND:
http://www.cms4site.ru/utility.php?utility=cocomoii will help you to calculate an estimated cost for your project.
If they're non-technical, it won't matter. It will be like trying to sell a high-end bike to people who don't know a bike from a car. 15k lines of code won't matter to them any more than 300k lines of code will.
You need to find something other than the actual code to wow them with.
Can you code up some demos and tell them how short time it will take them to build similar applications with your code? Like "If you use my code, you can build this multimedia application in 15 minutes without writing more than a few lines of code". Non-technical people generally love saving time and money.
It probably depends on how "slightly" they are in the non-technical department.
An investor only cares about money. Investors start at the exit and work backwards. Knowing this, pitch your project in terms of the return they will get in their investment.
Key points would include:
Your expertise: Do you know the market you want to sell in to? Are you leveraging your expertise in some way to make the project a reality?
Risk: Using your already existing code base lowers risk in terms of both time and money. They will probably do technical due diligence to validate your claims, so be honest here.
Time to Market: Having a code base in place will reduce their time to market, which may be significant.
Vision: They need to know that there is a future for your product. This is your chance to get them excited!
Investment is about the future, not the past, so understand that you need to achieve what you are promising. The path you trod to get to where you are now may be interesting, but largely irrelevant to the investor. What I'm trying to say is sell the vision, not where you are now or where you've been.
Good luck and hope you get what you need!
It's not clear to me from your question whether you're talking about people who would buy the use of your product or ownership of your product.
In either case, ask yourself these questions:
"What problem(s) does this product solve for my users, from their point of view?"
"What does this product let the users do, that they already want to do, but can't do without it?"
"What does this product let the users do, that they already want to do, but can't do as easily without it?"
Features don't matter. Menus and dialogs don't matter (unless they require explanation, in which case they matter in the negative sense).
If you want numbers that interest a potential buyer of (an instance of) the product, talk in terms of how much time or money the buyer can save by using your product.
If you want numbers that interest a potential buyer of shares in your company or product, talk in terms of the size of the market, how you've analyzed that market's needs, and the ROI of any investment.
I've had success showing potential customers our automated build cycle, in slideshow form. I took them through our "production line" as if it was a factory tour, and showed the nice colored bars of coverage reports, uptilted lines of historical lines of code, pie charts of breakdowns of lines of code per module.
Then I did the same for everything aroung the actual building. So there's a requirements pipeline where they are involved, and a test/validation cycle where they are again involved.
It may not mean anything to them, but it shows them you have control over your process, and control over the quality of the delivered end product.
Please note that although people may be non-technical, try to be as honest as possible. As soon as they discover one single tiny lie in your story, you're lost. And chances are that there's that one technical guy in the back who can ask that one question which makes your house of cards fall down.
Happy sales!
"good code" doesn't matter unless you are demonstrating the medium and long term advantages of it - enhanced flexibility, simplicity, which saves customer time/money while adding agility.
I think explaining the more complex aspects of the code and the work that went into it to any audience will help show how much work and effort have gone into a project.
Hours spent coding could be a good metric to give them.
Talk about the features. Explain what you have working or almost working. Go at it from what they are interested in.
Try to show them visuals that they care about if you can. I think a few minutes doodling on a board would be better than showing lines of code.
The only thing that is likely to matter to a buyer (particularly a non-technical one)is functionality. I would concentrate on selling the features. You might consider discussion how you have tested it to verify that it performs as you claim.
I wouldn't use code per se, since a non-techie wouldn't understand it. Boasting about quantity is probably meaningless (how does a non-techie know that a 1MLOC project is significant? As for quality, you can present, e.g., maintainability metrics, test coverage, things like that. Feel free to show off your excellent toolchain too (continuous integration and all that), your mastery of various performance-testing tools. Also, showing things like Workflow Foundation helps - customers like to see how their business processes can be turned directly into code with a diagram notation.
EDIT modified to reflect OP's clarification (in comment here) that these potential buyers are looking to re-sell the software
Re-sellers are going to be looking for three things:
Is anyone going to make something better, cheaper or more quickly?
Is this guy going to be able to use our investment effectively to produce more?
Can we sell what this guy has produced, and will produce?
How to address points 1 and 2 have been very well addressed in other answers, but it's question 3 which is the hardest to prove for us techie people. It's also extremely important - if you can go to these buyers and hand them 3 killer benefits which they can repeat with more flair and Powerpoint when they're doing their sales calls, you'll be off to a good start :)
The main thing you have to do is to take a step back from your work and look at the:
features: what does it do
advantages: why is it better
benefits: why should the customer care
Features are closest to what you care about as a developer, but are pretty much irrelevant to non-technical buyers. Advantages are an essential step in understanding your competition and the customers' alternatives.
By putting features and advantages together, you can hit the customer with a number of benefits, e.g.:
using my software will save you $0.01 per transaction, or $40,000 p.a.
my software will increase customer retention by 5%
your admins will need 15% less time to deploy changes using my software
These are the things that customers care about: what's going to be good for the company, and good for them.
To be brutally honest: the end customer don't care how much effort you put into it (LoC or any other metric), they don't care how well it's written (comments, tests, any other metric), they don't care how hard a problem it is to solve, they don't care about features.
Their only requirement is that it will save them time / effort / money. You know that how hard you've worked to solve the problem, and solve it well, is key to their requirement, but it's secondary. You need to make it perfectly clear why them buying their stuff will mean they'll get promoted.
For COCOMO - Project Cost Estimation
I found this website, it's kind of a manual process but it'll do.
http://www.cms4site.ru/utility.php?utility=cocomoii
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have a developer on my staff that chronically overshoots deadlines, and estimates. On several projects the last week or two everyday I hear "It should be done by the end of the day". This developer does good work.
I have already spoke to him about his problems. He seems genuinely frustrated, and miffed about what to do to correct them.
My Questions are:
What kinds of punishments for passing a deadline are effective?
What ways can I coerce this employee to police his actions (time estimates, etc.,) himself?
UPDATE:
Based on the responses; here's what I have figured out.
Punishment is a bad idea.
It is natural for an employee to be unable to fix estimating problems without intervention.
Don't make deadlines unless there's company consequences (lost contract) for not being done by then.
Utilize available methods (Agile, Joel's checklist) to help the developer estimate better.
Thanks for the links and information. Also thanks for updating my thinking.
I don't think the problem is that he is missing these deadlines.
I think he has a real problem in estimating the amount of time it will take to complete a task.
Have him start keeping a journal of what he says a task will take and how long it actually took him to complete the task. Eventually, this journal will become a sort of guide for him to create better estimates. Once he becomes better at estimating, he shouldn't feel as rushed or harried.
There is an interesting article by Joel Spolsky: Evidence Based Scheduling
1) Break ‘er down
When I see a schedule measured in days, or even weeks, I know it’s not going to work. You have to break your schedule into very small tasks that can be measured in hours. Nothing longer than 16 hours.
This forces you to actually figure out what you are going to do. Write subroutine foo. Create this dialog box. Parse the Fizzbott file. Individual development tasks are easy to estimate, because you’ve written subroutines, created dialogs, and parsed files before.
If you are sloppy, and pick big three-week tasks (e.g., “Implement Ajax photo editor”), then you haven’t thought about what you are going to do. In detail. Step by step. And when you haven’t thought about what you’re going to do, you can’t know how long it will take.
Setting a 16-hour maximum forces you to design the damn feature. If you have a hand-wavy three week feature called “Ajax photo editor” without a detailed design, I’m sorry to be the one to break it to you but you are officially doomed. You never thought about the steps it’s going to take and you’re sure to be forgetting a lot of them.
The main point is that he (and you) should learn from his mistakes, and take them into account on the next estimation.
Also, if you are a developer, I would do regular code review at the end of the day to get a better insight into his development process.
And, of course, smaller iterations and more granularity with tasks. Set the maximum task duration to 1 day. That's the rule we have.
If your first question is
what kind of punishments to be considering
I think you're on a loser straight off. If you feel he does good work you may have to look at the deadlines/estimates and see if they were realistic in the first place. Who set them, if the developer in question was not involved then that may be part of the problem.
I agree with #OTisler that pair programming and possibly a regular end of day progress review with yourself can help him through... although if the deadlines/estimates were unrealistic to begin with thats not where your problem lies.
Closer monitoring on a few specific tasks should highlight where any issues lie.
What kinds of punishments for passing
a deadline are effective?
None. If you anger him, he won't do the work, or he'll find another job. You should help him figure out why his estimates are off. There is a book by steve McConnell about making estimates. i would start there.
What ways can I coherence this
employee to police his actions(time
estimates, etc.,) himself?
By helping him find the right way to make estimates.
First, make sure you are crystal clear in your requirements.
I hate to say it, but in my experience, blown deadlines are just as often a matter of unclear requirements or weak specifications on the part of a supervisor. First thing to do is to make sure the problem isn't either originating with, or exacerbated by, you.
Also, make sure your requirements are realistic, as well as his estimates.
Make sure that your own expectations aren't pushing him to make unrealistic estimates in order to meet unrealistic requirements.
Remember, you do the requirements, but the developer ALWAYS does the estimates, and should not be swayed with "can we do this any faster" unless you are also specifying functionality to be dropped.
Then, make sure he is tracking his time/tasks accurately, so you can get a good view of what is going on with the project.
This process will show any lack of proper time/task tracking, which may end up being the first step to improvement. If you can't see after the project how long a particular item took, that is probably the cause of the problem right there - not enough definition in the estimate, or missing "dependency" tasks that are discovered mid-project, but never estimated.
You HAVE to know how much time was spent doing what, accurately, before you can find out where the creep was, or what can be done about it.
Then, see where his estimates are failing and figure out why. Go over an estimate of a blown project, make that into a project itself - a problem to be solved.
Once you've determined that his estimates are indeed the source of the problem, go over an estimate that went over with him, and perhaps another developer, and figure out why.
This will help you figure out what the cause of the problem is. A solid understanding of the problem will likely be the actual solution.
Lastly, if you actually reach a point where you have to try punishment or coercion, it's time to fire him and start over.
Punishment and Coercion are appropriate responses to willful wrongdoing in certain situations.
However, if this developer is actively trying to do a good job, then you would only worsen the situation by generating negative attitude and frustration.
If the problem can't be solved, and you are sure the problem is with him, and not you, then it's time to fire him and get a developer who can meet deadlines. Great work doesn't mean much when your costs are blown up and profit goes out the window.
Okay, this is fairly common--developers being optimistic. It's the job of Management to deal with it. If anyone should be punished, it's the manager (you?)
I'm glad you at least asked, It looks like you got some good answers off this list, I hope they help and you find a way to actually implement some that work.
When I was young, my first good manager dealt with it this way:
First of all, he had me come up with an itemized list--breaking tasks down to hours, and estimating each one with a very liberal estimate--no period should be less than 4 hours regardless of how small the task was.
Then he looked at them and told me to double all my estimates. (Developers, especially younger developers, don't think about the fact that you are only productive for about 1/2 the day, if you're lucky--and half of that is spent at things you didn't expect to have to do).
Then, before creating his schedule, he doubled all my estimates (Without telling me).
He turned them in this way regardless of schedule requirements from above. A good manager should realize that saying it needs to be done in 2 days, doesn't make it possible.
As I got better at estimating we both noticed and adjusted accordingly.
A managers job isn't just to make a project, it's to build a team. More often than not that's going to require training of some sort. This is also the reason that an engineering manager that is not an engineer is unacceptable, they can't really help with this kind of thing.
Failure of a project or schedule is VIRTUALLY NEVER the fault of the developer (except in a few chronic cases where he isn't really fixable or of any worth and needs to be fired). The manager has made bad decisions either in hiring the developer, trusting him, managing him or staffing the project.
And really, what is fault anyway? I suppose if the manager isn't very good at making the project happen, he's going to need someone to point at... If HIS manager is any good, he'll ask why it got this far, what you did to fix it, etc.
Hiring a manager is hiring someone to solve the problems. To make the developers productive. If he can't make them productive, he isn't the right person.
To your questions:
If you choose to punish people for missing deadlines you will not get good results. They will be demotivated and feel belittled. If you keep pushing people to meet deadlines the quality of work will suffer and you will end up with a lot of time spent bug fixing afterwards.
To improve his time estimates you could try using Joel Spolsky's evidence based scheduling which has a nice feedback loop to improve the resulting estimates.
But I have some questions that I think you need to think about.
Is he later than everybody else? If so why - is it because he is an over optimistic estimator or a slow worker? Over optimistic estimates are easy to fix - just multiply all his numbers by a factor as per evidence based scheduling above. If he is a slow worker why? Does he get distracted? Is he very careful to produce very low defect code? Is he over engineering solutions? Is he not re-using code effectively?
Do the deadlines matter, or are they just arbitrary dates based on the estimates for the purposes of reporting progress up the management hierarchy? If the latter you can solve this by tweaking his estimates yourself.
What kinds of punishments for passing
a deadline are effective?
You stated the point and missed it. The obvious punishment for passing a deadline is death. If the developer is still alive after passing a deadline the "deadline" obviously was not a real deadline. Do you think it's funny to put developers under pressure using martial language?
Fix your wording.
Motivation
First of all: Read Peopleware
Next. Why do you think punishment will be an effective way to manage people that is supposed to be creative? I think you have to rethink the whole approach to management vs. team.
As I see it the managers first, and most important, role is to make sure that the developers can be creative and productive. Not that they are productive. There is a big difference in those small words. To be creative you need a safe environment. By being constantly under pressure from both deadlines and threats of punishment you create the exact opposite of safe.
Also, as a manager, you need accurate information on which to base decisions. This also requires a safe environment. If there is a risk for punishment for being honest and outspoken you are guaranteed to get lies and absence of information. A very dangerous base to take decisions from.
Estimates
As other as pointed out, estimates are estimates. In our team we don't do any individual estimates at all, we do estimates as a team. (I'm a bit reluctant to call what we do Scrum, but most of it tries to emulate if nothing less) I think this is a really great way to do estimates: Each team member is given a deck of cards consisting of numbers 0,1/2,1,3,5,8,13,20,40,60,100 and when estimating a task each developer picks a card (the cards are hidden until everyone has picked a card to avoid influencing estimates) and the average of the selected cards is taken as the estimate.
Notice how the numbers gets progressively less accurate. This is by design because large estimates are by necessity less accurate.
For our team we have opted to use the unit "ideal man days" for estimates. As far back as any of us can remember an ideal day hasn't occurred yet, but it is a good basis when you know how to translate calender days to "ideal man days".
As Scrum prescribes, development is done in sprints of two weeks after which the new version is deployed in the production environment. After each sprint we take the sum of the estimates of the completed tasks and divide that by the planned man days for the sprint. This factor is then the basis of estimating how many "ideal man days" the team can spend in a two week period.
Actual work items done by an individual developer don't need an estimate. The first approximation is always 1/2 - 1 day to complete. If this estimate turns out to be false you just grab a fellow developer and do it together to get it done. Or you break down the work item in smaller tasks so it can be distributed better.
Set Milestones and try Agile as #OTisler suggested.
I don't think you should punish him. Just get him to understand how to make accurate estimates.
As a team lead I've had my team members tell me that it will be "no problem" to finish X feature by the deadline. Then I usually sit down with them and go over what tasks and sub-tasks I think need to be done in order for the feature to be finished, and how long the developer thinks each will take.
After we do this exercise, and add up all the task and sub-task estimates, it will inevitably take much longer than the developer thinks in their original estimate. I usually only have to do this exercise with them a few times before they start making more accurate estimates.
What amazes me is that you only have one of these guys.
Engineers are horrible at estimating how much time something will take. I bet if you look carefully at your other developers' estimates, you'll find a lot of padding. Sometimes the padding isn't necessary, but the task expands to fill the available time anyway.
The solution to this is to change around how you do estimates - for everyone. Developers may be bad at estimating absolute time, but they're pretty good at relative time. So on Monday, instead of "how long will it take to add a whoosiwhatsit?," ask "what can you get done on the whoosiwhatsit in less than a week?" That becomes their task for the week.
The following Monday you look at how it went. "Well, I got the floogle installed in two days but it turns out it impacted the mcphee...so this week I need to decouple those guys so the whoosiwhatsit files don't get overwritten." Ok, there's their task for the week.
You might think it won't help, because you still don't know when the whoosiwhatsit is going to be ready. That's true. You have two choices here:
If you need a deadline, then you have to force your errant developer to pad his estimates like everyone else. It won't take him long to get the hang of it, and in no time at all he'll be taking "2 weeks" to write something that should have taken a day.
Your other choice is to trade the fictitious estimates for more visibility. In the long run this approach gets you more productive and much happier engineers.
So the developer does good work, but is poor at estimating the amount of time for delivery? I'm not sure you have a punishment situation on your hands just yet.
Maybe going forward for some time, have him walk you through his process for estimating a delivery point. This can be an opportunity to ask him why steps X,Y, and Z take certain amounts of time. He may find himself revising his estimates simply by doing the exercise at what is almost certainly a slower pace.
ask yourself this: What entails your job?
If you're just blindly passing estimates from developers (who you know can't give good estimates) up the management line, and not deciding for yourself whether that estimate is achievable, then you're not doing your job.
Try to think in terms of "value-add" (One of my old employers used that term a lot , and I hated it, but it probably works for you in this situation). What value are you adding? If you're just passing stuff in both directions between upper management and the developers, then ultimately you're not earning your money. You could be removed , and nothing would change.
The best manager I ever had was one that looked through a set of requirements given to him by another team , and told them straight out that almost a third of them was bull, and had them removed, before I ever even saw the list. The worst one I ever had made me write all this extra management-type documentation which none of the other managers I'd ever had asked me to do (I really got the impression I was literally doing his job for him), didn't even give me project due dates, and hardly turned up to work. They were both in the same company , bizarrely enough.
90 hours is one common short project deadline. The easy way is instead of estimating "your time", you measure another. Computer programmers shoudn't make time estimates for their projects since evidence shows calculating one's own time results in larger error than observing another.
I have the task of developing an online examination software for a small university, I need to implement measures to prevent cheating...
What are your ideas on how to do this?
I would like to possibly disable all IE / firefox tabs, or some how log internet activity so I know if they are googling anwsers...is there any realistic way to do such things from a flex / web application?
Simply put, no there is no realistic way to accomplish this if it is an online exam (assuming they are using their own computers to take the exam).
Is opening the browser window full screen an option? You could possibly also check for the window losing focus and start a timer that stops the test after some small period of time.
#Chuck - a good idea.
If the test was created in Flash/Flex, you could force the user to make the application fullscreen in order to start the test (fullscreen mode has to be user-initiated). Then, you can listen for the Flash events dispatched when flash exits fullscreen mode and take whatever appropriate action you want (end the test, penalize the user, etc.).
Flash/Flex fullscreen event info.
blog.flexexamples.com has an example fo creating a fullscreen-capable app.
Random questions and large banks of questions help. Randomizing even the same question (say changing the numbers, and calculating the result) helps too. None of these will prevent cheating though.
In the first case, if the pool is large enough, so that no two students get the same question, all that means is that students will compile a list of questions over the course of several semesters. (It is also a ton of work for the professors to come up with so many questions, I've had to do it as a TA it is not fun.)
In the second case, all you need is one smart student to solve the general case, and all the rest just take that answer and plug in the values.
Online review systems work well with either of these strategies (no benefit in cheating.) Online tests? They won't work.
Finally, as for preventing googling... good luck. Even if your application could completely lock down the machine. The user could always run a VM or a second machine and do whatever they want.
My school has always had a download link for the Lockdown browser, but I've never taken a course that required it. You can probably force the student to use it with a user agent check, but it could probably be spoofed with some effort.
Proctored tests are the only way to prevent someone from cheating. All the other methods might make it hard enough to not be worth the effort for most, but don't discount the fact that certain types of people will work twice as hard to cheat than it would have taken them to study honestly.
Since you can't block them from using google, you've got to make sure they don't have time to google. Put the questions in images so they can't copy and paste (randomize the image names each time they are displayed).
Make the question longer (100 words or more) and you will find that people would rather answer the question than retype the whole thing in google.
Give them a very short time. like 30-45 seconds. Time to read the question, think for a moment, and click either A, B, C, D, E,
(having just graduated from CSUN I can tell you scantron tests work.)
For essay questions? do a reverse google lookup (meaning put their answer into google as soon as they click submit) and see if you get exact matches. If so, you know what to do.
Will they always take the test on test machines, or will they be able to take the test from any machine on the network? If it will be specific machines, just use the hosts file to prevent them from getting out to the web.
If it is any machine, then I would look at having the testing backend change the firewall rules for the machine the test is running on so the machine cannot get out to the interwebs.
I'd probably implement a simple winforms (or WPF) app that hosts a browser control in it -- which is locked in to your site. Then you can remove links to browsers and lock down the workstations so that all they can open is your app.
This assumes you have control over the workstations on which the students are taking the tests, of course.
As a teacher, I can tell you the single best way would be to have human review of the answers. A person can sense copy/paste or an answer that doesn't make sense given the context of the course, expected knowledge level of the students, content of the textbook, etc, etc, etc.
A computer can do things like check for statistical similarity of answers, but you really need a person for final review (or, alternatively, build a massive statistical-processing, AI stack that will cost 10x the cost of human review and won't be as good ;-))
No, browsers are designed to limit the amount of damage a website or application can do to the system. You might be able to accomplish your goals through Java, an activex control, or a custom plugin, but other than that you aren't going to be able to 'watch' what they're doing on their system, much less control it. (Think if you could! I could put a spy on this webpage, and if you have it open I get to see what other websites you have open?)
Even if you could do this, using a browser inside a VM would give them the ability to use one computer to browse during the test, and if you could fix that they could simply use a library computer with their laptop next to it, or read things from a book.
The reality is that such unmonitored tests either have to be considered "open book" or "honor" tests. You must design the test questions in such a manner that references won't help solve the problems, which also means that each student needs to get a slightly different test so there is no way for them to collude and generate a key.
You have to develop an application that runs on their computer, but even then you can't solve the VM problem easily, and cannot solve the side by side computers or book problem at all.
-Adam
Randomize questions, ask a random set of questions from a large bank... time the answers...
Unless you mean hacking your site, which is a different question.
Short of having the application run completely on the user's machine, I do not believe there is a way to make sure they are not google-ing the answers. Even then it would be difficult to check for all possible loop-holes.
I have taken classes that used web based quiz software and used to work for a small college as well. For basic cheating prevention I would say randomize the questions.
Try adding SMS messages into the mix.
I agree with Adam, that even with the limitations that I suggested, it would still be trivial to cheat. Those were just "best effort" suggestions.
Your only hopes are a strong school honor code and human proctoring of the room where the test is being given.
As many other posters have said, you can't control the student's computer, and you certainly can't keep them from using a second computer or an iPhone along side the one being used for the test -- note that an iPhone (or other cellular device) can bypass any DNS or firewall on the network, since it uses the cellular provider's network, not the college's.
Good luck; you're going to need it.
Ban them from using any wireless device or laptop and keylog the machines?
You could enforce a small time window during which the test is available. This could reduce the chance that a student who knows the answers will be free to help one who doesn't (since they both need to be taking the test at the same time).
If it's math-related, use different numbers for different students. In general, try to have different questions for different copies of the test.
If you get to design the entire course: try to have some online homeworks as well, so that you can build a profile for each student, such as a statistical analysis of how often they use certain common words and punctuations. Some students use semi-colons often; others never, for example. When they take the test you get a good idea of whether or not it's really them typing.
You could also ask a couple questions you know they don't know. For example, list 10 questions and say they must answer any 6 out of the 10. But make 3 of the questions based on materials not taught in class. If they choose 2 or 3 of these, you have good reason to be suspicious.
Finally, use an algorithm to compare for similar answers. Do a simple hash to get rid of small changes. For example, hash an answer to a list of lower-cased 3-grams (3 words in a row), alphabetize it, and then look for many collisions between different users. This may sound like an obvious technique, but as a teacher I can assure you this will catch a surprising number of cheaters.
Sadly, the real trouble is to actually enforce punishment against cheaters. At the colleges where I have taught, if a student objects to your punishment (such as flunking them on the test in question), the administration will usually give the student something back, such as a positive grade change. I guess this is because the student('s parents) have paid the university a lot of money, but it is still very frustrating as a teacher.
The full screen suggestions are quite limited in their effectiveness as the user can always use a second computer or w/ multi monitor a second screen to perform their lookups. In the end it is probably better to just assume the students are going to cheat and then not count online tests for anything important.
If the tests are helpful for the students they will then do better on the final / mid term exams that are proctored in a controlled setting. Otherwise, why have them in the first place...
Make the questions and answers jpeg images so that you cannot copy and paste blocks of text into a search engine or IDE (if it is a coding test). This combined with a tight time limit to answer each question, say three minutes, makes it much harder to cheat.
I second what Guy said. We also created a Flex based examination system which was hosted in a custom browser built in .NET. The custom browser launched fullscreen, all toolbars were hidden and shortcuts were disabled.
Here is tutorial on how to create a custom browser with C# and VB.NET.
This will solve your problem. http://www.neuber.com/usermonitor/index.html
This will allow you to view the student's browser history during and after the test as well as look in on their screen during the test. Any urls visited during test time will be logged, so you can show them the log when you put a big F on their report card. :)
No one can stop people from cheating, but everyone can receive different questions altogether.
I prefer you buy available online scripts in market as starting point for it. This will save you time, cost and testing efforts.
Below is one of the fine scripts that I worked with and it worked like charm. Using this as base I developed a online testing portal of over 1000 users using computer adaptive test.
http://codecanyon.net/item/online-skills-assessment/9379895
It is a good starting point for people looking to develop Online Exam System.
I customized the script with the help of their support.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
How do you go about the requirements gathering phase? Does anyone have a good set of guidelines or tips to follow? What are some good questions to ask the stakeholders?
I am currently working on a new project and there are a lot of unknowns. I am in the process of coming up with a list of questions to ask the stakeholders. However I cant help but to feel that I am missing something or forgetting to ask a critical question.
You're almost certainly missing something. A lot of things, probably. Don't worry, it's ok. Even if you remembered everything and covered all the bases stakeholders aren't going to be able to give you very good, clear requirements without any point of reference. The best way to do this sort of thing is to get what you can from them now, then take that and give them something to react to. It can be a paper prototype, a mockup, version 0.1 of the software, whatever. Then they can start telling you what they really want.
See obligatory comic below...
In general, I try and get a feel for the business model my customer/client is trying to emulate with the application they want built. Are we building a glorified forms processor? Are we retrieving data from multiple sources in a single application to save time? Are we performing some kind of integration?
Once the general businesss model is established, I then move to the "must" and "must nots" for the application to dictate what data I can retrieve, who can perform what functions, etc.
Usually if you can get the customer to explain their model or workflow, you can move from there and find additional key questions.
The one question I always make sure to ask in some form or another is "What is the trickiest/most annoying thing you have to do when doing X. Typically the answer to that reveals the craziest business/data rule you'll have to implement.
Hope this helps!
Steve Yegge talks fun but there is money to be made in working out what other people's requirements are so i'd take his article with a pinch of salt.
Requirements gathering is incredibly tough because of the manner in which communication works. Its a four step process that is lossy in each step.
I have an idea in my head
I transform this into words and pictures
You interpret the pictures and words
You paint an image in your own mind of what my original idea was like
And humans fail miserably at this with worrying frequency through their adorable imperfections.
Agile does right in promoting iterative development. Getting early versions out to the client is important in identifying what features are most important (what ships in 0.1 - 0.5 ish), helps to keep you both on the right track in terms of how the application will work and quickly identifies the hidden features that you will miss.
The two main problem scenarios are the two ends of the scales:
Not having a freaking clue about what you are doing - get some domain experts
Having too many requirements - feature pit. - Question, cull (prioritise ;) ) features and use iterative development
Yegge does well in pointing out that domain experts are essential to produce good requirements because they know the business and have worked in it. They can help identify the core desire of the client and will help explain how their staff will use the system and what is important to the staff.
Alternatives and additions include trying to do the job yourself to get into the mindset or having a client staff member occasionally on-site, although the latter is unlikely to happen.
The feature pit is the other side, mostly full of failed government IT projects. Too much, too soon, not enough thought or application of realism (but what do you expect they have only about four years to make themselves feel important?). The aim here is to work out what the customer really wants.
As long as you work on getting the core components correct, efficient and bug-free clients usually remain tolerant of missing features that arrive in later shipments, as long as they eventually arrive. This is where iterative development really helps.
Remember to separate the client's ideas of what the program will be like and what they want the program to achieve.
Some clients can create confusion by communicating their requirements in the form of application features which may be poorly thought out or made redundant by much simpler functionality then they think they require. While I'm not advocating calling the client an idiot or not listening to them I feel that it is worth forever asking why they want a particular feature to get to its underlying purpose.
Remember that in either scenario it is of imperative importantance to root out the quickest path to fulfilling the customers core need and put you in a scenario where you are both profiting from the relationship.
Wow, where to start?
First, there is a set of knowledge someone should have to do analysis on some projects, but it really depends on what you are building for who. In other words, it makes a big difference if you are modifying an enterprise application for a Fortune 100 corporation, building an iPhone app, or adding functionality to a personal webpage.
Second, there are different kinds of requirements.
Objectives: What does the user want to accomplish?
Functional: What does the user need to do in order to reach their objective? (think steps to reach the objective/s)
Non-functional: What are the constraints your program needs to perform within? (think 10 vs 10k simultaneous users, growth, back-up, etc.)
Business rules: What dynamic constraints do you have to meet? (think calculations, definitions, legal concerns, etc.)
Third, the way to gather requirements most effectively, and then get feedback on them (which you will do, right?) is to use models. User cases and user stories are a model of what the user needs to do. Process models are another version of what needs to happen. System diagrams are just another model of how different parts of the program(s) interact. Good data modeling will define business concepts and show you the inputs, outputs, and changes that happen within your program. Models (and there are more than I listed) are really the key to the concern you list. A few good models will capture the needs and from models you can determine your requirements.
Fourth, get feedback. I know I mentioned this already, but you will not get everything right the first time, so get responses to what your customer wants.
As much as I appreciate requirements, and the models that drive them, users typically do not understand the ramifications of of all their requests. Constant communication with chances for review and feedback will give users a better understanding of what you are delivering. Further, they will refine their understanding based on what they see. Unless you're working for the government, iterations and / or prototypes are helpful.
First of all gather the requirements before you start coding. You can begin the design while you are gathering them depending on your project life cicle but you shouldn't ever start coding without them.
Requirements are a set of well written documents that protect both the client and yourself. Never forget that. If no requirement is present then it was not paid for (and thus it requires a formal change request), if it's present then it must be implemented and must work correctly.
Requirements must be testable. If a requirement cannot be tested then it isn't a requirement. That means something like, "The system "
Requirements must be concrete. That means stating "The system user interface shall be easy to use" is not a correct requirment.
In order to actually "gather" the requirements you need to first make sure you understand the businness model. The client will tell you what they want with its own words, it is your job to understand it and interpret it in the right context.
Make meetings with the client while you're developing the requirements. Describe them to the client with your own words and make sure you and the client have the same concept in the requirements.
Requirements require concise, testable example, but keep track of every other thing that comes up in the meetings, diagrams, doubts and try to mantain a record of every meeting.
If you can use an incremental life cycle, that will give you the ability to improve some bad gathered requirements.
You can never ask too many or "stupid" questions. The more questions you ask, the more answers you receive.
According to Steve Yegge that's the wrong question to ask. If you're gathering requirement it's already too late, your project is doomed.
High-level discussions about purpose, scope, limitations of operating environment, size, etc
Audition a single paragraph description of the system, hammer it out
Mock up UI
Formalize known requirements
Now iterate between 3 and 4 with more and more functional prototypes and more specs with more details. Write tests as you go. Do this until you have functional software and a complete, objective, testable requirements spec.
That's the dream. The reality is usually after a couple iterations everybody goes head-down and codes until there's a month left to test.
Gathering Business Requirements Are Bullshit - Steve Yegge
read the agile manifesto - working software is the only measurement for the success of a software project
get familiar with agile software practices - study Scrum , lean programming , xp etc - this will save you tremendous amount of time not only for the requirements gathering but also for the entire software development lifecycle
keep regular discussions with Customers and especially the future users and key-users
make sure you talk to the Persons understanding the problem domain - e.g. specialists in the field
Take small notes during the talks
After each CONVERSATION write an official requirement list and present it for approving. Later on it would be difficult to argue against all agreed documentation
make sure your Customers know approximately what are the approximate expenses in time and money for implementing "nice to have" requirements
make sure you label the requirements as "must have" , "should have" and "nice to have" from the very beginning, ensure Customers understand the differences between those types also
integrate all documents into the latest and final requirements analysis (or the current one for the iteration or whatever agile programming cycle you are using ... )
remember that requirements do change over the software life cycle , so gathering is one thing but managing and implementing another
KISS - keep it as simple as possible
study also the environment where the future system will reside - there are more and more technological restraints from legacy or surrounding systems , since the companies do not prefer to throw to the garbage the money they have invested for decades even if in our modern minds 20 years old code is garbage ...
Like most stages of the software development process its iteration works best.
First find out who your users are -- the XYZ dept,
Then find out where they fit into the organisation -- part of Z division,
Then find out what they do in general terms -- manage cash
Then in specific terms -- collect cash from tills, and check for till fraud.
Then you can start talking to them.
Ask what problem they want you want to solve -- you will get an answer like write a bamboozling system using OCR with shark technoligies.
Ignore that answer and ask some more questions to find out what the real problem is -- they cant read the till slips to reconcile the cash.
Agree a real solution with the users -- get a better ink ribbon supplier - or connect the electronic tills to the network and upload the logs to a central server.
Then agree in detail how they will measure the success of the project.
Then and only then propose and agree a detailed set of requirements.
I would suggest you to read Roger-Pressman's Software Engineering: A Practitioner's Approach
Before you go talking to the stakeholders/users/anyone be sure you will be able to put down the gathered information in a usefull and days-lasting way.
Use a sound-recorder if it is OK with the other person and the information is bulky.
If you heard something important and you need some reasonable time to write it down, you have two choices: ask the other person to wait a second, or say goodbye to that precious information. You wont remember it right, ask any neuro-scientist.
If you detect that a point need deeper review or that you need some document you just heard of, make sure you make a commitment with the other person to send that document or schedule another meeting with a more specific purpose. Never say "I'll remember to ask for that xls file" because in most cases you wont.
Not to long after the meeting, summarize all your notes, recordings and fresh thoughts. Just summarize it rigth. Create effective reminders for the commitments.
Again, just after the meeting, is the perfect time to understand why the gathering you just did was not as right as you thought at the end of the meeting. That's when you will be able to put down a lot of meaningful questions for another meeting.
I know the question was in the perspective of the pre-meeting, but please be aware that you can work on this matters before the meeting and end up with a much usefull, complete and quality gathering.
I've been using mind mapping (like a work breakdown structure) to help gather requirements and define the unknowns (the #1 project killer). Start at a high level and work your way down. You need to work with the sponsors, users and development team to ensure you get all the angles and don't miss anything. You can't be expected to know the entire scope of what they want without their involvement...you - as a project manager/BA - need to get them involved (most important part of the job).
There are some great ideas here already. Here are some requirements gathering principles that I always like to keep in mind:
Know the difference between the user and the customer.
The business owners that approve the shiny project are usually the customers. However, a devastating mistake is the tendency to confuse them as the user. The customer is usually the person that recognizes the need for your product, but the user is the person that will actually be using the solution (and will most likely complain later about a requirement your product did not meet).
Go to more than one person
Because we’re all human, and we tend to not remember every excruciating detail. You increase your likelihood of finding missed requirements as you talk to more people and cross-check.
Avoid specials
When a user asks for something very specific, be wary. Always question the biases and see if this will really make your product better.
Prototype
Don’t wait till launch to show what you have to the user. Do frequent prototypes (you can even call them beta versions) and get constant feedback throughout the development process. You’ll probably find more requirements as you do this.
I recently started using the concepts, standards and templates defined by the International Institute of Business Analysts organization (IIBA).
They have a pretty good BOK (Book of Knowledge) that can be downloaded from their website. They do also have a certificate.
Requirements Engineering is a bit of an art, there are lots of different ways to go about it, you really have to tailor it to your project and the stakeholders involved. A good place to start is with Requirements Engineering by Karl Wiegers:
http://www.amazon.com/Software-Requirements-Second-Pro-Best-Practices/dp/0735618798/ref=pd_bbs_sr_2?ie=UTF8&s=books&qid=1234910330&sr=8-2
and a requirements engineering process which may consist of a number of steps e.g.:
Elicitation - for the basis for discussion with the business
Analysis and Description - a technical description for the purpose of the developers
Elaboration, Clarification, Verification and Negotiation - further refinement of the requirements
Also, there are a number of ways of documenting the requirements (Use Cases, Prototypes, Specifications, Modelling Languages). Each have their advantages and disadvantages. For example prototypes are very good for elicitation of ideas from the business and discussion of ideas.
I generally find that writing a set of use cases and including wireframe prototypes works well to identify an initial set of requirements. From that point it's a continual process of working with technical people and business people to further clarify and elaborate on the requirements. Keeping track of what was initially agreed and tracking additional requirements are essential to avoid scope creep. Negotiation plays a bit part here also between the various parties as per the Broken Iron Triangle (http://www.ambysoft.com/essays/brokenTriangle.html).
IMO the most important first step is to set up a dictornary of domain-specific words. When your client says "order", what does he mean? Something he receives from his customers or something he sends to his suppliers? Or maybe both?
Find the keywords in the stakeholders' business, and let them explain those words until you comprehend their meaning in the process. Without that, you will have a hard time trying to understand the requirements.
i wrote a blog article about the approach i use:
http://pm4web.blogspot.com/2008/10/needs-analysis-for-business-websites.html
basically: questions to ask your client before building their website.
i should add this questionnaire sheet is only geared towards basic website builds - like a business web presence. totally different story if you are talking about web-based software. although some of it is still relavant (e.g. questions relating to look and feel).
LM
I prefer to keep my requirements gathering process as simple, direct and thorough as possible. You can download a sample document that I use as a template for my projects at this blog posting: http://allthingscs.blogspot.com/2011/03/documenting-software-architectural.html