we use a tool that operates on a certain database updating certain values when we makes some changes to the tool.
This takes a long time for some simple task.
I just need to find out which table,column and the value for that column gets updated.
for this i need to find out the whole database which column_name has value "XYZ" and the corresponding table/tables.
Any scripts for these.
Just because something can be done does not mean it should be done.
I know you've got your process designed this way and you very likely don't want to change it but, really, your life will get a whole lot better if you redesign this to avoid doing something that really, seriously, shouldn't be done. Searching through every text field in an entire database in search of some magical character string is a Bad Idea. It's actually only ONE of The Big Bad Ideas and it probably isn't the Baddest Idea, but it's a big enough bad enough idea that you should give Serious Consideration to doing something else, better.
OK, so what's wrong with it?
First, it indicates that you're not using a database, you're using a midden. You dump stuff in and then hope to dig it out later. This is the kind of thing people did thousands of years ago (it was popular back when flint was cutting edge technology), and while it helps keep archaeologists employed digging through these trash heaps, we are software developers, not archaeologists, and we don't want to have to do this kind of thing on a regular basis.
Second, this is a serious performance killer. You're going to either write some god-awful static code to laboriously check every field in every table, or you're going to write some middling-bright code to dynamically create some even more god-awful query that will laboriously check every field in every table. The word to focus on here is "laborious". And "god-awful", if it comes to that. Scanning through every row in every table in your database and testing every field in all of those every rows is going to be slow. Very, very slow. It's going to be dead-turtle-on-the-side-of-the-road-with-tire-marks-on-its-shell slow. This is not a good thing to do, unless you own stock in the local electric utility and want to make sure every generated electron has a happy home in your employers computer.
Third, people will have strong emotions when they see your code. Those destined for careers in management will laugh, for they know that they won't have to maintain it or try to solve the performance issues. The technically challenged will cry, because they'll know there's nothing they can do to fix it. The true Code Warriors will stare in amazement for a moment, and will then grit their teeth, hunt you down, and beat you to death with their ceremonial Wands Of Green-Bar, for only they will know that this evil could have been prevented.
So give some thought to a re-design. Once again, just because something can be done does not mean it should be done.
Share and enjoy.
I have the responsibility of ensuring that a colleague who is just learning R knows the basics before a course where that is a requirement. The colleague has gone through a couple of tutorials so hopefully she is ok, but I would like to give her a test to gauge it.
I was therefore wondering if anyone knew if there were any materials on the web that would be suitable? and is possible had both questions and answers.
PS Cross-posted to r-help#stat.math.ethz.ch
There is a whole set of exercises with solutions from the book Data Analysis and Graphics Using R. (Maindonald & Braun, 2nd edn, CUP 2007) available online : http://maths.anu.edu.au/~johnm/r-book/2edn/exercises/
Next to that, a quick search using the obscure randomized pagecollector Google brought to me a set of exercises where you can pick out whatever you want. Try the magic phrase "R exercises". ;)
Some I found interesting :
http://www2.imperial.ac.uk/~das01/RCourse/Exercises.pdf (very nice)
http://dial.liacs.nl/Courses/MicroArrayDataAnalysis/Exercises/Introduction_to_R_Exercises_Nov_2004.pdf (rather basic)
http://www.shlrc.mq.edu.au/masters/students/raltwarg/altwargslp802.html (rather basic)
If this is just a one time single person evaluation then a oral style exam is probably going to tell you a lot more than a set of fixed problems. Get a data set and have her read it into R, do some basic data manipulation, a couple of plots, and a standard analysis or two. Based on what she does well or has a challenge with you can modify the direction that you have things go and what additional questions you ask.
I am currently in the analysis phase of developing some sort of Locale-based Stock Screener ( please see Google's' for similar work) and I would appreciate advice from the SO Experts.
Firstly the Stock Screener would obviously need to store the formulas required to perform Calculations. My initial conclusion would that the formulae would need to be stored in the Database Layer. What are your ideas on this? Could I improve speed( very important) by storing formulas in a flat file(XML/TXT)?
Secondly, I would also like to ask advice on the internal execution of formulae by the Application. Currently I am leaning towards executing formulae on parameters AT RUN TIME as against running the formulae on parameters whenever these parameters are provided to the system and storing the execution results in the DB for simple retrieval later( My Local Stock Exchange currently does NOT support Real Time Stock Price updates). While I am quite certain that the initial plan ( executing at run time) is better initially , the application could potentially handle a wide variety of formulae as well as work on a wide variety of input parameters. What are your thoughts on this?
I have also gone through SO to find information on how to store formulae in a DB but wanted to enquire the possible ways one could resolve recursive formulae i.e. formaulae which require the results of other formulae to perform calculations? I wouldn't mind pointers to other questions or fora at all.
[EDIT]
[This page]2 provides a lot of infromation as to what I am trying to achieve but what is different is the fact that I need to design some formulae with SPECIAL tokens such as SP which would represent Stock Price for the current day and SP(-1) would represent price for the previous day. These special token would require the Application to perform some sort of DB access to retrieve the values which they are replaced with.
An example formula would be:
(SP/SP(-1)) / 100
which calculates Price Change for Securities and my idea is to replace the SP tokens with the values for the securities when Requested by the user and THEN perform the calculation and send the result to the user.
Thanks a lot for all your assistance.
Kris, I don't mean to presume that I have a better understanding of your requirements than you, but by coincidence I read this article this afternoon after I posted my earlier comment;
http://thedailywtf.com/Articles/Soft_Coding.aspx
Are you absolutely sure that the "convenience" of updating formulae without recompiling code is worth the maintenance head ache that such a solution may possibly become down the line?
I would strongly recommend that you hard code your logic unless you want someone without access to the source to be updating formulae on a fairly regular basis.
And I can't see this happening too often anyway, given that the particular domain here, stock prices, has a well established set of formulae for calculating the various relevant metrics.
I think your effort will be much better spent in making a solid and easily extensible "stock price" framework, or even searching for some openly available one with a proven track record.
Anyway, your project sounds very interesting, I hope it works out well whatever approach you decide to take. Good luck!
I'll do a demo of my code to slightly non-technical audience, and I need to show them what I've got in my project (about 15K lines of code). I'm trying to convince them that I've spend time on the project and it's in a good state.
These guys planning to invest money into this product. Therefore I should convince them that this app worth the price that they are going to spend and justify the time I've spent, secondly they should see that this is something takes time and I know what I'm doing (basically I need to win their trust) .
What metrics I can use other than "lines of code"? (Maybe lines of comment?)
What are the best tools (preferably free) to generate a report from .NET Projects?
UPDATE :
Also a way to provide "project cost - cocomo" would be cool, like this one :
FOUND:
http://www.cms4site.ru/utility.php?utility=cocomoii will help you to calculate an estimated cost for your project.
If they're non-technical, it won't matter. It will be like trying to sell a high-end bike to people who don't know a bike from a car. 15k lines of code won't matter to them any more than 300k lines of code will.
You need to find something other than the actual code to wow them with.
Can you code up some demos and tell them how short time it will take them to build similar applications with your code? Like "If you use my code, you can build this multimedia application in 15 minutes without writing more than a few lines of code". Non-technical people generally love saving time and money.
It probably depends on how "slightly" they are in the non-technical department.
An investor only cares about money. Investors start at the exit and work backwards. Knowing this, pitch your project in terms of the return they will get in their investment.
Key points would include:
Your expertise: Do you know the market you want to sell in to? Are you leveraging your expertise in some way to make the project a reality?
Risk: Using your already existing code base lowers risk in terms of both time and money. They will probably do technical due diligence to validate your claims, so be honest here.
Time to Market: Having a code base in place will reduce their time to market, which may be significant.
Vision: They need to know that there is a future for your product. This is your chance to get them excited!
Investment is about the future, not the past, so understand that you need to achieve what you are promising. The path you trod to get to where you are now may be interesting, but largely irrelevant to the investor. What I'm trying to say is sell the vision, not where you are now or where you've been.
Good luck and hope you get what you need!
It's not clear to me from your question whether you're talking about people who would buy the use of your product or ownership of your product.
In either case, ask yourself these questions:
"What problem(s) does this product solve for my users, from their point of view?"
"What does this product let the users do, that they already want to do, but can't do without it?"
"What does this product let the users do, that they already want to do, but can't do as easily without it?"
Features don't matter. Menus and dialogs don't matter (unless they require explanation, in which case they matter in the negative sense).
If you want numbers that interest a potential buyer of (an instance of) the product, talk in terms of how much time or money the buyer can save by using your product.
If you want numbers that interest a potential buyer of shares in your company or product, talk in terms of the size of the market, how you've analyzed that market's needs, and the ROI of any investment.
I've had success showing potential customers our automated build cycle, in slideshow form. I took them through our "production line" as if it was a factory tour, and showed the nice colored bars of coverage reports, uptilted lines of historical lines of code, pie charts of breakdowns of lines of code per module.
Then I did the same for everything aroung the actual building. So there's a requirements pipeline where they are involved, and a test/validation cycle where they are again involved.
It may not mean anything to them, but it shows them you have control over your process, and control over the quality of the delivered end product.
Please note that although people may be non-technical, try to be as honest as possible. As soon as they discover one single tiny lie in your story, you're lost. And chances are that there's that one technical guy in the back who can ask that one question which makes your house of cards fall down.
Happy sales!
"good code" doesn't matter unless you are demonstrating the medium and long term advantages of it - enhanced flexibility, simplicity, which saves customer time/money while adding agility.
I think explaining the more complex aspects of the code and the work that went into it to any audience will help show how much work and effort have gone into a project.
Hours spent coding could be a good metric to give them.
Talk about the features. Explain what you have working or almost working. Go at it from what they are interested in.
Try to show them visuals that they care about if you can. I think a few minutes doodling on a board would be better than showing lines of code.
The only thing that is likely to matter to a buyer (particularly a non-technical one)is functionality. I would concentrate on selling the features. You might consider discussion how you have tested it to verify that it performs as you claim.
I wouldn't use code per se, since a non-techie wouldn't understand it. Boasting about quantity is probably meaningless (how does a non-techie know that a 1MLOC project is significant? As for quality, you can present, e.g., maintainability metrics, test coverage, things like that. Feel free to show off your excellent toolchain too (continuous integration and all that), your mastery of various performance-testing tools. Also, showing things like Workflow Foundation helps - customers like to see how their business processes can be turned directly into code with a diagram notation.
EDIT modified to reflect OP's clarification (in comment here) that these potential buyers are looking to re-sell the software
Re-sellers are going to be looking for three things:
Is anyone going to make something better, cheaper or more quickly?
Is this guy going to be able to use our investment effectively to produce more?
Can we sell what this guy has produced, and will produce?
How to address points 1 and 2 have been very well addressed in other answers, but it's question 3 which is the hardest to prove for us techie people. It's also extremely important - if you can go to these buyers and hand them 3 killer benefits which they can repeat with more flair and Powerpoint when they're doing their sales calls, you'll be off to a good start :)
The main thing you have to do is to take a step back from your work and look at the:
features: what does it do
advantages: why is it better
benefits: why should the customer care
Features are closest to what you care about as a developer, but are pretty much irrelevant to non-technical buyers. Advantages are an essential step in understanding your competition and the customers' alternatives.
By putting features and advantages together, you can hit the customer with a number of benefits, e.g.:
using my software will save you $0.01 per transaction, or $40,000 p.a.
my software will increase customer retention by 5%
your admins will need 15% less time to deploy changes using my software
These are the things that customers care about: what's going to be good for the company, and good for them.
To be brutally honest: the end customer don't care how much effort you put into it (LoC or any other metric), they don't care how well it's written (comments, tests, any other metric), they don't care how hard a problem it is to solve, they don't care about features.
Their only requirement is that it will save them time / effort / money. You know that how hard you've worked to solve the problem, and solve it well, is key to their requirement, but it's secondary. You need to make it perfectly clear why them buying their stuff will mean they'll get promoted.
For COCOMO - Project Cost Estimation
I found this website, it's kind of a manual process but it'll do.
http://www.cms4site.ru/utility.php?utility=cocomoii
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have a developer on my staff that chronically overshoots deadlines, and estimates. On several projects the last week or two everyday I hear "It should be done by the end of the day". This developer does good work.
I have already spoke to him about his problems. He seems genuinely frustrated, and miffed about what to do to correct them.
My Questions are:
What kinds of punishments for passing a deadline are effective?
What ways can I coerce this employee to police his actions (time estimates, etc.,) himself?
UPDATE:
Based on the responses; here's what I have figured out.
Punishment is a bad idea.
It is natural for an employee to be unable to fix estimating problems without intervention.
Don't make deadlines unless there's company consequences (lost contract) for not being done by then.
Utilize available methods (Agile, Joel's checklist) to help the developer estimate better.
Thanks for the links and information. Also thanks for updating my thinking.
I don't think the problem is that he is missing these deadlines.
I think he has a real problem in estimating the amount of time it will take to complete a task.
Have him start keeping a journal of what he says a task will take and how long it actually took him to complete the task. Eventually, this journal will become a sort of guide for him to create better estimates. Once he becomes better at estimating, he shouldn't feel as rushed or harried.
There is an interesting article by Joel Spolsky: Evidence Based Scheduling
1) Break ‘er down
When I see a schedule measured in days, or even weeks, I know it’s not going to work. You have to break your schedule into very small tasks that can be measured in hours. Nothing longer than 16 hours.
This forces you to actually figure out what you are going to do. Write subroutine foo. Create this dialog box. Parse the Fizzbott file. Individual development tasks are easy to estimate, because you’ve written subroutines, created dialogs, and parsed files before.
If you are sloppy, and pick big three-week tasks (e.g., “Implement Ajax photo editor”), then you haven’t thought about what you are going to do. In detail. Step by step. And when you haven’t thought about what you’re going to do, you can’t know how long it will take.
Setting a 16-hour maximum forces you to design the damn feature. If you have a hand-wavy three week feature called “Ajax photo editor” without a detailed design, I’m sorry to be the one to break it to you but you are officially doomed. You never thought about the steps it’s going to take and you’re sure to be forgetting a lot of them.
The main point is that he (and you) should learn from his mistakes, and take them into account on the next estimation.
Also, if you are a developer, I would do regular code review at the end of the day to get a better insight into his development process.
And, of course, smaller iterations and more granularity with tasks. Set the maximum task duration to 1 day. That's the rule we have.
If your first question is
what kind of punishments to be considering
I think you're on a loser straight off. If you feel he does good work you may have to look at the deadlines/estimates and see if they were realistic in the first place. Who set them, if the developer in question was not involved then that may be part of the problem.
I agree with #OTisler that pair programming and possibly a regular end of day progress review with yourself can help him through... although if the deadlines/estimates were unrealistic to begin with thats not where your problem lies.
Closer monitoring on a few specific tasks should highlight where any issues lie.
What kinds of punishments for passing
a deadline are effective?
None. If you anger him, he won't do the work, or he'll find another job. You should help him figure out why his estimates are off. There is a book by steve McConnell about making estimates. i would start there.
What ways can I coherence this
employee to police his actions(time
estimates, etc.,) himself?
By helping him find the right way to make estimates.
First, make sure you are crystal clear in your requirements.
I hate to say it, but in my experience, blown deadlines are just as often a matter of unclear requirements or weak specifications on the part of a supervisor. First thing to do is to make sure the problem isn't either originating with, or exacerbated by, you.
Also, make sure your requirements are realistic, as well as his estimates.
Make sure that your own expectations aren't pushing him to make unrealistic estimates in order to meet unrealistic requirements.
Remember, you do the requirements, but the developer ALWAYS does the estimates, and should not be swayed with "can we do this any faster" unless you are also specifying functionality to be dropped.
Then, make sure he is tracking his time/tasks accurately, so you can get a good view of what is going on with the project.
This process will show any lack of proper time/task tracking, which may end up being the first step to improvement. If you can't see after the project how long a particular item took, that is probably the cause of the problem right there - not enough definition in the estimate, or missing "dependency" tasks that are discovered mid-project, but never estimated.
You HAVE to know how much time was spent doing what, accurately, before you can find out where the creep was, or what can be done about it.
Then, see where his estimates are failing and figure out why. Go over an estimate of a blown project, make that into a project itself - a problem to be solved.
Once you've determined that his estimates are indeed the source of the problem, go over an estimate that went over with him, and perhaps another developer, and figure out why.
This will help you figure out what the cause of the problem is. A solid understanding of the problem will likely be the actual solution.
Lastly, if you actually reach a point where you have to try punishment or coercion, it's time to fire him and start over.
Punishment and Coercion are appropriate responses to willful wrongdoing in certain situations.
However, if this developer is actively trying to do a good job, then you would only worsen the situation by generating negative attitude and frustration.
If the problem can't be solved, and you are sure the problem is with him, and not you, then it's time to fire him and get a developer who can meet deadlines. Great work doesn't mean much when your costs are blown up and profit goes out the window.
Okay, this is fairly common--developers being optimistic. It's the job of Management to deal with it. If anyone should be punished, it's the manager (you?)
I'm glad you at least asked, It looks like you got some good answers off this list, I hope they help and you find a way to actually implement some that work.
When I was young, my first good manager dealt with it this way:
First of all, he had me come up with an itemized list--breaking tasks down to hours, and estimating each one with a very liberal estimate--no period should be less than 4 hours regardless of how small the task was.
Then he looked at them and told me to double all my estimates. (Developers, especially younger developers, don't think about the fact that you are only productive for about 1/2 the day, if you're lucky--and half of that is spent at things you didn't expect to have to do).
Then, before creating his schedule, he doubled all my estimates (Without telling me).
He turned them in this way regardless of schedule requirements from above. A good manager should realize that saying it needs to be done in 2 days, doesn't make it possible.
As I got better at estimating we both noticed and adjusted accordingly.
A managers job isn't just to make a project, it's to build a team. More often than not that's going to require training of some sort. This is also the reason that an engineering manager that is not an engineer is unacceptable, they can't really help with this kind of thing.
Failure of a project or schedule is VIRTUALLY NEVER the fault of the developer (except in a few chronic cases where he isn't really fixable or of any worth and needs to be fired). The manager has made bad decisions either in hiring the developer, trusting him, managing him or staffing the project.
And really, what is fault anyway? I suppose if the manager isn't very good at making the project happen, he's going to need someone to point at... If HIS manager is any good, he'll ask why it got this far, what you did to fix it, etc.
Hiring a manager is hiring someone to solve the problems. To make the developers productive. If he can't make them productive, he isn't the right person.
To your questions:
If you choose to punish people for missing deadlines you will not get good results. They will be demotivated and feel belittled. If you keep pushing people to meet deadlines the quality of work will suffer and you will end up with a lot of time spent bug fixing afterwards.
To improve his time estimates you could try using Joel Spolsky's evidence based scheduling which has a nice feedback loop to improve the resulting estimates.
But I have some questions that I think you need to think about.
Is he later than everybody else? If so why - is it because he is an over optimistic estimator or a slow worker? Over optimistic estimates are easy to fix - just multiply all his numbers by a factor as per evidence based scheduling above. If he is a slow worker why? Does he get distracted? Is he very careful to produce very low defect code? Is he over engineering solutions? Is he not re-using code effectively?
Do the deadlines matter, or are they just arbitrary dates based on the estimates for the purposes of reporting progress up the management hierarchy? If the latter you can solve this by tweaking his estimates yourself.
What kinds of punishments for passing
a deadline are effective?
You stated the point and missed it. The obvious punishment for passing a deadline is death. If the developer is still alive after passing a deadline the "deadline" obviously was not a real deadline. Do you think it's funny to put developers under pressure using martial language?
Fix your wording.
Motivation
First of all: Read Peopleware
Next. Why do you think punishment will be an effective way to manage people that is supposed to be creative? I think you have to rethink the whole approach to management vs. team.
As I see it the managers first, and most important, role is to make sure that the developers can be creative and productive. Not that they are productive. There is a big difference in those small words. To be creative you need a safe environment. By being constantly under pressure from both deadlines and threats of punishment you create the exact opposite of safe.
Also, as a manager, you need accurate information on which to base decisions. This also requires a safe environment. If there is a risk for punishment for being honest and outspoken you are guaranteed to get lies and absence of information. A very dangerous base to take decisions from.
Estimates
As other as pointed out, estimates are estimates. In our team we don't do any individual estimates at all, we do estimates as a team. (I'm a bit reluctant to call what we do Scrum, but most of it tries to emulate if nothing less) I think this is a really great way to do estimates: Each team member is given a deck of cards consisting of numbers 0,1/2,1,3,5,8,13,20,40,60,100 and when estimating a task each developer picks a card (the cards are hidden until everyone has picked a card to avoid influencing estimates) and the average of the selected cards is taken as the estimate.
Notice how the numbers gets progressively less accurate. This is by design because large estimates are by necessity less accurate.
For our team we have opted to use the unit "ideal man days" for estimates. As far back as any of us can remember an ideal day hasn't occurred yet, but it is a good basis when you know how to translate calender days to "ideal man days".
As Scrum prescribes, development is done in sprints of two weeks after which the new version is deployed in the production environment. After each sprint we take the sum of the estimates of the completed tasks and divide that by the planned man days for the sprint. This factor is then the basis of estimating how many "ideal man days" the team can spend in a two week period.
Actual work items done by an individual developer don't need an estimate. The first approximation is always 1/2 - 1 day to complete. If this estimate turns out to be false you just grab a fellow developer and do it together to get it done. Or you break down the work item in smaller tasks so it can be distributed better.
Set Milestones and try Agile as #OTisler suggested.
I don't think you should punish him. Just get him to understand how to make accurate estimates.
As a team lead I've had my team members tell me that it will be "no problem" to finish X feature by the deadline. Then I usually sit down with them and go over what tasks and sub-tasks I think need to be done in order for the feature to be finished, and how long the developer thinks each will take.
After we do this exercise, and add up all the task and sub-task estimates, it will inevitably take much longer than the developer thinks in their original estimate. I usually only have to do this exercise with them a few times before they start making more accurate estimates.
What amazes me is that you only have one of these guys.
Engineers are horrible at estimating how much time something will take. I bet if you look carefully at your other developers' estimates, you'll find a lot of padding. Sometimes the padding isn't necessary, but the task expands to fill the available time anyway.
The solution to this is to change around how you do estimates - for everyone. Developers may be bad at estimating absolute time, but they're pretty good at relative time. So on Monday, instead of "how long will it take to add a whoosiwhatsit?," ask "what can you get done on the whoosiwhatsit in less than a week?" That becomes their task for the week.
The following Monday you look at how it went. "Well, I got the floogle installed in two days but it turns out it impacted the mcphee...so this week I need to decouple those guys so the whoosiwhatsit files don't get overwritten." Ok, there's their task for the week.
You might think it won't help, because you still don't know when the whoosiwhatsit is going to be ready. That's true. You have two choices here:
If you need a deadline, then you have to force your errant developer to pad his estimates like everyone else. It won't take him long to get the hang of it, and in no time at all he'll be taking "2 weeks" to write something that should have taken a day.
Your other choice is to trade the fictitious estimates for more visibility. In the long run this approach gets you more productive and much happier engineers.
So the developer does good work, but is poor at estimating the amount of time for delivery? I'm not sure you have a punishment situation on your hands just yet.
Maybe going forward for some time, have him walk you through his process for estimating a delivery point. This can be an opportunity to ask him why steps X,Y, and Z take certain amounts of time. He may find himself revising his estimates simply by doing the exercise at what is almost certainly a slower pace.
ask yourself this: What entails your job?
If you're just blindly passing estimates from developers (who you know can't give good estimates) up the management line, and not deciding for yourself whether that estimate is achievable, then you're not doing your job.
Try to think in terms of "value-add" (One of my old employers used that term a lot , and I hated it, but it probably works for you in this situation). What value are you adding? If you're just passing stuff in both directions between upper management and the developers, then ultimately you're not earning your money. You could be removed , and nothing would change.
The best manager I ever had was one that looked through a set of requirements given to him by another team , and told them straight out that almost a third of them was bull, and had them removed, before I ever even saw the list. The worst one I ever had made me write all this extra management-type documentation which none of the other managers I'd ever had asked me to do (I really got the impression I was literally doing his job for him), didn't even give me project due dates, and hardly turned up to work. They were both in the same company , bizarrely enough.
90 hours is one common short project deadline. The easy way is instead of estimating "your time", you measure another. Computer programmers shoudn't make time estimates for their projects since evidence shows calculating one's own time results in larger error than observing another.