Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
When and who undertakes the work to sufficiently gather answers so that we can start to write stories for an upcoming sprint. Is this work done continuously and in parallel to existing sprints by the product owner? I guess this then creates tasks for a sprint such as investigate x and y. What if the PO suddenly requires a developer to answer some of the questions by trying stuff out? I understand the idea of spiking and creating r & d tasks. I guess I want to avoid the main dev of a feature being delayed to a following sprint too often.
The team determines how much new story work it can do during a sprint. The amount of time they have to do that work is some percentage of the work day. Depending on the responsibilities of team members (customer support, bug fixes, emails, PTO, other duties) that amount varies from team to team. I like to see 10-15% of the work day dedicated to "planning" for the next sprint. That includes helping the PO research, writing stories, breaking up stories, design sessions, what-if scenarios, etc. I think the key is not to shoe-horn every one of these types of tasks into a sprint but rather to set the correct time allocation to doing the sprint work. Maybe something like 30 hours/wk is an average number.
So to directly answer your question; the planning work is done in parallel to the current sprint work.
We usually have one or two meetings to talk about future stories. Also, we reserve some overhead time in each sprint to check out things we need to know to start a story. The meetings help determining which stories will probably shop up in the next sprint, so we know which questions to get answers to during the reserved time in the current sprint.
For us, if it's a large project, we will have kickoff meetings to brainstorm the project. There is often a knowledge gap for PO's between what they want to do and what they don't know we can do that these meetings can fill.
When new stories are created, we try to assign story points to them at some point before the next planning meeting so the PO has time to prioritize the list before that meeting.
I'm not sure of the kind of situation you describe where a PO would "suddenly" need a dev to try stuff out. In that case, I would offer a spike in the next sprint. Generally using new technologies isn't something that happens every sprint so this should suffice. If not, perhaps the sprints are a bit too long for this purpose (a trade off to be considered at least) Another alternative would be to introduce an evergreen story for trying stuff out. I've seen teams have these kinds of stories for tech debt payback - you could off an either/or situation. Sometimes dev fixes tech debt, sometimes they try stuff out. And if you run out of tech debt somehow, you can always grab another regular story to put in its place.
We typically reserve a sprint or two after a big release for research and proof of concept stories. Doing research as part of the regular sprint seems like it would be problematic. You'd probably use that time to absorb mis-estimations for value-adding stories and end up never using it for actual research.
If a new story drops into the backlog that needs research and the PO runs it up to the top of your backlog then the team should include some research time into their actual estimate. I would only do that if I didn't have the luxury of a research/prototyping sprint ahead of time though since estimating research can be a bit nebulous.
Who: Product owner. Stories and Product backlog are his responsibilities. Product owners are generally experienced people; even if they are not technical they can certainly perceive implementation complexities at abstract level. Still, if a story has gray area PO must ask right people the right question. He can ask developers, testers, peers, clients and even scrum masters.
When: all the time.. Continuously. PO must not do anything but (1) provide (or get) answers for the team’s questions regarding scope and function, (2) and gather data that would refine the stories and their scope: thus proactively solving the queries of his team.
Bottom line is if product owner is not giving good stories to the team then he is not doing his job. Stories can be written by anyone but in the end it’s PO who ensure that Product Backlog is in order and that top priority stories are defined.
Related
Closed. This question is not about programming or software development. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed 11 hours ago.
Improve this question
Sprint planning has completed, sprint goals set, all is well. 2 days in the PO asks the lead developer if they could add a new item into the sprint, "it's an easy fix" "another team wants this".
What should the developer and the Scrum Master do? What is the conversation?
The Scrum Master may have a conversation with the Product Owner to understand the workflow on how an item comes into a Sprint backlog. If the Product Owner understands the workflow eg: Create item, Item satisfies "Definition of Ready" (User story template, estimate...), Item lands in Backlog, Item may be pulled in Sprint Backlog if there is free capacity and it does not affect any Sprint Goal.
The Scrum Master can plan ahead for this in multiple ways:
Plan less story points to the sprints in case this happens again, leaving room to add some extra ad-hocs tickets when they come
Take out a ticket from the current sprint with the same amount of story points that the ticket the PO is trying to add
Help the POs reflect and ask them: "what happen if we don't do this ad-hoc right now?" "Adding this ticket to the on going sprint will aid or delay the progress towards the sprint goal?"
Alternatively, manage expectations by creating a list together with the lead dev and the PO with a set of rules about when it is ok to add a ticket to an on going sprint, so everybody is on the same page
Scrum has always accounted for this situation from the very beginning. The simple answer is that the Sprint Backlog is "locked down" at the start of the Sprint and cannot be changed. This, "Oh, it's just a quick fix." situation was exactly described by Ken in the original book as a prime source of distractions that get in the way of the Team from succeeding.
There's only one recourse, which is a "premature termination of the Sprint". The Sprint ends immediately, nothing is delivered and a new Sprint is started with a new Sprint Backlog and a new end date.
It sounds harsh, but the truth is that the organization won't respect the Team's time unless you say "No" in these situations.
Another aspect to this is that someone early on, I cannot remember who, said the Sprint length should be set to the length of time that the organization go without changing its mind. So if this is a real problem, then shorten up the Sprints.
Back at the beginning, most teams used 4 week Sprints, and this is now seen as way too long. If you can swing it, a 1 week Sprint can be way better. In the example given in the question, 2 days into the Sprint is 3 days away from the end of a 1 week Sprint, and waiting for the next Sprint for the "quick win" is probably going to be fine.
The answer to most Scrum problems is often to make your stuff smaller.
What should the SM do? Well... nothing.
If the SM has coached the team properly, and they understand that they have committed to the Sprint Goal and understand it cannot be endangered, then you can just assume that the person being asked to do the work feels confident that the extra task and the Sprint Goal can be met.
If the extra task is competed and the Sprint Goal is failed, then that's a learning exercise for the Retro - Does everyone in the team understand that the Sprint Goal is a commitment and that it shouldn't be endangered? You could also cover techniques for helping the team handle such situations.
There's nothing wrong with the team doing extra tasks - as long as the Sprint Goal is met. This covers the self-management part of Scrum. The team decide for themselves if they are capable enough to do both activities in the Sprint timebox, and if not, they need to be coached to push back.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
So we start Scrum today and start going over story points estimates.
The first story that comes up is a new screen that needs to be developed. It has 1 sentence to describe the screen and 3 user acceptance tests.
This starts a fight between the development team and the product owner.
Product owner says that stories do not need to be speced out and they will just be fleshed out during the sprint.
We say that the story needs to be completely speced out for the sprint.
But now I am starting to be unsure about who is right....
Any good articles on this that I can send to the team about how defined a user story has to be?
What happened during sprint planning?
It appears that you did not review the sprint plan to see the stories in advance of starting the sprint.
That's okay.
Stories are fleshed out during the sprint. That's the point. Relax.
Flesh out the story quickly, build quickly.
At some point, the one sentence story may become rather complex. If that's the case, break it up into something you will finish during the sprint, and stuff you will not finish. It's okay to have some stuff that was not known and did not get built.
Relax.
Do not overspecify everything. Do not specify every nuance of the story before the sprint. Just build something that will work. As quickly as possible. That's why it's called a "sprint".
Don't build everything you imagine. Build enough that the story can be performed by the user.
The point is to build something that works on schedule. If you have to adjust the scope of the story, that's okay.
Any good articles on this that I can send to the team about how defined a user story has to be?
A story is typically made of one sentence based on the following template: In order to <benefits>, as a <role> I want <action> (and I like to add "how to demo" steps that help to understand the story and to build acceptance tests). The idea is to capture the essence, not the details. Details are captured using face to face conversation during the sprint (and may be added as high level notes to the story). But a user story is not a contract, it's a promise for a conversation (about the scenario for which the story is the title). If you need some guidelines, following the INVEST model has worked well for us.
PS: No offense but the development team seems to react very defensively (asking for full speced things sounds like "hey, we did it as it is written", i.e. CYA). A user story leaves some space for creativity. Isn't that nice? If you need more details, take your responsibilities, go gather them. And if for any reason you can't get required clarifications or details, raise an impediment and have your ScrumMaster work on it. Personally, I enjoy having some space for creativity.
IMHO fighting is not good - Product Owner, Scrum Master and Development Team form the Scrum Team so they need to work together. They want to achieve the same thing - building a great product.
To me the question is how important it is for the Product Owner how the end result looks like. If he says: "We hired the best people on the market, you're the experts, whatever you come up with is fine with me as long as the user need is fulfilled", then I'd fine with the PO statement. But of course he can not complain afterwards that he does not like the look or the colors!
Another point is that the team needs to be confident that they can commit to this story. Usually teams estimate story size with planing poker so if the development team can not estimate, you need to invest time before that you can estimate (e.g. talking about the story before, spiking and negotiate with the PO about the story). Sometimes the designer/UX guy needs to work ahead and create mockups for the upcoming user stories.
It's always about finding the balance between planning and doing :-)
fs
I believe Martin Fowler's blog post on Conversational Stories probably answers your question best. You really don't want to be in a situation where the Product Owner is required to spec out everything in detail. You've got a team of smart, creative people that are perfectly capable of making good implementation suggestions as well as asking the right questions as they come up during the sprint. You don't want to lose out on that creativity and input by locking down requirements up front.
The story should be clear enough that the team understands what the feature is and small enough that the team can complete it in one sprint before it is added to the sprint backlog. The rest of the details should be handled via conversations during the sprint.
In our practice we do internal investigation of tickets for the next spring before planing with stakeholders. We usually find a lot of questions to clarify. If we don't answers before sprint starts we can't estimate it. If we found new issues/question during sprint we inform stakeholders and usually such story will be transferred to the next ticket.
So, my answer will be: the story don't need to be completely speced out for the sprint. But team need to know all answer to questions required for implementation as well as business decisions.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
From wikipedia:
During each “sprint”, typically a two
to four week period (with the length
being decided by the team), the team
creates a potentially shippable
product increment (for example,
working and tested software). The set
of features that go into a sprint come
from the product “backlog,” which is a
prioritized set of high level
requirements of work to be done. Which
backlog items go into the sprint is
determined during the sprint planning
meeting. During this meeting, the
Product Owner informs the team of the
items in the product backlog that he
or she wants completed. The team then
determines how much of this they can
commit to complete during the next
sprint. During a sprint, no one is
allowed to change the sprint backlog,
which means that the requirements are
frozen for that sprint. After a sprint
is completed, the team demonstrates
the use of the software.
I was reading this and two questions immediately popped into my head:
1) If a sprint is only a couple of weeks long, decided in a single meeting, how can you accurately plan what can be achieved? High-level tasks can't be estimated accurately in my experience, and can easily double what seems reasonable. As a developer, I hate being pushed into committing what I can deliver in the next month based on a set of customer requirements. This goes against everything I know about generating reliable estimates rather than having to roughly estimate and then double it!
2) Since the requirements are supposed to be locked and a deliverable product be available at the end, what happens when something does take twice as long? What if this feature is only 1/2 done at the end of the sprint?
The wiki article goes on to talk about Sprint planning, where things are broken down into much smaller tasks for estimation (<1 day) but this is after the Sprint features are already planned and the release agreed, isn't it? Kind of like a salesman promising something without consulting the developers.
BTW:
Although the word is not an acronym,
some companies implementing the
process have been known to spell it
with capital letters as SCRUM. This
may be due to one of Ken Schwaber’s
early papers, which capitalized SCRUM
in the title.
You are supposed to use the velocity to plan the next sprint. The velocity neatly handles the fact that your estimates are wrong, but they are consistently wrong. Also note that stories are supposed to be short, I'd say maximum 2-3 days. Stories that are bigger than that should be broken down into smaller stories.
If one story is not completed as planned, then your velocity goes down and you wont be able to take on as much work in the next iteration.
The wiki article goes on to talk about Sprint planning, where things are broken down into much smaller tasks for estimation (<1 day) but this is after the Sprint features are already planned and the release agreed, isn't it?
Wrong, they are done in the same meeting. The sprint stories are not agreed upon until everyone leaves the sprint planning meeting. Whatever questions you need to ask the PO to enable your commitment to the stories; you do before or in the SP meeting
Since the requirements are supposed to be locked and a deliverable product available at the end, what happens when something does take twice as long? What if this feature is only 1/2 done at the end of the sprint
The functional objective of the story is locked, not the implementation details. The details come out in conversation during the sprint. Any details considered to large to be contained in the current sprint scope are put back on the Backlog for later prioritization. Remember, you are building incremental products here. Its like peeling an onion. The story must be satisfied and the code must be working at the end of the sprint. That doesn't mean the whole feature is entirely complete and releasable to a user.
If a sprint is only a couple of weeks, decided in a single meeting, how can you accurately plan what can be achieved? High-level tasks can't be estimated accurately in my experience, and can easily double what seems reasonable. As a developer, I hate being pushed into committing what I can deliver in the next month based on a set of customer requirements, this goes against everything I know about generating reliable estimates rather than having to roughly estimate and then double it!
You are correct here, you can't estimate accurately. Scrum embraces this fact and uses velocity, trending, averaging, and gut-feel to get close. If you don't come to grips with forgetting about accurate hour increment measurements you won't ever feel comfortable with scrum.
In answer to #2, if a feature isn't done at the end of the sprint, you don't deliver it. You may be able to deliver part of it, and if you can do so in a useful fashion, do so. But if you can't deliver it this sprint, remove it, and deliver it in the next sprint.
In answer to #1, there are numerous ways to try to improve the accuracy of your estimates. You might use function-point analysis or just a simple exercise where the entire project team takes the list of tasks separately and comes up with their own estimates for each task, then reviews each task and shares their estimates, and discusses the reasons why (for instance) Bob's estimate for this task is 8h and Tina's is 16h. The team figures out who is right (hopefully) or comes to consensus, and uses that as the estimate.
Over time, you'll come to learn which of your estimates tend to be overly optimistic, and which are overly pessimistic, and thereby improve your ability to estimate your own tasks.
The burn-down chart can really help you here. It is an early warning system for the whole project team, to let you all know when one or more people are falling behind. Then the team can reorganize to help make the sprint commitment if necessary, or cancel the sprint due to unforeseen circumstances, and kick off a new sprint with their improved understanding of the problem space.
Finally, you might consider that statistics are on your side. If you overestimate 10% of the tasks, and underestimate 10% of the tasks, you'll probably be OK.
When we did SCRUM in an earlier project, we first agreed on a rough sprint plan including high-level features (stories), then refined the plan by breaking each of these down to groups of concrete tasks of preferably 1 days max length, estimating each task. After this we often found out that the original plan was overcommitted to some extent (typically because we didn't take into account that developing a story includes unit testing, code review and documentation too), so we adjusted it accordingly. Btw we used "estimation poker": each member chose a card with a number on it (work hours/days) and everyone showed his card to the count of 3. If the numbers differed a lot, we briefly discussed why, and then had a new round until we reached near consensus.
Note also that estimation is very domain and technology dependent. In that project, we understood both fairly well, and we were building a new app from scratch, so our estimations were fairly accurate. In my current project we are working with legacy code, in a domain we don't quite understand yet, so our estimates are often wildly out of range.
As the project rolls on, estimates are gradually getting better (related to the fact that more and more risks and tricky issues are being resolved, and the team's domain expertise grows), so the velocity of the team can actually grow over time.
in answer to #1, I'm not sure I agree with some built in assumptions in the question. In my experience, Agile (including Scrum) is not about time estimates. The whole idea is to move away from that and instead move to a system where you have a known velocity and specific sprint times. For instance, you release every 2 weeks (a good sprint time) with some new code. You see how many story points (not time units, but story points) you get done over a sprint, and then another, and after you've done a few sprints you know your rough velocity (ie, how many story points you can do on average per sprint).
The idea is that the customer gets continuous updates to the application as each sprint is finished and can see constant progress. They know which items are scheduled to come in future sprints but they are aware that if something slips (because of an incorrect difficulty rating, ie. the story point estimation, or an outside problem) it will instead come in the next sprint and everything else will be moved out a little.
So its not about developing the software based on some seemingly arbitrary estimation. Instead, its about planning what functionality or features you want and assigning difficulty (story points) to those features (relative to the other features) and working through them to determine a velocity. Only then can rough estimates be obtained. Once the average velocity is known, we can make some rough guesses about time frames. However, even these should be considered rough approximations because again, its not about time, its about constant feature releases. Clearly, this mindset must exists with EVERYONE on the team, not just the engineers.
Here is a link (wikipedia) that goes into it a bit more.
Anyway, I hope this helps you, good luck!
It's actually quite simple:
You priorize the Tasks and if you see that you don't have enough time then the low-priority tasks simply get dropped or moved into the next sprint.
The Project Owner decides what he wants and sets the priorities and you develop following that order. You should have a useable product at the end of the sprint, not the fully-featured product.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
I am the scrum master for a small team of 4 developers. The project we are working on has a lot of tasks that we have never done before and cannot easily estimate in a sprint planning meeting. What is the best way for me to run a sprint with this uncertainty? I am finding it very hard to finish a sprint with a potentially releasable product. I am also finding it hard to plan sprints when there is a lot of unknown length tasks.
I'm not sure what the term is in Scrum, but in User Story terminology you would do a "spike", which is basically a very short period of research into the topic so that your team will be able to estimate the task at the end of the spike.
Example:
Story:
Analyst wants to be able to review
financial data in pie charts.
Your team doesn't use any charting tools, so you need to know how long it would take to build something like this. Or perhaps instead, you could invest in third party tooling and integrate a tooling set with your application.
You would do a spike to research these venues and come up with estimations on them, then decide which route to take.
Are the "tasks" things that someone in the world has done before, or are they just new to your team. I will assume the later. If this is the case then what you are finding is that you do not have the necessary experience on your team to solve the problem. Thus you will be developing that experience as you go. All this means is that the complexity of your stories is higher. In the first couple of sprints you may score some of the stories as 13 and then later on they become 8s because you then have the knowledge you need.
You don't need to know how to do the stories to estimate them. You just need to take on less of them due to your experience gap.
I like to reserve "Spikes" (yes that is the term used in scrum) for attempting to solve business domain problems that have no known solution. Not for the team to do training.
If you really need to do research to get a good estimate, you could do the research as a task in itself, or set it aside and have it done (by someone) before the sprint planning.
Generally, I think that if you can't get a good estimate, you should either go with a bad estimate (i.e. a wild guess) or you should time-box the task, so you set aside a fixed amount of time for it in a sprint. After that, you will either have a done solution, or you will have a better understaning of it so you can estimate it or break it down into subtasks for the next sprint (or a later sprint).
Do you really mean tasks or are you talking about Product Backlog Items (PBIs)? Actually, I find it hard to believe that a task is not estimable. If they really aren't, they are very likely too big (tasks shouldn't exceed 16h, which is already huge).
If you are talking about PBIs, the situation you are describing is quite surprising and should theoretically not happen. In the worst case, just assign them a high number of story points, this precisely means that there is a lot of uncertainty on them. But, because PBIs that are ready for a Sprint shouldn't exceed the half of your velocity (or you'll put too much risk on your sprint), the obvious way to solve this situation is to divide such items into smaller chunks which may include exploration. But the important part is to keep things timeboxed, even (or especially) R&D. Keep in mind that with Scrum, everything is timeboxed.
In other words, to reduce uncertainty, break things down into smaller things (be them items or tasks)!
If the tasks seem un-estimable, I think the best approach would be to break down those tasks into smaller tasks that you can estimate. It might take several iterations, but you will probably come up with a pseudo design while you are at it. Joel mentions this in one of his articles.
Split the unestimatable task into a task to remove the uncertainty, and "the rest". Remove uncertainty with Proof-of-concept tests or spike solutions. Either schedules the spikes this sprint and the rest of the work next sprint, Or delay the start of the sprint for a week of spiking.
We often don't know enough to break a story down into tasks. We have a period of discovery before we know what the tasks will be. "Spikes" seem tricky to manage. For one, you may not be able to time box the discovery period. Second, I can't effectively plan a sprint without knowing how long a story will take.
Seems like another option is to do the spike in Sprint 1 and the tasks in Sprint 2. The downside is that it seems like the process forces an unnatural breakdown of the work. Why discover this week and then wait a while before starting the the work.
We use "contingents" or a specific backlog for such tasks.
The Scrum Tool Agilo is supporting this way of working and calculates those issues too, e.g. in the Burndown. In this way you get a good control on the "un-plannable" items.
Are you confusing precision with accuracy?
The idea behind Agile estimation is to come up with a number that is good enough, not a number that is exact. That is why using story points for backlog item estimation is a best practice; it emphasizes effort/complexity instead of duration.
You don't need to know how long each task necessary to implement a backlog item in a sprint will take. What you need to know is, given the work you've previously committed to in this sprint, can you commit to this backlog item? Because we know that we can't know exactly how much time each backlog item will take, we have to make an educated guess.
More important, what does it mean to fail in Scrum? Is not getting every sprint backlog item completed a failure? No... if you got four out of five items done and the fifth one is mostly done, you'll get credit for the four completed items (in terms of velocity for the sprint), and when you finish the remaining tasks for that fifth item in a future sprint you will get full credit for that item. But, would you have gotten any more done if you weren't using Scrum? The only failure in Scrum is failing to learn from your mistakes, to keep doing the same dysfunctional things repeatedly while expecting different results.
So, in your sprint planning meeting, don't spend a lot of time worrying about something you're not going to be able to know. Let the team think about the work, and then let them sign up for the amount of work they feel comfortable they can complete during the sprint. If they undercommit you can always drag something into the backlog, or end the sprint early. If they overcommit, then you finish the backlog items you can in priority order and discuss why the unfinished items couldn't be finished in the sprint retrospective, along with how to prevent having unfinished items in future sprints.
By the way, I know this was probably a poor choice of words on your part, but an effective Scrum Master doesn't run the sprint. The team runs the sprint, and the Scrum Master actively looks for impediments that lower their productivity and interfere with their ability to meet their commitments. Scrum Masters aren't managers, they're a combination of referee, coach, and scorekeeper. They are the Keeper of the Process, they help the team follow the process, they protect the team from outside agents that try to go around the process, and they track progress during the sprint via ensuring the sprint backlog is updated and the sprint burndown chart reflects reality, on a daily basis. In the situation you've described, where the team isn't sure how much work they should sign up for, the Scrum Master should let the team decide as a reflection of respect for team ownership of the commitment. Whatever the decision is, it won't be wrong.
Spikes should be time boxed. It puts on pressure on the team to limit the scope and have a better idea on the cost-benefits that the research will entail; ie it is useless to carry out 3 days of research for a task which would cost a few bucks.
This is also backed up by Latham's work on Goal Setting Theory where he specifically tackles this issue.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
We use Scrum. We are experiencing problems during sprints when we find the user stories are not sufficiently granular to capture the effort required to complete the sprint.
In particular, we find that we are supplied with UI wireframes that contain much more complexity than the original stories would have implied (e.g. functionality duplicated for usability reasons). This leads to the burndown chart looking like everything is completed on the final day of the sprint.
We spend the Monday at the start of each 2-week sprint going over the stories as created by the project team, during which time we typically refine the stories a little and break them down into tasks, estimating the hours for each to create the burndown chart. During this day it doesn't feel like we have time to meaningfully improve the quality of the stories.
How best to break the cycle of incomplete / insufficient stories for our sprints?
Is this a failure of the project team to nail down the stories sufficiently at the outset, or should we (i.e. the dev team) take some of the responsibility?
So are you saying that:
Customers/users talk to project team
Project team writes stories and creates wireframes
Development team breaks down stories into tasks and estimates
Is there a possibility of the development team actually talking to the customers/users? User stories are sometimes seen as a way to kick off a conversation, as opposed to requirement documents/specifications.
EDIT: Some links:
A user story is to a use case as a gazelle is to a gazebo
Six Features of a Good User Story - INVEST Model
The Customer is Always Available
EDIT: Martin Fowler made a blog post yesterday on ConversationalStories that covers this far better than I did.
Are you running sprint retrospectives? At the end of the retrospective you should have high priority actionable items to improve on what happened in the previous sprint. The same things shouldnt be going wrong repeatedly.
Is your product owner accessible during a sprint? If not you may need to add extra to any estimation as the detail of a user story is incomplete.
#Pascal suggestion to dedicate 5% of your sprint to product backlog grooming is a good one. This should enable the user stories to be in a more detailed place before the start of your sprint.
We spend the Monday at the start of
each 2-week sprint going over the
stories as created by the project
team, during which time we typically
refine the stories a little and break
them down into tasks, estimating the
hours for each to create the burndown
chart. During this day it doesn't feel
like we have time to meaningfully
improve the quality of the stories.
It sounds like this is your sprint planning session, do you have control over what user stories you are commiting to complete during the sprint? How can you commit if you dont have sufficient detail?
This takes you back to having a good retrospective and solving the issues raised.
How best to break the cycle of
incomplete / insufficient stories for
our sprints?
Retrospectives, planning, backlog grooming.
Is this a failure of the project team
to nail down the stories sufficiently
at the outset, or should we (i.e. the
dev team) take some of the
responsibility?
Its the responsibility of the team as a whole. Finding blame isnt going to give value, but everyone taking responsibility will give everyone a chance of completing the project succesfully.
Maybe during those Monday morning planning sessions you can involve whoever is writing the user stories / wireframes and work together to find out what detail is missing from them, what detail would make your estimations easier and more accurate. Maybe a template of what they should include could be drawn up.
We had (and continue to in some respects) this same problem. I think this problem is a lacking of upfront analysis and a lack of devs spending enough time estimating a user story.
You might start with a story like:
As an administrative user I can create a new widget.
OK, what does that mean? After some analysis, it might mean:
As an administrative user I can create a new widget in created status with complex data validation errors.
So after that a listing of fields, how big, and what the required fields are for saving to the database. A basic UI mock up would be nice as well.
Another user story for the next sprint might be:
As an administrative user I can edit a created widget and correct the complex data validation issue to move the widget to completed status.
Then list of the complex validation rules.
We spend the Monday at the start of each 2-week sprint going over the stories as created by the project team, during which time we typically refine the stories a little.
At the start of the sprint, the stories should be READY. If you need to refine them a bit, I think that you (the dev team, the ScrumMaster, the project team) should do that a bit ahead, during the previous sprint.
How best to break the cycle of incomplete / insufficient stories for our sprints?
You are maybe underestimating stories or they are too big and too vague. In both case, this sounds like an estimation problem and a good way to improve is to reduce the size of stories. To work on this issue, you could dedicate some time (e.g. 5% of every sprint) to Product Backlog Grooming in order to prepare the most important stories and reduce their size by putting them on diet if required before the next sprint. And this will actually make the sprint planning meeting smoother.
Is this a failure of the project team to nail down the stories sufficiently at the outset, or should we (i.e. the dev team) take some of the responsibility?
The responsibility isn't that important IMHO (except for political reasons maybe but they do not produce much value anyway), both the dev team and the project team are working together and "failing" together. What is important here is to inspect and adapt to remove the obstacle. So, it's the dev team responsibility to make this problem visible (it is an impediment). And it's the ScrumMaster responsibility to work on this impediment. The failure would be to not work on it. Backlog Grooming sessions are one way to do it. And at the end, I'm sure the project team will improve and get a better understanding of what the dev team is expecting. And you will both produce better results.
Lots of good ideas here already on the scrum aspects of your problem. Based on your comment:
particular, we find that we are supplied with UI wireframes that contain much more complexity than the original stories would have implied (e.g. functionality duplicated for usability reasons).
I also have a concern that you might need to work on you development process as well though. Accessing functionality from multiple locations in a UI should be a simple addition that consumes almost no time at all. If you are finding this to be a common problem then your functionality is too tightly coupled to particular UI elements. Your team might need to improve their design skills (eg: pattern usage).
This is interesting. It would appear that you are doing the sprint planning in the sprint? And that the Sprint Backlog is committed before the Sprint Planning? If so, how are the team commiting to the sprint backlog without discussing the details of the stories?
An alternative approach could be to make the Product Owner aware that certain stories cannot be added to the sprint backlog due to lack of clarity. In particular that the acceptance criteria are not fully understood. This could provoke the necessary conversation with the Product Owner. Ideally it shouldn't come to this. It should be discussed and resolved in the retrospective.