Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
From wikipedia:
During each “sprint”, typically a two
to four week period (with the length
being decided by the team), the team
creates a potentially shippable
product increment (for example,
working and tested software). The set
of features that go into a sprint come
from the product “backlog,” which is a
prioritized set of high level
requirements of work to be done. Which
backlog items go into the sprint is
determined during the sprint planning
meeting. During this meeting, the
Product Owner informs the team of the
items in the product backlog that he
or she wants completed. The team then
determines how much of this they can
commit to complete during the next
sprint. During a sprint, no one is
allowed to change the sprint backlog,
which means that the requirements are
frozen for that sprint. After a sprint
is completed, the team demonstrates
the use of the software.
I was reading this and two questions immediately popped into my head:
1) If a sprint is only a couple of weeks long, decided in a single meeting, how can you accurately plan what can be achieved? High-level tasks can't be estimated accurately in my experience, and can easily double what seems reasonable. As a developer, I hate being pushed into committing what I can deliver in the next month based on a set of customer requirements. This goes against everything I know about generating reliable estimates rather than having to roughly estimate and then double it!
2) Since the requirements are supposed to be locked and a deliverable product be available at the end, what happens when something does take twice as long? What if this feature is only 1/2 done at the end of the sprint?
The wiki article goes on to talk about Sprint planning, where things are broken down into much smaller tasks for estimation (<1 day) but this is after the Sprint features are already planned and the release agreed, isn't it? Kind of like a salesman promising something without consulting the developers.
BTW:
Although the word is not an acronym,
some companies implementing the
process have been known to spell it
with capital letters as SCRUM. This
may be due to one of Ken Schwaber’s
early papers, which capitalized SCRUM
in the title.
You are supposed to use the velocity to plan the next sprint. The velocity neatly handles the fact that your estimates are wrong, but they are consistently wrong. Also note that stories are supposed to be short, I'd say maximum 2-3 days. Stories that are bigger than that should be broken down into smaller stories.
If one story is not completed as planned, then your velocity goes down and you wont be able to take on as much work in the next iteration.
The wiki article goes on to talk about Sprint planning, where things are broken down into much smaller tasks for estimation (<1 day) but this is after the Sprint features are already planned and the release agreed, isn't it?
Wrong, they are done in the same meeting. The sprint stories are not agreed upon until everyone leaves the sprint planning meeting. Whatever questions you need to ask the PO to enable your commitment to the stories; you do before or in the SP meeting
Since the requirements are supposed to be locked and a deliverable product available at the end, what happens when something does take twice as long? What if this feature is only 1/2 done at the end of the sprint
The functional objective of the story is locked, not the implementation details. The details come out in conversation during the sprint. Any details considered to large to be contained in the current sprint scope are put back on the Backlog for later prioritization. Remember, you are building incremental products here. Its like peeling an onion. The story must be satisfied and the code must be working at the end of the sprint. That doesn't mean the whole feature is entirely complete and releasable to a user.
If a sprint is only a couple of weeks, decided in a single meeting, how can you accurately plan what can be achieved? High-level tasks can't be estimated accurately in my experience, and can easily double what seems reasonable. As a developer, I hate being pushed into committing what I can deliver in the next month based on a set of customer requirements, this goes against everything I know about generating reliable estimates rather than having to roughly estimate and then double it!
You are correct here, you can't estimate accurately. Scrum embraces this fact and uses velocity, trending, averaging, and gut-feel to get close. If you don't come to grips with forgetting about accurate hour increment measurements you won't ever feel comfortable with scrum.
In answer to #2, if a feature isn't done at the end of the sprint, you don't deliver it. You may be able to deliver part of it, and if you can do so in a useful fashion, do so. But if you can't deliver it this sprint, remove it, and deliver it in the next sprint.
In answer to #1, there are numerous ways to try to improve the accuracy of your estimates. You might use function-point analysis or just a simple exercise where the entire project team takes the list of tasks separately and comes up with their own estimates for each task, then reviews each task and shares their estimates, and discusses the reasons why (for instance) Bob's estimate for this task is 8h and Tina's is 16h. The team figures out who is right (hopefully) or comes to consensus, and uses that as the estimate.
Over time, you'll come to learn which of your estimates tend to be overly optimistic, and which are overly pessimistic, and thereby improve your ability to estimate your own tasks.
The burn-down chart can really help you here. It is an early warning system for the whole project team, to let you all know when one or more people are falling behind. Then the team can reorganize to help make the sprint commitment if necessary, or cancel the sprint due to unforeseen circumstances, and kick off a new sprint with their improved understanding of the problem space.
Finally, you might consider that statistics are on your side. If you overestimate 10% of the tasks, and underestimate 10% of the tasks, you'll probably be OK.
When we did SCRUM in an earlier project, we first agreed on a rough sprint plan including high-level features (stories), then refined the plan by breaking each of these down to groups of concrete tasks of preferably 1 days max length, estimating each task. After this we often found out that the original plan was overcommitted to some extent (typically because we didn't take into account that developing a story includes unit testing, code review and documentation too), so we adjusted it accordingly. Btw we used "estimation poker": each member chose a card with a number on it (work hours/days) and everyone showed his card to the count of 3. If the numbers differed a lot, we briefly discussed why, and then had a new round until we reached near consensus.
Note also that estimation is very domain and technology dependent. In that project, we understood both fairly well, and we were building a new app from scratch, so our estimations were fairly accurate. In my current project we are working with legacy code, in a domain we don't quite understand yet, so our estimates are often wildly out of range.
As the project rolls on, estimates are gradually getting better (related to the fact that more and more risks and tricky issues are being resolved, and the team's domain expertise grows), so the velocity of the team can actually grow over time.
in answer to #1, I'm not sure I agree with some built in assumptions in the question. In my experience, Agile (including Scrum) is not about time estimates. The whole idea is to move away from that and instead move to a system where you have a known velocity and specific sprint times. For instance, you release every 2 weeks (a good sprint time) with some new code. You see how many story points (not time units, but story points) you get done over a sprint, and then another, and after you've done a few sprints you know your rough velocity (ie, how many story points you can do on average per sprint).
The idea is that the customer gets continuous updates to the application as each sprint is finished and can see constant progress. They know which items are scheduled to come in future sprints but they are aware that if something slips (because of an incorrect difficulty rating, ie. the story point estimation, or an outside problem) it will instead come in the next sprint and everything else will be moved out a little.
So its not about developing the software based on some seemingly arbitrary estimation. Instead, its about planning what functionality or features you want and assigning difficulty (story points) to those features (relative to the other features) and working through them to determine a velocity. Only then can rough estimates be obtained. Once the average velocity is known, we can make some rough guesses about time frames. However, even these should be considered rough approximations because again, its not about time, its about constant feature releases. Clearly, this mindset must exists with EVERYONE on the team, not just the engineers.
Here is a link (wikipedia) that goes into it a bit more.
Anyway, I hope this helps you, good luck!
It's actually quite simple:
You priorize the Tasks and if you see that you don't have enough time then the low-priority tasks simply get dropped or moved into the next sprint.
The Project Owner decides what he wants and sets the priorities and you develop following that order. You should have a useable product at the end of the sprint, not the fully-featured product.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I am new to scrum , I would be happy if you tell me as a scrum master what should I do in a backlog grooming session and when sprint planning happens exactly and how we estimate the amount of time that we need for each item in the backlog and what planning poker estimates exactly.
It's definitely worth looking at the Scrum Guide for some guidance, though that certainly leaves a lot of room for interpretation. For sprint planning, there are three parts, noted here: https://scrumguides.org/scrum-guide.html#sprint-planning
You're going to want to establish a preliminary sprint goal (the guide notes that it doesn't have to be finalized until the end of planning), then identify which backlog items need to come in to meet that sprint goal, and finally, put together a preliminary plan on how the work will be done. Of course, this is all collaborative. Generally speaking, the PO focuses on the most important work being done first to deliver the most value and the developers focus on what's a reasonable workload and how doing different tasks together may be more effective than others.
Refinement is purposefully vague. It's whatever conversation need to happen to make upcoming Sprint Plannings successful.
Planning poker is a rather broad topic - it is simply a technique for relative estimation. These sizes represent the overall size of an item relative to others. This is not a direct translation to time. Rather, there is a correlation when discussing similar work with a similar team. A great analogy from Mike Cohn is with distance running. A 10k is one size bigger than a 5k, but it doesn't tell us how long a given runner would take and you can't assume that any given runner will take twice as long to run a 10k as they do to run a 5k (or any other clear ratio). However, once someone runs a number of these, you can start making rough time estimates about how long it will take that runner to run similar courses.
Product Backlog refinement is the act of breaking down and further defining Product Backlog items into smaller more precise items. Scrum master helps the Scrum Team understand the need for clear and concise Product Backlog items.
It should be done before Sprint planning and the team should do it frequent basis. In backlog refinement, the Scrum team discusses/refine stories. If possible then assign story points as well(Size estimation, could be t-shirt sizing or story pointing). Story points are provided by taking some small stories as a base story and by comparison we can give story points. Or we can give story points based on work that needs to be done and the efforts it will require. Before starting with the story pointing team should agree on some mutual measuring unit(Can take a story as a base, Required Efforts, many more ways). Usually, it's kind of a rough estimation. It's good if you have some field in your tool to make stories as ready/draft/purposed...
The scrum master needs to ensure this session should happen and facilitate the same.
In Planning, Majorly team decides what can be achieved in a sprint. Value of the sprint and plan on how we will be achieving the targets. The team sits together and takeup stories from the refined backlog. Before starting taking up stories team should calculate team capacity by capacity planning. Usually, the team should take up stories that are in a ready state(Means already discussed). Scrum Master facilitate this event. Usually, planning is done by the scrum development team.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 5 years ago.
Improve this question
We have just started doing scrum at my company. We are spending a bit of time estimating Effort using planning poker and then when the detailed tasks are worked out a time estimate is put on each task.
The problem we have is that the time estimates are constantly wrong (usually over estimated). Although we can all agree on an effort, getting a team to agree on time for a task is much harder - what takes 1 person an hour might take someone else 3 hours. We end up going somewhere in the middle.
Who should be coming up with the time estimate for a task and when does this happen?
Is this just something we need more practice at, or are we doing it wrong?
The people actually doing the work estimate the cost involved. If you are using raw time as a metric for estimation, Agile methodologies frown on it. Your team should be using an abstraction to estimate cost, such as 'points'. You can start with a rough baseline of 1 hour per point with a minimum of 1 point. Then developers can make raw estimates of how long something should take. Slap them or anyone else on the wrist if they talk in hours or in any other unit of time.
The point is that as development moves along through multiple sprints, project managers can adjust 'point' time estimates provided by the team to match reality -- This can even be done per individual developer. Participants will become better and better at estimation as projects progress. So, since Sprints are an iterative process, time estimates improve with more iterations.
This begs another question: Why are you worried about time? Time is basically cost in the Waterfall model. In Agile, the goal is developing software to VALUE not cost. The reason points are used is that it is an abstract basis of comparison that business owners, project managers and creators (developers) can all view in an abstract light. (Unbiased from different participants' cultural, social or psychological perceptions of time.) Business owners can take a look at available points in a given sprint -- and knowing the points available -- they can elect functionality that is most important. It is always a bit of a tough decision, but again, the goal is to develop toward value and away from time boxing or feature stuffing.
"Who should be coming up with the time estimate for a task and when does this happen?" Depends on how you run your team. Do you let the team members truly self-manage, so tasks are assigned when a person grabs it during the sprint? You may have to keep using the time to complete based on the abilities of an average developer on the team. Do you have a team lead that assigns the tasks to people as they are created during the Sprint Planning meeting? Let the person assigned estimate the time to complete the task.
I agree removing time from the effort estimate is a bit confusing. The big question is: what does it matter that you are overestimating the task time? Is the team sitting around for 4-5 days at the end of a sprint with nothing to do? If so, go to the Product Owner and let her know the team wants to add one or two small items into the Sprint. You don't normally add stuff to an ongoing sprint, but Scrum is a framework to manage work, and as long as the team signs off on adding the new items, there is no need not to let Scrum work for your team....not force your team to work for Scrum.
Also, your questions seems to indicate your team has a greater velocity than what is being planned. If your 2-week sprint (10 work days) has a velocity of 10, but your team is getting finished with everything by day 7, just up your story points on the next sprint to 11 or 12.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
When and who undertakes the work to sufficiently gather answers so that we can start to write stories for an upcoming sprint. Is this work done continuously and in parallel to existing sprints by the product owner? I guess this then creates tasks for a sprint such as investigate x and y. What if the PO suddenly requires a developer to answer some of the questions by trying stuff out? I understand the idea of spiking and creating r & d tasks. I guess I want to avoid the main dev of a feature being delayed to a following sprint too often.
The team determines how much new story work it can do during a sprint. The amount of time they have to do that work is some percentage of the work day. Depending on the responsibilities of team members (customer support, bug fixes, emails, PTO, other duties) that amount varies from team to team. I like to see 10-15% of the work day dedicated to "planning" for the next sprint. That includes helping the PO research, writing stories, breaking up stories, design sessions, what-if scenarios, etc. I think the key is not to shoe-horn every one of these types of tasks into a sprint but rather to set the correct time allocation to doing the sprint work. Maybe something like 30 hours/wk is an average number.
So to directly answer your question; the planning work is done in parallel to the current sprint work.
We usually have one or two meetings to talk about future stories. Also, we reserve some overhead time in each sprint to check out things we need to know to start a story. The meetings help determining which stories will probably shop up in the next sprint, so we know which questions to get answers to during the reserved time in the current sprint.
For us, if it's a large project, we will have kickoff meetings to brainstorm the project. There is often a knowledge gap for PO's between what they want to do and what they don't know we can do that these meetings can fill.
When new stories are created, we try to assign story points to them at some point before the next planning meeting so the PO has time to prioritize the list before that meeting.
I'm not sure of the kind of situation you describe where a PO would "suddenly" need a dev to try stuff out. In that case, I would offer a spike in the next sprint. Generally using new technologies isn't something that happens every sprint so this should suffice. If not, perhaps the sprints are a bit too long for this purpose (a trade off to be considered at least) Another alternative would be to introduce an evergreen story for trying stuff out. I've seen teams have these kinds of stories for tech debt payback - you could off an either/or situation. Sometimes dev fixes tech debt, sometimes they try stuff out. And if you run out of tech debt somehow, you can always grab another regular story to put in its place.
We typically reserve a sprint or two after a big release for research and proof of concept stories. Doing research as part of the regular sprint seems like it would be problematic. You'd probably use that time to absorb mis-estimations for value-adding stories and end up never using it for actual research.
If a new story drops into the backlog that needs research and the PO runs it up to the top of your backlog then the team should include some research time into their actual estimate. I would only do that if I didn't have the luxury of a research/prototyping sprint ahead of time though since estimating research can be a bit nebulous.
Who: Product owner. Stories and Product backlog are his responsibilities. Product owners are generally experienced people; even if they are not technical they can certainly perceive implementation complexities at abstract level. Still, if a story has gray area PO must ask right people the right question. He can ask developers, testers, peers, clients and even scrum masters.
When: all the time.. Continuously. PO must not do anything but (1) provide (or get) answers for the team’s questions regarding scope and function, (2) and gather data that would refine the stories and their scope: thus proactively solving the queries of his team.
Bottom line is if product owner is not giving good stories to the team then he is not doing his job. Stories can be written by anyone but in the end it’s PO who ensure that Product Backlog is in order and that top priority stories are defined.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
I am the scrum master for a small team of 4 developers. The project we are working on has a lot of tasks that we have never done before and cannot easily estimate in a sprint planning meeting. What is the best way for me to run a sprint with this uncertainty? I am finding it very hard to finish a sprint with a potentially releasable product. I am also finding it hard to plan sprints when there is a lot of unknown length tasks.
I'm not sure what the term is in Scrum, but in User Story terminology you would do a "spike", which is basically a very short period of research into the topic so that your team will be able to estimate the task at the end of the spike.
Example:
Story:
Analyst wants to be able to review
financial data in pie charts.
Your team doesn't use any charting tools, so you need to know how long it would take to build something like this. Or perhaps instead, you could invest in third party tooling and integrate a tooling set with your application.
You would do a spike to research these venues and come up with estimations on them, then decide which route to take.
Are the "tasks" things that someone in the world has done before, or are they just new to your team. I will assume the later. If this is the case then what you are finding is that you do not have the necessary experience on your team to solve the problem. Thus you will be developing that experience as you go. All this means is that the complexity of your stories is higher. In the first couple of sprints you may score some of the stories as 13 and then later on they become 8s because you then have the knowledge you need.
You don't need to know how to do the stories to estimate them. You just need to take on less of them due to your experience gap.
I like to reserve "Spikes" (yes that is the term used in scrum) for attempting to solve business domain problems that have no known solution. Not for the team to do training.
If you really need to do research to get a good estimate, you could do the research as a task in itself, or set it aside and have it done (by someone) before the sprint planning.
Generally, I think that if you can't get a good estimate, you should either go with a bad estimate (i.e. a wild guess) or you should time-box the task, so you set aside a fixed amount of time for it in a sprint. After that, you will either have a done solution, or you will have a better understaning of it so you can estimate it or break it down into subtasks for the next sprint (or a later sprint).
Do you really mean tasks or are you talking about Product Backlog Items (PBIs)? Actually, I find it hard to believe that a task is not estimable. If they really aren't, they are very likely too big (tasks shouldn't exceed 16h, which is already huge).
If you are talking about PBIs, the situation you are describing is quite surprising and should theoretically not happen. In the worst case, just assign them a high number of story points, this precisely means that there is a lot of uncertainty on them. But, because PBIs that are ready for a Sprint shouldn't exceed the half of your velocity (or you'll put too much risk on your sprint), the obvious way to solve this situation is to divide such items into smaller chunks which may include exploration. But the important part is to keep things timeboxed, even (or especially) R&D. Keep in mind that with Scrum, everything is timeboxed.
In other words, to reduce uncertainty, break things down into smaller things (be them items or tasks)!
If the tasks seem un-estimable, I think the best approach would be to break down those tasks into smaller tasks that you can estimate. It might take several iterations, but you will probably come up with a pseudo design while you are at it. Joel mentions this in one of his articles.
Split the unestimatable task into a task to remove the uncertainty, and "the rest". Remove uncertainty with Proof-of-concept tests or spike solutions. Either schedules the spikes this sprint and the rest of the work next sprint, Or delay the start of the sprint for a week of spiking.
We often don't know enough to break a story down into tasks. We have a period of discovery before we know what the tasks will be. "Spikes" seem tricky to manage. For one, you may not be able to time box the discovery period. Second, I can't effectively plan a sprint without knowing how long a story will take.
Seems like another option is to do the spike in Sprint 1 and the tasks in Sprint 2. The downside is that it seems like the process forces an unnatural breakdown of the work. Why discover this week and then wait a while before starting the the work.
We use "contingents" or a specific backlog for such tasks.
The Scrum Tool Agilo is supporting this way of working and calculates those issues too, e.g. in the Burndown. In this way you get a good control on the "un-plannable" items.
Are you confusing precision with accuracy?
The idea behind Agile estimation is to come up with a number that is good enough, not a number that is exact. That is why using story points for backlog item estimation is a best practice; it emphasizes effort/complexity instead of duration.
You don't need to know how long each task necessary to implement a backlog item in a sprint will take. What you need to know is, given the work you've previously committed to in this sprint, can you commit to this backlog item? Because we know that we can't know exactly how much time each backlog item will take, we have to make an educated guess.
More important, what does it mean to fail in Scrum? Is not getting every sprint backlog item completed a failure? No... if you got four out of five items done and the fifth one is mostly done, you'll get credit for the four completed items (in terms of velocity for the sprint), and when you finish the remaining tasks for that fifth item in a future sprint you will get full credit for that item. But, would you have gotten any more done if you weren't using Scrum? The only failure in Scrum is failing to learn from your mistakes, to keep doing the same dysfunctional things repeatedly while expecting different results.
So, in your sprint planning meeting, don't spend a lot of time worrying about something you're not going to be able to know. Let the team think about the work, and then let them sign up for the amount of work they feel comfortable they can complete during the sprint. If they undercommit you can always drag something into the backlog, or end the sprint early. If they overcommit, then you finish the backlog items you can in priority order and discuss why the unfinished items couldn't be finished in the sprint retrospective, along with how to prevent having unfinished items in future sprints.
By the way, I know this was probably a poor choice of words on your part, but an effective Scrum Master doesn't run the sprint. The team runs the sprint, and the Scrum Master actively looks for impediments that lower their productivity and interfere with their ability to meet their commitments. Scrum Masters aren't managers, they're a combination of referee, coach, and scorekeeper. They are the Keeper of the Process, they help the team follow the process, they protect the team from outside agents that try to go around the process, and they track progress during the sprint via ensuring the sprint backlog is updated and the sprint burndown chart reflects reality, on a daily basis. In the situation you've described, where the team isn't sure how much work they should sign up for, the Scrum Master should let the team decide as a reflection of respect for team ownership of the commitment. Whatever the decision is, it won't be wrong.
Spikes should be time boxed. It puts on pressure on the team to limit the scope and have a better idea on the cost-benefits that the research will entail; ie it is useless to carry out 3 days of research for a task which would cost a few bucks.
This is also backed up by Latham's work on Goal Setting Theory where he specifically tackles this issue.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 5 years ago.
Improve this question
We use Scrum. We are experiencing problems during sprints when we find the user stories are not sufficiently granular to capture the effort required to complete the sprint.
In particular, we find that we are supplied with UI wireframes that contain much more complexity than the original stories would have implied (e.g. functionality duplicated for usability reasons). This leads to the burndown chart looking like everything is completed on the final day of the sprint.
We spend the Monday at the start of each 2-week sprint going over the stories as created by the project team, during which time we typically refine the stories a little and break them down into tasks, estimating the hours for each to create the burndown chart. During this day it doesn't feel like we have time to meaningfully improve the quality of the stories.
How best to break the cycle of incomplete / insufficient stories for our sprints?
Is this a failure of the project team to nail down the stories sufficiently at the outset, or should we (i.e. the dev team) take some of the responsibility?
So are you saying that:
Customers/users talk to project team
Project team writes stories and creates wireframes
Development team breaks down stories into tasks and estimates
Is there a possibility of the development team actually talking to the customers/users? User stories are sometimes seen as a way to kick off a conversation, as opposed to requirement documents/specifications.
EDIT: Some links:
A user story is to a use case as a gazelle is to a gazebo
Six Features of a Good User Story - INVEST Model
The Customer is Always Available
EDIT: Martin Fowler made a blog post yesterday on ConversationalStories that covers this far better than I did.
Are you running sprint retrospectives? At the end of the retrospective you should have high priority actionable items to improve on what happened in the previous sprint. The same things shouldnt be going wrong repeatedly.
Is your product owner accessible during a sprint? If not you may need to add extra to any estimation as the detail of a user story is incomplete.
#Pascal suggestion to dedicate 5% of your sprint to product backlog grooming is a good one. This should enable the user stories to be in a more detailed place before the start of your sprint.
We spend the Monday at the start of
each 2-week sprint going over the
stories as created by the project
team, during which time we typically
refine the stories a little and break
them down into tasks, estimating the
hours for each to create the burndown
chart. During this day it doesn't feel
like we have time to meaningfully
improve the quality of the stories.
It sounds like this is your sprint planning session, do you have control over what user stories you are commiting to complete during the sprint? How can you commit if you dont have sufficient detail?
This takes you back to having a good retrospective and solving the issues raised.
How best to break the cycle of
incomplete / insufficient stories for
our sprints?
Retrospectives, planning, backlog grooming.
Is this a failure of the project team
to nail down the stories sufficiently
at the outset, or should we (i.e. the
dev team) take some of the
responsibility?
Its the responsibility of the team as a whole. Finding blame isnt going to give value, but everyone taking responsibility will give everyone a chance of completing the project succesfully.
Maybe during those Monday morning planning sessions you can involve whoever is writing the user stories / wireframes and work together to find out what detail is missing from them, what detail would make your estimations easier and more accurate. Maybe a template of what they should include could be drawn up.
We had (and continue to in some respects) this same problem. I think this problem is a lacking of upfront analysis and a lack of devs spending enough time estimating a user story.
You might start with a story like:
As an administrative user I can create a new widget.
OK, what does that mean? After some analysis, it might mean:
As an administrative user I can create a new widget in created status with complex data validation errors.
So after that a listing of fields, how big, and what the required fields are for saving to the database. A basic UI mock up would be nice as well.
Another user story for the next sprint might be:
As an administrative user I can edit a created widget and correct the complex data validation issue to move the widget to completed status.
Then list of the complex validation rules.
We spend the Monday at the start of each 2-week sprint going over the stories as created by the project team, during which time we typically refine the stories a little.
At the start of the sprint, the stories should be READY. If you need to refine them a bit, I think that you (the dev team, the ScrumMaster, the project team) should do that a bit ahead, during the previous sprint.
How best to break the cycle of incomplete / insufficient stories for our sprints?
You are maybe underestimating stories or they are too big and too vague. In both case, this sounds like an estimation problem and a good way to improve is to reduce the size of stories. To work on this issue, you could dedicate some time (e.g. 5% of every sprint) to Product Backlog Grooming in order to prepare the most important stories and reduce their size by putting them on diet if required before the next sprint. And this will actually make the sprint planning meeting smoother.
Is this a failure of the project team to nail down the stories sufficiently at the outset, or should we (i.e. the dev team) take some of the responsibility?
The responsibility isn't that important IMHO (except for political reasons maybe but they do not produce much value anyway), both the dev team and the project team are working together and "failing" together. What is important here is to inspect and adapt to remove the obstacle. So, it's the dev team responsibility to make this problem visible (it is an impediment). And it's the ScrumMaster responsibility to work on this impediment. The failure would be to not work on it. Backlog Grooming sessions are one way to do it. And at the end, I'm sure the project team will improve and get a better understanding of what the dev team is expecting. And you will both produce better results.
Lots of good ideas here already on the scrum aspects of your problem. Based on your comment:
particular, we find that we are supplied with UI wireframes that contain much more complexity than the original stories would have implied (e.g. functionality duplicated for usability reasons).
I also have a concern that you might need to work on you development process as well though. Accessing functionality from multiple locations in a UI should be a simple addition that consumes almost no time at all. If you are finding this to be a common problem then your functionality is too tightly coupled to particular UI elements. Your team might need to improve their design skills (eg: pattern usage).
This is interesting. It would appear that you are doing the sprint planning in the sprint? And that the Sprint Backlog is committed before the Sprint Planning? If so, how are the team commiting to the sprint backlog without discussing the details of the stories?
An alternative approach could be to make the Product Owner aware that certain stories cannot be added to the sprint backlog due to lack of clarity. In particular that the acceptance criteria are not fully understood. This could provoke the necessary conversation with the Product Owner. Ideally it shouldn't come to this. It should be discussed and resolved in the retrospective.