In software development, some functions can be discussed, implemented, tested and then shipped. Other functions need several iterations to really build the right thing in a good way. Incremental building and iterative development are completely different things and need different approaches to manage.

Iterative and incremental work are fundamentally different in nature:

• Incremental work items, once done, do not need much attention any longer. They are quite easy to plan.
• Iterative work items however need to be tackled step by step: do something, check the result and decide on how to proceed next. Predicting the needed number of iterations can be difficult or even impossible. Thus also the time needed may be tough to predict. Looking ahead of iterative work is a nightmare.

The typical consequence of this difference is that iterative work is usually not well estimated and not enough funds are given. Once a team realizes this gap, they can either deliver a solution that does not meet expectations or increase the resources they have available.

The following work is iterative at heart:

• Making users have a great time using a product.
• Meeting expectations.
• Finding a successful spot in the market
• Developing machine learning algorithms that work as well as desired.
• High-end performance of a software system.
• Creating a piece of hardware that fulfills the conflicting specification about size, power supply, protection from environmental influences, ease of use and fulfilling the intended function.

To make the list shorter: Teams need iterations to reach or keep a certain level of quality and to resolve conflicting inputs. Adding more software functions with a basic quality is usually incremental work.

Burning down incremental work items

Agile teams plan incremental work using backlog items. Forecasting is done e.g. with burn-down charts. Such charts show how the backlog size is reduced over time.

Iterations means reworking things already done. The backlog increases. The consequence, other items need to be removed or the development prolonged. The underlying message – we can deliver fewer features than we thought we would be able – is usually not well received. The software project management frameworks like SAFe, Scrum, Kanban in use today are very good at managing the incremental part of the work. The iterative quality is however more difficult to integrate.

Upfront iterations

Some organisations thus split the team in an upfront team (or product owner team) and a development team. The upfront team takes the up-coming items a few sprints ahead of time and specifies the functional solution. Usually, organisations split by discipline: business analysts collect the wishes and ideas, then work-out and write down the requirements. UX designers then take over, add a fancy mock-up to it. Finally the development team implements it. Another favorite split is to have a large hierarchy of product owners who do it, supported by UX experts.

Such setups can work. They however have many drawbacks of waterfall development. E.g. everybody assumes that the up-front team does the iterative work so the development team can just build the features right. Experience shows: such first time right approaches work better for simple features. Complex features need feedback from usage and a more interdisciplinary collaboration to make them right.

Another approach are technology projects, research epics, or similar. Here organisations take out really big chunks into a separate budget and remove them from the development backlogs. Works well. Has however the danger that those doing the technology projects form upfront teams and become detached from implementation.

Approaches

Here are a few promising approaches. Organisations usually take a combination of them.

Continuous exploration: Iterative work, e.g. feasibility spikes and user experience prototyping, is done by the team. The idea: Do quick iterations with simple prototypes to reduce the uncertainty to an acceptable level before the more expensive programming starts. The iterative upfront work reduces the team’s capacity to work on the incremental backlog items. The velocity gets reduced and can now much better forecast what will be delivered.

Works well when iterations return relevant results without testing with product versions. Has the risk of working out too expensive solutions that cannot be built. To alleviate this effect, teams try to build features up:

Building up: Teams release the very basics of a feature quickly to a selected audience. From the feedback, they decide how much to still add and what to change. Collecting some numbers really helps in deciding whether a feature is good enough or not.

The big risk here is that the first thing that users experience is too basic and does not fulfill real life requirements. Quick improvements, excellent communication and budget to do it is an absolut must. An approach to reserve some budget are higher level backlog items.

Higher level backlog items: Backlog items ideally are small and can be closed after a couple of days. Therefore, when another iteration is needed, additional items are added to the backlog. To frame this, a team could create higher level backlog items that represent a larger lump of functionality. When estimating, the team also estimates effort needed for iterations. When using story poker – e.g. with 10 times bigger sizes, i.e. 5, 10, 20, 30, 50, 80, 130 and so on – a team can derive a velocity and do forecasting using the higher level items that now include iterative work.

This can work quite well. However not every item needs the same amount of iterations. There are quite a few things a team can easily work out with just mock-ups of the user interface. Other items however need feedback from usage to get to the bottom of things.

Mark uncertainty: I usually tag higher level backlog items to indicate the uncertainty these items still have and thus give an indication of the iteration potential still needed. Simple to do and it creates transparency.

• Stars – the best approach is not identified yet.
• Clouds – Approach is agreed. Still many things are unclear. We are unsure what we need to do in more details and we quite certainly missed some of the things to do.
• Trees – we can enumerate the backlog items we need to make it work, even though for some we are still unsure whether we should do them or how we should do them.
• Apples – we know what we still need to do so it is going to work.

The team then decides how to reduce the uncertainty. Options are e.g. testing with UX prototypes, learning about technology with spikes, finalize a concept, do user tests with increments or even collect feedback from a real release. Quite often, the level apple is only reached once some feedback from usage (e.g. in an alpha test) can be collected.

Steps in the dark: Some things are so unclear, that teams have no idea how many iterations they will need and how long it will take them. There are two useful options: (1) drop them; (2) do a first step, review the situation and decide again.

In a recent device development, the hardware team detailed just the coming step: the mock-up to build and what questions should be answered. The team was able to estimate the step and people could see that a positive answer would bring them closer to a real device.

Can work well. Organisations however need to adopt a mindset of affordable loss and be ready to walk in the dark for some time: a negative answer might lead to yet another step or even the end of the development.

Finally

Agile teams love incremental work items. They are quite easy to estimate and plan and once done they get the praise for doing a great job.

Iterations however are not that welcome. First, everybody wants to know when something is ready and iterations interfere with the forecasting by creating unforeseen work. Second, the first things developed do not meet expectations and many organisations then start the blame game. It follows that teams try to box iterative work, declare it as such and do it up-front. The goal: do the bulk of the work with plannable, incremental steps.

While this gives good and quick results for less complex tasks, some other tasks need an unknown number of iterations and yet other tasks even need a product launch to do them right. In such cases, there is no other way than doing some steps in the dark that might lead to nowhere.

My conclusions

• Be aware that iterative work is not easily plannable and very difficult to forecast.
• Create transparency about where iterations will be needed and reserve budget for such iterations.
• Up-front iterations work well for simpler tasks.
• Avoid upfront teams.
• Be prepared to take steps in the dark for complex tasks.

.