«Abstract Large IT projects may not deliver what they promise. Often they are late. Often they are over budget. Often what is eventually delivered is ...»
The changes to E represent revision and understanding of requirements (i.e. drift). A change may remove an old requirement or add a new one. The probability q of this happening to any individual requirement is very low. For example, in the experiments we will describe shortly, it was taken to 0.1%. At each simulation step, each requirement changes with probability q, so that on average (if we have 1000 potential requirements) one requirement will change with each step. Some of those changes will be to respecify (and consequently, unimplement) an expected requirement. Others will be to introduce a new requrement. This represents requirements drift.
16 Why Large IT Projects Fail - http://www.ecs.soton.ac.uk/˜ph/LargeIT.pdf
Figure 3: Two scenarios, showing requirements drifting over time. Percentages show achievement.
4.1 Experimental Results The results of a couple of experiments done with this model are shown in Figure 3 and Figure 4.
For both scenarios, the rate of drift q was set to 0.1%. For scenario A, the development rate p is 0.5% and for scenario B it is double that, at 1%.
The graph in Figure 4 showns that scenario A, with the lower development rate reaches the initial requirements after around 525 days, but at that time in reality it has only implemented 87% of the drifted requirements. In fact, this development never reaches the drifting requirements. If we accept 95% as our target, it takes considerably more than twice as long to get there.
Scenario B shows the same behaviour. With productivity doubled, it takes nearly 300 days to reach the original requirements and by day 600 we have still only implemented 95% of the drifted requirements.
4.2 How results support our conclusions The models of course are purely illustrative. They are themselves conjectures. We use them here to show the way that development proceeds and how the modest requirements drift makes an apparently acceptable plan actually return failure according to the criteria that we have established.
In scenario A, for example, the project may well have planned conservatively to deliver at 600 days anticipating the kind of diminishing returns that set in as the product grows. But at 600 days they have ony achieved 87% of what is then understood as the requirements. Anticipating drift, the project would have invested more at the outset and perhaps achieved 95% of requirements by day 600. The amount of additional investment would however be about double, as a comparison between scenarios A and B will support.
It is likely, in practice, of course that realisation that the target is not going to be achieved will be apparent some time before the due date and a correction will be made at that time. This will reduce the total cost somewhat, but it will not be less than if the initial investment had been adequate.
400 300 200 100
We claim that a recognition of requirements drift by all parties to the project, but in particular the engineers and their immediate managers will lead to more conservative decisions relating to the plans made for a given investment. Being aware that drift happens and is an inevitable consequence of the need for an organisation to learn, will lead to a project management and execution behaviour that constantly looks for drift. Attending to the requirements, for example by engaging relevant parties in analysis sessions in which the requirements are continuously revisited, will likely expose drift early enough that its eﬀect (of elongating the schedule, increasing the cost or reducing the delivered quality) can at least be anticipated at an earlier stage.
The problem, of course, is believing that drift is so pernicious. It is, after all, just a lot of very small changes, each of which is of little consequence. But if the project team really believes that this is the underlying cause of failure, then they will be prepared to pay more attention to being pedantic about their understanding of requirements. There is no silver bullet here. It’s not that simply “involving the customer earlier” will solve the problem. The team have to believe that, even when they do this, the requirements will continue to drift. Indeed, giving too much control to the customer, will simply accelerate the drift.
These observations are just as true when we adopt an Open Systems approach [Krechmer 2005], using components that implement open standards and are to a large extent plug-and-play [Veryard 2000].
This approach may raise the level of complexity with which a project can cope and are normally essential to a modular approach, allowing subsystems developed independently to interoperate. But they do not solve the problem of understanding the requirements and absorbing the drift.
On the other hand, the Open Source movement, which builds on open standards, may have some answers. Here components evolve in the true sense of that word, and the systems they contribute to also evolve. The advantage of this as a development method is that there is no cost associated with production. Schedules still obey some laws of evolution [Henderson 1998] and defy inﬁnite compression, but there is no doubt that quality components do appear from this process. It is diﬃcult to imagine “planning” a system delivery based on the anticipated production of new components by the Open Source method, but it is increasingly common to rely on Open Source components that exist at the outset of a project. The diminishing returns that exacerbate the drift we have observed will not be so apparent in these circumstances, since we will not be paying anything for development.
18 Why Large IT Projects Fail - http://www.ecs.soton.ac.uk/˜ph/LargeIT.pdf
We have argued that, many of the reasons given for the failure of large IT projects is an inadequte respect for the project’s understanding of requirements. We have tried to show that requirements drift either underlies the main reasons given for failure, or contributes to them in a signiﬁcant way.
We have shown, in a model, that the diminishing returns experienced in a large projects as its product gets bigger is exacerbated by drift to the extent that projects may never achieve the kind of quality that makes their product acceptable. And ﬁnally, while oﬀering no silver bullet, we have suggested that a true belief in requirements drift among engineers and their managers may itself go a long way towards mitigating its eﬀects.
[Axelrod 99] Axelrod, R and M. D. Cohen, Harnessing Complexity - Organisational Implications of a Scientiﬁc Frontier. Free Press, NY, 1999 [BCS 2004] British Computer Society, The Challenges of Complex IT Projects. The report of a working group from the Royal Academy of Engineering and the British Computer Society, British Computer Society, 2004 [Boehm 1981] Boehm, B, Software engineering economics.. Prentice-Hall, Englewood Cliﬀs, NJ, 1981 [Boehm 2000] Boehm, B et al, Software cost estimation with Cocomo II. Prentice-Hall, Englewood Cliﬀs, NJ, 2000 [Britton 2003] Britton, N. F, Essential Mathematical Biology. Springer-Verlag, London, 2003 [Brookes 1979] Brookes, F. P, The Mythical Man-month. Addison Wesley, 1979 [Cambridge 2001] Cambridge Reporter, CAPSA and its implementation. Cambridge Reporter, 2/11/01, http://www.admin.cam.ac.uk/reporter [Chapman 2002] Chapman, J, System Failure. DEMOS, 2002 [Chatters 1998] Chatters, B, P. Henderson and C. Rostron, SIMMER: Software and Systems Integration Modelling Metrics and Risks (Getting to Level 4). EuroSPI 98, Goteborg, November 1998 [Grey 1995] Grey, S, Practical Risk Assessment for Project Management. Wiley, 1995 [Hayes 2003] Hayes, B, On the Threshold. American Scientist, Vol.91, Jan 2003 [Henderson 1998] Henderson, P, Laws for Dynamic Systems. Proceedings of International Conference on Software Reuse (ICSR 98), IEEE Computer Society Press, 1998
[Humphrey 2004] Humphrey, W.S, Why Big Software Projects Fail: The 12 Key Questions. The Journal of Defense Software Engineering, 2004 [Intellect 2003] Intellect, IT Supplier - Code of Best Practice. http://www.intellectuk.org, 2003 [Kauﬀman 2000] Kauﬀman, S, Investigations. OUP, 2000 [Krechmer 2005] Krechmer, K, The Meaning of Open Standards. The International Journal of IT Standards and Standardization Research, Vol. 4 No. 1, 2005.
[Neumann 1995] Neumann, P, Computer Related Risks. Addison Wesley, 1995 [NAO 2006] National Audit Oﬃce, National Programme for IT in the NHS Report by the Comptroller and Auditor General, HC 1173, http://www.nao.org.uk, 2006
[Petroski 1992] Petroski, H, To Engineer is Human - The role of failure in successful design.
Vintage Books, 1992 [Sauer 2003] Sauer, C and C. Cuthbertson, The State of IT Project Management in the UK 2002Templeton College, Oxford, www.cw360ms.com/pmsurveyresults/surveyresults.pdf, 2003 [Veryard 2000] Veryard, R, The Component Based Business - Plug and Play. Springer Practitioner Series [Wolstenholme 1990] Wolstenholme, E, System Enquiry, A System Dynamic Approach. John Wiley & Sons, New York 1990
[Xia 2004] Xia, W and G. Lee Grasping the Complexity of IS Development Projects. Comm. ACM, Vol.47, No.5, 2004