«Abstract Large IT projects may not deliver what they promise. Often they are late. Often they are over budget. Often what is eventually delivered is ...»
6 Why Large IT Projects Fail - http://www.ecs.soton.ac.uk/˜ph/LargeIT.pdf These phenomena are well known in the industry. They are consistently overlooked. Because they are not Laws in the sense of Laws of Physics [Henderson 1998], manager after manager will convince themselves that they do not apply in their particular circumstances.
2.1.5 Complexity Last, we must consider the inherent complexity of large IT projects. Clearly, this varies from project to project, but there is no doubt that software in general can be very complex (in both the formal and informal senses) [Axelrod 99, Kauﬀman 2000, Chapman 2002, BCS 2004, Xia 2004].
Complexity usually means that the separability of the components of a product are severely constrained. Changes to one component induce changes in many others. This is referred to as being tightly-coupled. A good design will seek loose-coupling or near-independence between components.
However, good design isn’t often achievable in large projects - because of their inherent complexity;
a cyclic problem if ever there was a one. Few project managers have the courage to invest more in design in order to save later on building.
There is some threshold of connectedness above which the complexity of a large IT system becomes intractable, not unlike the thresholds discovered in combinatorial problems [Hayes 2003]. Below this threshold, the work required to solve a problem is eﬀectively linear in some aspect of the size of the problem. But above the threshold, work increases exponentially (or worse) and soon problems just a little above the threshold are, for all practical purposes, unsolvable.
Complexity is often exaccerbated within the project, where building loosely-coupled solutions takes longer than building tightly-coupled ones. The project looks only at the short-term costs and not at the total-cost-of-ownership. A too-quickly developed solution will usually be more tightly-coupled than it could be, and that will lead to problems with internal complexity much later.
2.1.6 On the relationship between Size, Quality, Cost, Schedule and Complexity
Clearly these ﬁve aspects of largeness in a large project are themselves related. We have referred already to some of these relationships, for example larger projects will cost more. In fact we even suggested that cost was a reasonable measure of (post facto) size. We will not try to formalise the relationship, but an informal discussion is warranted.
Figure 1 is a diagrammatic representation of the relationship between these ﬁve values and is based on a model that we have used in earlier experiments [Chatters 1998]. The box in the centre of the diagram is intended to denote the development process (think of it as a factory) which takes in components of a certain Quality and builds and outputs a product of a certain Quality. The dominant measures of the size of the task at hand are the perceived Size and Complexity of the product. By doing Work on the product (i.e. designing, building and testing it) we can aﬀect the Quality of the output. We can also aﬀect the delivery Schedule.
Similarly, if the input Quality of basic components from suppliers is high, we would expect to achieve a target output Quality for less Work. Here, we align Work with Cost, on the basis that person-years is the dominant cost in large projects.
In [Chatters 1998] we showed that output Quality is related to Work by a law of diminishing returns; that as the project gets bigger the amount of delivered Quality per unit of Work reduces.
This rather obvious property was supported in that paper by some rudimentary data, but data of that sort is very diﬃcult to acquire because it can be very sensitive. We found that the data we did collect was rather unreliable and so it was not possible, at that time, to establish the relationship precisely.
2.2 What Failure Means
Given these ﬁve Measures we can state what it means for a large IT project to be perceived as failing, or to have failed if it has been abandoned. Since it is often the case that large IT projects are perceived as failing long before they are ﬁnished, we address the symptoms in the order in which they are usually reported. First we discover that the project is going Over Budget, then we discover that it is Over Time and ﬁnally we discover that the customer sees it has having Low Quality.
2.2.1 Over Budget Many IT projects cost a little more than originally budgeted. The characteristic of large IT projects seems to be that they go over budget by a very large factor. Then, the fact that their budget is already very high, means that the additional cost will be newsworthy.
Since this cost over-run will normally happen before the original schedule has run its course, it will normally be the need for the suppliers to renegotiate their contract that is the ﬁrst indication that a project is failing. As we shall see, it is the realisation that the requirements apparently agreed in the initial contract cannot be delivered for the initially quoted price that kicks oﬀ this renegotiation.
2.2.2 Over Time
than anticipated, so more Work is required (i.e. more Cost) and it will take longer, because we cannot trade person years (except within fairly tight limits).
2.2.3 Low Quality Finally, when a ﬁrst product release is eventually available for end-user evaluation, it is reported that the product does not fulﬁl the requirements that the users now have for it. The GedankenExperiment that we discussed earlier, of the end-user apparently ticking-oﬀ requirements on a long list and then measuring Quality as the number of ticks that the product receives, gives us a crude measure. Often this measure is low when a large IT project is delivered. The reasons are many and various, as we shall elaborate in the next section. But primary among them is that the requirements as understood now by the end-user are some way from those that were agreed at the outset.
Formally, the diﬀerence may be small. In practice, it may be all that matters when it comes to reporting on the perceived quality of the eventual product.
2.2.4 Comments on Perception of Failure As we discussed above, the three perceptions of failure, respectively being too expensive, being too late or being of poor quality are interrelated. Often we have all three, but even having one of them would be considered a failure.
We will argue that it is an inadequate attention to requirements that leads to failure. There are two aspects of this. Of course, over the lifetime of a long-running project, the requirements will change.
Perhaps more signiﬁcantly, the initial requirements will also “change” in the sense that, as they become more fully understood on the part of both supplier and customer, it is increasingly realised that the initial agreement was based on “weak” mutual understanding. What was perceived as agreement, is now seen as a basis for conﬂict. The supplier wants to deliver what they originally undertook to deliver, the customer wants that too, but disagrees as to the details of that initial agreement. The customer now wants something additional however, something more reﬁned or more elaborate that the initially agreed requirements. This puts the customer in a weak position to negotiate a compromise.
2.3 Reasons why Large Projects fail
In the overview, we brieﬂy described ten reasons why large IT project fail. Now we have some vocabulary with which to discuss those reasons. The order in which they are addressed is rather arbitrary. Since they are all interrelated, no order is perfect. Eventually we will show that the root cause is our failure to manage the requirements process adequately and, in particular, that requirements drift goes largely unnoticed.
Both customer and supplier expect more than is realistically possible. The customer expects that the product will solve all their problems. The supplier sells that dream. Both may well believe that the agreed requirements can be delivered for the agreed cost and that those agreed requirements will actually meet the customer’s need. They are too optimistic.
2.3.2 Investment There is too little investment at the beginning.
This optimism leads to a conﬁdence in the estimated cost which results in an initial investment that is far too small. The investment is likely to be related to a spending forecast that anticipates an unfaultering project plan. Something like 25% is added to cope with “unforseen” circumstances.
The reality is that there was never really any prospect of delivering the project for 200% of the estimate, let alone 125%. The Size and Complexity have been underestimated. The Quality of existing components (in the sense of their ability to nearly match some of the requirements) has been misjudged, and hence the rework necessary has been underestimated. This is true even when Open Systems components implementing open standards have been used. The work required to actually understand how these components work, and in particular how they interoperate with others, is usually much more diﬃcult than is anticipated.
2.3.3 Procrastination Suﬃcient investment is made eventually, but is too late.
So reality dawns. More investment is needed. This is the ﬁrst of what might eventually be a number of “stumbles”. Cynically, we might say that, for all large projects, one or two stumbles are required as the organisation (the suppier and the customer) learn about the complexity of the task they have embarked upon.
However, had the initial investment been larger, the project would probably not be so far behind.
This is something that our models show. Anticipating a more distant objective can accelerate progress. So having an initial establishment that makes more distant objectives possible, means that the eventual cost of delivering a given function to a given quality will actually be less than that which will be achieved with insuﬃcient initial investment, even if that is eventually “topped-up” to an adequate amount.
2.3.4 Technical Know-how There is not enough technical know-how in the project team.
Of course, making suﬃcient investment probably assumes that the staﬀ that the supplier uses for the project are of suﬃcient technical quality at the outset. Optimism sets in here too. We all assume that we are able to achieve more in a given time than in fact we are capable of achieving.
We usually over-estimate our own technical ability. Experienced project managers allow for this.
There are generic technical skills which can be established and allowed for at the outset. But 10 Why Large IT Projects Fail - http://www.ecs.soton.ac.uk/˜ph/LargeIT.pdf there are speciﬁc technical skills, knowing the capabilities of the components we are building from, knowing the application domain suﬃciently well that the requirements can be understood, which are likely to be misjudged. We are optimistic. We expect the supplied components to be “better” than they eventually prove to be. We believe we understand the customers’ requirements at least as well as the customer understands them. The reality is that the components are not nearly as ﬁt-for-purpose as we anticipate (even when they adhere to open standards) and what we understand about the requirements is far from what the customer intends, even when they reach agreement and sign the contract.
2.3.5 Human Factors There are opposing Human Factors aspects - including insuﬃcient consideration of teams, project management and risks.
This is a large topic, and many writers will suggest that solving project management problems is the single most important aspect of delivering Quality on Schedule and for an agreed Cost [Brookes 1979, Petroski 1992, Grey 1995, BCS 2004, ACM, Neumann 1995]. These claims are of course valid. A poorly managed project, without an appropriately organised team structure and without continual reassessment of the risks ahead, will not deliver on any of the three measures of success. However, many large IT projects are managed correctly in these respects and yet they still fail [Intellect 2003, Sauer 2003, Xia 2004, Humphrey 2004]. So, while getting the “people” issue sorted is necessary, it is not suﬃcient.
2.3.6 Incremental Delivery The project does not plan incremental roll-out and fails to anticipate the eﬀect of big-bang on the organisation.
A project that introduces new technology into an organisation incrementally is more likely to eventually be seen as successful. There are two reasons for this. The ﬁrst is simply that each increment is the product of a smaller project. The second is that, where there are problems that are related to misunderstandings between supplier and customer, these problems will be encountered earlier and have smaller consequences.
A component-based solution, using commodity oﬀ-the-shelf components (COTS) can make incremental delivery possible, especially where these components adhere to open standards as expected in an Open Systems approach. There are many beneﬁts to introducing these new components incrementally, not least that problems are encountered early, even if it is only problems of perception
- understanding what the components actually do and how they actually match (or fail to match) the customers requirements.
IT oﬀers a business more eﬃcient and more eﬀective ways of running their business processes. But where, as is usual today, the project to introduce new IT starts with an existing software solution (e.g for ERP or CRM) much of the engineering that is involved is specialising the IT to the business processes of the customer. Where the customer insists on continuing to run the business as before, these engineering changes can be overwhelming. One suspects that for many projects they are actually impossible.
The IT packages that are built upon will have built-in generic business process models. Where the customer is willing and able to change their business practices to more nearly match those generic processes, then the amount and complexity of the engineering that will be required is reduced, often substantially.