Agile Software Development: A Study of the Northern Ireland Software Industry (2012)

The agile model is now a mainstream approach to software development. Although historically the concept was only introduced in 2001, its popularity meant that by 2008, roughly 70% of software development organisations worldwide were claiming some degree of involvement in agile use. The rate of adoption has continued rapidly since then to the point where some organisations now have agile development as their central strategy, making adjustments, as necessary, for projects that are defined in an unfavourable way.
In late 2009, a local survey was undertaken to determine the capability and capacity of the Northern Ireland software industry to engage in agile software development. This work addressed five related research questions:
• What does it mean to adopt an agile approach to software development?
• To what extent is agile adoption in Northern Ireland indicative of agile adoption worldwide?
• What are the main inhibitors and accelerators for agile adoption?
• To what extent does the agile approach help or hinder individual aspects of the software development process?
• In what ways is the quality of the software product affected by an agile approach?
The survey was completed in March 2010. The results revealed that approximately two-thirds of the companies that participated used agile development routinely in commercial projects and in doing so reported a range of resulting benefits in almost all aspects of the development process.
The current report brings those results up to date by returning to the companies that participated in 2010 to record any change in their views and identify areas of difficulty in establishing more mature agile engineering practices.
Thirty-two of the original thirty-seven respondents from 2010 were able to take part (some are no longer in operation) and eight new companies agreed to be involved, bringing the total surveyed to forty.
The main findings from the 2012 survey were:
1. The level of adoption of agile methods continues to grow. The organisations surveyed fall into two groups: (i) those using agile methods for some or all of their projects; and (ii) those with either experimental or no direct experience of such methods. In the 2010 survey, roughly two-thirds of companies were in the first group (24 out of 37), using agile methods routinely, which rose to three-quarters in 2012 (30 out of 40).
2. Agile is becoming the default approach to development. In 2009-10, most companies saw agile development as an addition to their repertoire, rather than a replacement for traditional techniques. By 2012, that position has changed, with half of those surveyed now using agile as their general (default) approach across the organisation. In some cases this meant using agile development even in circumstances that seem unfavourable and then compensating as necessary.
3. The level of experience and associated maturity in agile development in the province has increased substantially. In the 2010 survey, 40% of the companies participating had completed five or fewer agile projects (as indicated by a product release) but by 2012, most had completed at least 6-10 such projects, with a quarter having over 50. Maturity also seemed to be accelerated by the reflective cycles of development involved, where issues and opportunities for improvement were considered routinely at the end of each cycle.
4. Scrum, which is an iterative incremental management framework for agile projects, is the dominant approach used among the companies surveyed. In some cases, Scrum is being used exactly as defined but there is now a growing level of experimentation, with many companies effectively defining their own methodology. There is also a growing interest in lean software engineering as a way of implementing agile practices, using ideas such as Kanban, from lean manufacturing.
5. Implementing an agile approach to software development involves the use of specific agile techniques. These are often associated with the practices of XP (eXtreme Programming) or Scrum, with examples including ‘continuous integration’, ‘daily stand-ups’ and ‘sprint retrospectives’. The three areas of greatest growth noted in the survey were ‘continuous integration’, ‘test-driven development’ and ‘small releases’. Many have found the practice of having an ‘onsite customer’ difficult to achieve in practice, which can affect other practices and require adjustment to the agile process itself.
6. A wide range of tools is used to support agile projects, ranging from those designed specifically for agile development down to general tools, such as whiteboards and spreadsheets. It was found that there was now greater use of physical post-in notes on whiteboards to summarise the state of a project, as these facilitated daily stand-up meetings and provided a highly visible indication of progress. With respect to electronic tools, a market leader has yet to emerge but Microsoft TFS seems popular. The perceived need for greater tool support in agile development has increased by nearly 50% over the opinion reported in the 2010 survey.
7. Overall, substantial benefits from the use of agile methods were reported. Experiences varied across companies, however, so some were not always clear that they had a gain in specific areas. On average, improvements over traditional approaches were noted in all nine areas of software development considered. In the 2010 survey, the most significant gain was in ‘developer productivity’, but this has been overtaken by ‘testing’ in the current survey. The average level of gain was 17% greater than in 2010. One drop in satisfaction was with requirements management. Coping with evolving requirements is inherently difficult and the lower rating may just reflect a greater appreciation of that difficulty as more projects are completed.
8. Developer satisfaction was found to be another positive aspect of using an agile approach, with an increase in benefit reported of 18% over the 2010 survey. The greatest gain was in developers having a clearer view of their role.
9. Customer satisfaction was also found to increase through agile techniques. In particular, there is often better engagement with developers and a faster response to changing requirements, leading to improved product quality overall. Of the six aspects of product quality considered, all indicated improvement, with ‘functionality’ and ‘usability’, on average, providing the greatest gain. Again, the levels of benefit were up on those reported in 2010—by 30% in this case.
10. In general, management satisfaction also indicated benefits across the range of areas considered. The one exception was contract negotiation, with agile development perceived as less well suited to fixed price contracts. Overall, however, management satisfaction had the greatest increase in reported gain over the 2010 survey, with levels up by 48%, on average. The areas of greatest gain were in ‘tracking and ‘reporting’, though there was a drop in confidence with ‘quality control’.
11. In the 2012 survey, two new questions were added to determine the extent to which organisations are achieving optimal benefit from an agile approach and so highlight areas of shortfall where additional research and development may be needed. There was a wide variation in responses. Some companies felt that their maturity of practice, processes and techniques was close to optimal, though there were residual concerns about making full use of tools and avoiding all negative side effects in development. There was unexpectedly high confidence in using agile in unfavourable circumstances, which seemed connected with working in an environment where such projects were the normal case, and so appropriate expertise had been developed. It was interesting to note that while the greatest area of difficulty reported was in the handling of user requirements, the greatest gain cited was in the creation of a better product. This suggests that requirements handling is more a relative difficulty within the agile approach, with substantial gains achieved in the process overall—consistent with the extensive and rapid adoption of the agile approach in recent years.
12. The Higher Education institutions in Northern Ireland give all of their students an introduction to agile software development, and in some cases provide opportunities to develop skills in particular practices and to use specific tools. Aspects of agile development are covered at both undergraduate and postgraduate level, with speakers from industry often contributing details of practice in their own organisations.
13. The report includes summaries of the experiences of some local companies in their adoption and use of agile development. These are presented as 2-page ‘case studies’. Such studies could have been developed around a large number of local companies so those included are just an illustration of the variety of experiences possible in adopting agile practices. It is hoped that most other organisations will be able to identify with at least one study presented.