Software development is seen as chronically chaotic and complex to the extent that project management can achieve little control over projects or outcome [1, 2]. Recently, we have come into a new era of hope; hope of getting people - real people, users, both naïve and sophisticated - more involved with, more relevant to, and more visible in software development.
This hope is not merely on one front. In software engineering, agile methods (such as Extreme Programming) offer the promise of manageable projects, honest schedules, and increasingly useful end products. Similarly, usability methods (such as Scenario-Based Design) promise the possibility of clean, usable interfaces and interactions. Both of these areas rely heavily on heuristics. Heuristics are defined by two central ideas: they are guidelines, rather than fixed boundaries, and are designed to evaluate and explore the problem at hand in an efficient manner.
However, there remains a divide; software engineers largely look at the internals of software and usability engineers look at the externals. Heuristic methods, such as those described above, have revolutionized and re-energized each field and offer us a bridge. Rather than one group seeking to tightly control the design process, all developers can (with these methods) use their energies to identify and leverage design opportunities. When all developers share and integrate, rather than make fiefdoms, all can make the best use of their resources.
A New Hope
As software engineering matured, new human-centric methods emerged that can deal with the socially constructed needs of conformability and changeability [3]. Similarly, the ‘software psychology' of the seventies matured into usability engineering [4]. There was a drive to treat programmers and users as humans, not merely logicians and input devices. The remainder of this section will describe the overlap between these two areas and how they can mutually benefit each other.
Connections Between Extreme Programming and Scenario-Based Design
Extreme Programming (XP) [5] is a developer-centered agile method that is well-documented, well-studied, and embodies core principles common to most agile methods [6, 7]. Scenario-Based Design (SBD) was created not only to design user interactions with software, but to manage larger-scale issues of how people and software interact. Though they focus on different areas of development, XP and SBD share many common and analogous practices.
1. Story-Driven Development
In XP's method for eliciting and documenting system needs, developers (and often users) write and discuss stories about the system within the context of its use-thereby making validation issues as visible to the designers as verification issues. Similarly, SBD uses stories of users interacting with the system, written in the form of scenarios, to help guide developers in constructing the system. Stories are dynamic; they show goals and sequences, but they also show uncertainty, decision-making, and other complex aspects of the environment. Formal requirements often de-emphasize, avoid, or design these aspects out of the process. It is through this process of contextualization that the potential impact and value of the technology becomes central to the design, thus increasing the usefulness and usability of the end product. Acceptance criteria for heurists lie in users and uses, rather than test protocols generated at some arbitrary point in development.
2. User Involvement
The logic behind user involvement in the development project is simple; the more active a role the user takes in development, the more likely it is for the product to represent user's needs. As with XP, users are a critical part of SBD development efforts. SBD does not express a static role for the user. Users can author, analyze, and discuss scenarios, analyze claims, participate in brainstorming, or even share in developing solutions - all elements of participatory design. Implicit in all participatory design is a shift in power; the developer still maintains a degree of control, but the engaged user-participant wields more power than the typical user-as-subject of traditional development efforts.
3. Testing Emphasis
{C}{C}A biased view is that a 'complete and correct solution' is derived from a proof-based design, and thus needs no further testing. Correctness and completeness, while valuable in theoretical computer science, are not achievable in real-world software development. Heuristic methods thus always require thorough testing. XP emphasizes the role of defects in users' perceptions of a system, and that catching errors early through comprehensive tests at the unit, integration, and user level, minimizes costs [8]. Similarly, SBD has a heavy focus on testing via claims analysis (in the form of claims and tradeoff analysis); for each claim about the system, both positive and negative impacts are explicitly explored and reviewed over time. Another form of testing in SBD is careful observation of user interactions (in part because users do not always understand the implications of what they are doing).
4. Iteration and Redesign
Because design and development are tightly coupled, with agile methods there is an explicit assumption that the system will need to undergo continuous modifications, especially in initial development. The typical notion of ‘correcting one's mistakes' is not appropriate here; it comes with a strong implication that the correctness and completeness are possible, and this is contrary to the heurist's creed. Because technology and context are tightly coupled, the organization will change with the introduction of the new technology [3], possibly in ways that designer did not intend.
As with XP, SBD rejects the assumption that interactions can be 'gotten right' at the first attempt. iteration{C}{C}takes the form of a task-artifact cycle [9] in which development creates scenarios. These scenarios are then used to understand an interaction, which is used to develop a system feature (or artifact). Claims analysis of is used to evaluate the positives and negatives of that artifact, the claims are used to modify the scenarios, and the cycle continues. Iteration allows the heurist to break from any existing task to take advantage of the fortuitous occurrence.
5. Multiple Inputs
Both XP and SBD de-emphasize the traditional view of development as solitary effort; whether emphasizing participatory design, stakeholder review of regular deliverables, or pair programming, the goal is to constantly review all artifacts. This can be chalked up to the maxim that ‘two heads are better than one.'
Meeting of the Minds: An Integration Opportunity
Scott Ambler has begun drawing practical connections between his own work on Agile Modeling and User-Centered Design and User Experience Design [10]. Some work has been done in recognizing the overlap and working to integrate the two areas; in particular, Alexander and Maiden's Scenarios, Stories and Use Cases{C}{C}{C}{C} [11] has looked at story driven development from an integrated standpoint, although heuristics are not emphasized. Lynn Miller reported on the development organization at Alias, where separate agile and usability teams worked in interconnected, parallel development tracks, using an iterative, heuristic design approach [12].
{C}{C}{C}We have shown the areas where XP and SBD are similar and compatible ( ). The integration opportunity is one of synergy, the belief that the whole will be greater than the sum of the parts. By combining heuristic approaches in software engineering with compatible approaches in usability engineering, we will have a unified development process (an admirable goal), and also an improved process centered on people, their tasks, and their needs, but bounded by achievable limits and value considerations. Tables 2 and 3 show the complementary features of the two approaches. These are features that are not shared, but can provide new insights to the other.
To conclude that we have all of the answers denies the fundamental, exploratory tenet of heuristics. XP and SBD have brought advances to software engineering and usability engineering, but advancements still can and need to be made, especially through integrating these two methods. Aside from the core components of system and interaction, we have opportunities for better system integration into environments, better documentation, lower-cost and lower-risk projects, and longer-term system viability. Integrating and sharing between heuristic methods will improve these and other development aspects.
A Bold New Future
Uniting heuristic methods not only provides an avenue for improving those methods, but also creates the potential for faster adoption. Below we describe some mechanisms for developing and implementing combined methods in industry, education, and research.
In Industry
Industry often leads academia. Industry likes what works, and clearly XP and SBD work. We must encourage developers to strengthen their skills in both areas. One concern common in industry is funding. Fortunately, one of the principle advantages of heuristic methods is efficiency; this is clear from a management standpoint [13], and new models are emerging showing the cost advantage of agile methods [14] and usability [15].
Those software developers that believe in heuristic methods such as Extreme Programming should not let themselves be fooled by simple guidelines for interface development. These crutches are playing off developers' tendency to be drawn towards 'correct and complete' approaches. Instead, they should look for and be pointed towards heuristic interface design methods, such as those found in SBD.
In Research
Research communities learn from industry by monitoring it for success, failure, and opportunities for improvements. Researchers have time to analyze fine detail, the freedom to try sometimes aggressive approaches, and the requirement to explain and connect practice to theory. Industry can take advantage of this thoroughness. SBD has roots in industrial and academic research, but research based around real-world problems [9]. The balance found in grounded, practical research is a fruitful testing ground for new ideas, as it demands rigor in both academic thoroughness and industrial applicability.
Researchers must work for the cross-pollination of software and usability engineering. Coordination and collaboration are necessary, but research projects need not focus on method development in both areas. Many research projects require software development and user interface development, and the methods that have proved productive in the field can (and should) be adopted in the lab. This cross-use may be sufficient to begin cross-pollination, as the researcher will inevitably develop and refine the methods and heuristics, and publish these additions, alterations, and improvements.
In Education
We have long recognized that students leave typical computer science undergraduate programs without the skills to develop software in industry. Undergraduate programs in software development, software engineering, information systems, and HCI are at the forefront of the march towards addressing this problem. These programs are one venue for coursework that integrates agile methods and Scenario-Based Design. 'Usability and usefulness‘ will (where appropriate) supplant 'correctness and completeness‘ as a mantra. The ideal education will give students a good theoretical understand as well as practical experience with cutting-edge industry practice. One final note: the best possible implementation of these concepts as course material would be integrated, in either a single course or a coordinated pair of courses.
Final Comments
Heuristics are neither formal nor wrong; this fact violates the beliefs of many about appropriate, 'scientific' methods. Heuristics are powerful tools [16], and those of us who use heuristic methods would benefit greatly from joining together and learning from each other. The goal is not merely to justify ourselves and our work, but to advance the state of knowledge. Working towards an integrated community of discourse among heurists in industry, research, and education will help us toward both ends.
Acknowledgements
The authors would like to thank Steve Sawyer, Jack Carroll, and Scott Ambler for their comments on an earlier draft of this paper. They would also like to thank Arun Batchu, Robert Biddle, Fred George, Kyle Larson, Scott McCrickard, Mary Beth Rosson, and Laurie Williams for sharing their insight into methods.
About the Authors
Joshua B. Gross has spent ten years in the software development industry, as a software engineer, software architect, and project manager, working for Fortune 500 companies and startups. He holds an MS in software engineering from the University of St. Thomas. He is currently an independent consultant while he completes his PhD at Penn State.
John M. Daughtry III has been involved in software development since 2000. Currently, he is an Associate Research Engineer and PhD candidate at Penn State. He holds an MS in Information Sciences and Technology from Penn State, and a BS in Computer Science from North Carolina State University.
Jason Chong Lee is a PhD student at Virginia Tech exploring ways to integrate agile software development and usability engineering methodologies. He holds a BS from the University of Virginia, and is currently collaborating with Meridium, Inc. to research and develop agile usability methodologies.
References
[1] F. P. Brooks, Jr., The Mythical Man-Month: Addison-Wesley, 1975/1995.
[2] F. P. Brooks, Jr., "No Silver Bullet: Essence and Accidents of Software Engineering," IEEE Computer, vol. 20, pp. 10-19, 1987.
[3] W. J. Orlikowski, "Using Technology and Constituting Structures: A Practice Lens for Studying Technology in Organizations," Organization Science, vol. 11, pp. 404-428, 2000.
[4] J. Nielsen, Usability Engineering: Morgan Kaufmann, 1994.
[5] K. Beck, Extreme Programming Explained: Addison-Wesley Professional, 1999.
[6] A. Cockburn, Surviving Object-Oriented Projects: Addison-Wesley Professional, 1997.
[7] A. S. Koch, Agile Software Development: Evaluating the Methods for Your Organization: Artech House Publishers, 2004.
[8] B. Boehm, Software Engineering Economics: Prentice Hall, 1981.
[9] J. M. Carroll, Making Use: Scenario-Based Design of Human-Computer Interaction. Cambridge, MA: The MIT Press, 2000.
[10] S. Ambler, "Agile Usability: User Experience Activities on Agile Development Projects."
[11] I. Alexander and N. Maiden, Scenarios, Stories and Use Cases: John Wiley amp; Sons, 2004.
[12] L. Miller, "Case Study of Customer Input for a Successful Product," presented at Agile 2005, 2005.
[13] M. Poppendieck and T. Poppendieck, Lean Software Development: An Agile Toolkit for Software Development Managers: Addison-Wesley, 2003.
[14] M. Denne and J. Cleland-Huang, Software By Numbers: Low-Risk, High-Return Development: Prentice Hall, 2003.
[15] R. G. Bias and D. J. Mayhew, Cost-Justifying Usability, 2 ed: Morgan Kaufmann, 2005.
[16] G. Gigerenzer and P. M. Todd, Simple Heuristics that Make Us Smart. Oxford, UK: Oxford University Press, 1999.
