Many claims are made about how certain tools, technologies, and practices improve software development. But which claims are verifiable, and which are merely wishful thinking? In this book, leading thinkers such as Steve McConnell, Barry Boehm, and Barbara Kitchenham offer essays that uncover the truth and unmask myths commonly held among the software development community. Their insights may surprise you.
Are some programmers really ten times more productive than others?
Does writing tests first help you develop better code faster?
Can code metrics predict the number of bugs in a piece of software?
Do design patterns actually make better software?
What effect does personality have on pair programming?
What matters more: how far apart people are geographically, or how far apart they are in the org chart?
Contributors include:
Jorge Aranda
Tom Ball
Victor R. Basili
Andrew Begel
Christian Bird
Barry Boehm
Marcelo Cataldo
Steven Clarke
Jason Cohen
Robert DeLine
Madeline Diep
Hakan Erdogmus
Michael Godfrey
Mark Guzdial
Jo E. Hannay
Ahmed E. Hassan
Israel Herraiz
Kim Sebastian Herzig
Cory Kapser
Barbara Kitchenham
Andrew Ko
Lucas Layman
Steve McConnell
Tim Menzies
Gail Murphy
Nachi Nagappan
Thomas J. Ostrand
Dewayne Perry
Marian Petre
Lutz Prechelt
Rahul Premraj
Forrest Shull
Beth Simon
Diomidis Spinellis
Neil Thomas
Walter Tichy
Burak Turhan
Elaine J. Weyuker
Michele A. Whitecraft
Laurie Williams
Wendy M. Williams
Andreas Zeller
Thomas Zimmermann
General Principles of Searching For and Using Evidence
Chapter 1 The Quest for Convincing Evidence
In the Beginning
The State of Evidence Today
Change We Can Believe In
The Effect of Context
Looking Toward the Future
References
Chapter 2 Credibility, or Why Should I Insist on Being Convinced?
How Evidence Turns Up in Software Engineering
Credibility and Relevance
Aggregating Evidence
Types of Evidence and Their Strengths and Weaknesses
Society, Culture, Software Engineering, and You
Acknowledgments
References
Chapter 3 What We Can Learn from Systematic Reviews
An Overview of Systematic Reviews
The Strengths and Weaknesses of Systematic Reviews
Systematic Reviews in Software Engineering
Conclusion
References
Chapter 4 Understanding Software Engineering Through Qualitative Methods
What Are Qualitative Methods?
Reading Qualitative Research
Using Qualitative Methods in Practice
Generalizing from Qualitative Results
Qualitative Methods Are Systematic
References
Chapter 5 Learning Through Application: The Maturing of the QIP in the SEL
What Makes Software Engineering Uniquely Hard to Research
A Realistic Approach to Empirical Research
The NASA Software Engineering Laboratory: A Vibrant Testbed for Empirical Research
The Quality Improvement Paradigm
Conclusion
References
Chapter 6 Personality, Intelligence, and Expertise: Impacts on Software Development
How to Recognize Good Programmers
Individual or Environment
Concluding Remarks
References
Chapter 7 Why Is It So Hard to Learn to Program?
Do Students Have Difficulty Learning to Program?
What Do People Understand Naturally About Programming?
Making the Tools Better by Shifting to Visual Programming
Contextualizing for Motivation
Conclusion: A Fledgling Field
References
Chapter 8 Beyond Lines of Code: Do We Need More Complexity Metrics?
Surveying Software
Measuring the Source Code
A Sample Measurement
Statistical Analysis
Some Comments on the Statistical Methodology
So Do We Need More Complexity Metrics?
References
Specific Topics in Software Engineering
Chapter 9 An Automated Fault Prediction System
Fault Distribution
Characteristics of Faulty Files
Overview of the Prediction Model
Replication and Variations of the Prediction Model
Building a Tool
The Warning Label
References
Chapter 10 Architecting: How Much and When?
Does the Cost of Fixing Software Increase over the Project Life Cycle?
How Much Architecting Is Enough?
Using What We Can Learn from Cost-to-Fix Data About the Value of Architecting
So How Much Architecting Is Enough?
Does the Architecting Need to Be Done Up Front?
Conclusions
References
Chapter 11 Conway’s Corollary
Conway’s Law
Coordination, Congruence, and Productivity
Organizational Complexity Within Microsoft
Chapels in the Bazaar of Open Source Software
Conclusions
References
Chapter 12 How Effective Is Test-Driven Development?
The TDD Pill—What Is It?
Summary of Clinical TDD Trials
The Effectiveness of TDD
Enforcing Correct TDD Dosage in Trials
Cautions and Side Effects
Conclusions
Acknowledgments
General References
Clinical TDD Trial References
Chapter 13 Why Aren’t More Women in Computer Science?
Why So Few Women?
Should We Care?
Conclusion
References
Chapter 14 Two Comparisons of Programming Languages
A Language Shoot-Out over a Peculiar Search Algorithm
Plat_Forms: Web Development Technologies and Cultures
So What?
References
Chapter 15 Quality Wars: Open Source Versus Proprietary Software
Past Skirmishes
The Battlefield
Into the Battle
Outcome and Aftermath
Acknowledgments and Disclosure of Interest
References
Chapter 16 Code Talkers
A Day in the Life of a Programmer
What Is All This Talk About?
A Model for Thinking About Communication
References
Chapter 17 Pair Programming
A History of Pair Programming
Pair Programming in an Industrial Setting
Pair Programming in an Educational Setting
Distributed Pair Programming
Challenges
Lessons Learned
Acknowledgments
References
Chapter 18 Modern Code Review
Common Sense
A Developer Does a Little Code Review
Group Dynamics
Conclusion
References
Chapter 19 A Communal Workshop or Doors That Close?
Doors That Close
A Communal Workshop
Work Patterns
One More Thing…
References
Chapter 20 Identifying and Managing Dependencies in Global Software Development
Why Is Coordination a Challenge in GSD?
Dependencies and Their Socio-Technical Duality
From Research to Practice
Future Directions
References
Chapter 21 How Effective Is Modularization?
The Systems
What Is a Change?
What Is a Module?
The Results
Threats to Validity
Summary
References
Chapter 22 The Evidence for Design Patterns
Design Pattern Examples
Why Might Design Patterns Work?
The First Experiment: Testing Pattern Documentation
The Second Experiment: Comparing Pattern Solutions to Simpler Ones
The Third Experiment: Patterns in Team Communication
Lessons Learned
Conclusions
Acknowledgments
References
Chapter 23 Evidence-Based Failure Prediction
Introduction
Code Coverage
Code Churn
Code Complexity
Code Dependencies
People and Organizational Measures
Integrated Approach for Prediction of Failures
Summary
Acknowledgments
References
Chapter 24 The Art of Collecting Bug Reports
Good and Bad Bug Reports
What Makes a Good Bug Report?
Survey Results
Evidence for an Information Mismatch
Problems with Bug Reports
The Value of Duplicate Bug Reports
Not All Bug Reports Get Fixed
Conclusions
Acknowledgments
References
Chapter 25 Where Do Most Software Flaws Come From?
Studying Software Flaws
Context of the Study
Phase 1: Overall Survey
Phase 2: Design/Code Fault Survey
What Should You Believe About These Results?
What Have We Learned?
Acknowledgments
References
Chapter 26 Novice Professionals: Recent Graduates in a First Software Engineering Job
Study Methodology
Software Development Task
Strengths and Weaknesses of Novice Software Developers
Reflections
Misconceptions That Hinder Learning
Reflecting on Pedagogy
Implications for Change
References
Chapter 27 Mining Your Own Evidence
What Is There to Mine?
Designing a Study
A Mining Primer
Where to Go from Here
Acknowledgments
References
Chapter 28 Copy-Paste as a Principled Engineering Tool
An Example of Code Cloning
Detecting Clones in Software
Investigating the Practice of Code Cloning
Our Study
Conclusions
References
Chapter 29 How Usable Are Your APIs?
Why Is It Important to Study API Usability?
First Attempts at Studying API Usability
If At First You Don’t Succeed...
Adapting to Different Work Styles
Conclusion
References
Chapter 30 What Does 10x Mean? Measuring Variations in Programmer Productivity
Individual Productivity Variation in Software Development
Issues in Measuring Productivity of Individual Programmers
Team Productivity Variation in Software Development
Andy Oram is an editor at O'Reilly Media, a highly respected book publisher and technology information provider. An employee of the company since 1992, Andy currently specializes in free software and open source technologies. His work for O'Reilly includes the first books ever published commercially in the United States on Linux, and the 2001 title Peer-to-Peer. His modest programming and system administration skills are mostly self-taught.
Greg Wilson has worked on high-performance scientific computing, data visualization, and computer security, and is currently project lead at Software Carpentry (http://software-carpentry.org). Greg has a Ph.D. in Computer Science from the University of Edinburgh, and has written and edited several technical and children's books, including "Beautiful Code" (O'Reilly, 2007).
After thoroughly enjoying Oram and Wilson's Beautiful Code, I was very much anticipating this next amalgamation of the thoughts of some of the industry's leading voices. Greg was a professor of Computer Science at U of T, my alma mater, and has been a great advisor to me in my career. He currently spends his time working on Software Carpentry, an effort to teach programming practices to scientists.
Does TDD work? Is Python better than Java? Are good programmers really 10 times more productive? How do you measure programming performance? Is open source software better than proprietary software? Do design patterns work in practice?
It is enough to whisper one of these questions around a group of programmers to begin an impassioned debate. Can anyone actually be right? How can we answer these seemingly subjective questions? Making Software attempts to find credible qualitative and quantitative evidence to answer such questions. It is no longer adequate to present arguments without showing the facts. It is time we apply the scientific method to these questions, gather some solid evidence and impartially evaluate the implications.
In 2009, Thoughtworks' Martin Fowler gave a talk entitled "Three Years of Real-World Ruby" in which he presented the results of the 41 Ruby projects his company had worked on during that period. He surveyed programmers to see how they felt about working with the language. He argued for the adoption of Ruby by showing evidence of its success within his organization. This was fascinating real-world study and is exactly what the authors of Making Software would like to see more of.
Internally many software development companies are gathering evidence of their failures and successes in hopes of finding the magical formula for developing quality software fast. Few companies are willing to release such information to the public. This is part of why we don't have an abundance of empirical studies on software development. However, in recent years, more such studies have been appearing. High quality studies are out there waiting to be referenced.
My favourite chapter of the book was Steve McConnell's "What Does 10x Mean? Measuring Variations in Programmer Productivity". McConnell, of course, being famous for the highly successful Code Complete and other popular works such as Rapid Development and Software Estimation: Demystifying the Black Art. In this essay, McConnell provides substantial evidence that the "order of magnitude" difference in programming productivity is not merely anecdotal but a provable hypothesis. I've definitely seen this difference in productivity during my time at Electronic Arts; there were numerous experienced game developers who were clearly getting things done much faster than I could. However, I would imagine that familiarity with the code base played a major factor in that particular example. More interesting are the studies where programming teams are given new projects to work on and they are more or less on an even playing field. I find this is a very hot topic as such research may reveal the productivity secrets of the elite programmers. Now that's information us mere mortals are dying to hear.
What makes this book interesting is that it attempts to treat issues in software development in the same manner that we would treat anthropological issues. The authors take on controversial topics that programmers love to argue about and give us meaningful evidence to further the debates. Making Software is a great read for all programmers, regardless of whether you are 10x more productive or not.
