We're good... Right?

How do you know when you are performing at your best? What measure is appropriate for performance of software development?

With the waterfall mindset, a project is performing at peak efficiency when the project comes in on-time and under budget. However, this really says little about the efficiency of the software development process. Two identical projects may be greatly successful in one company, and a dismal failure in another simply because they started with different timelines and budgets.

I recently read a book by David Anderson which gives an interesting insight into this problem. The measurement should be the throughput of value through the software development process. Its an interesting concept which I've been thinking about lately. At first I was very skeptical of the 'Assembly' line metaphor for software development, since it is at its heart, still very innovative. However, as the author lays it out, I think several parallels can be made.

The theme is that the software development process should examined using many of the 'Just-In-Time' manufacturing concepts from operations management and research, namely the 'Theory of Constraints' (TOC) pioneered by Eli Goldratt. The idea is that the output of your system is governed by the bottleneck. All other components of the system need to operate at the same pace as the constraint, or you are just generating waste.

While the TOC is a valid observation in software development, and certainly should be considered when managing your process, I thought the measurement of productivity was the most interesting. In my experience, managers are looking at a timeline orient view of how effective their employees are. On some projects, I've been able to easily meet the expectations simply because the schedules were padded. Alternatively, I've been subject to missing deadlines that were completely unrealistic, and performance judged poorly.

Instead of relying on assumptions made at the beginning of the project with poor information, Anderson lays out the idea that throughput of value to the customer is the best measure of performance. Every feature that the customer wants in the system is kept on the project backlog, and after an iteration, the completed features are removed from the system and recorded. The number of features completed within an iteration represents the value created in the iteration. By keeping track of the features in 'Inventory' (The Features on the backlog) and the features in Finished Goods (In Production) the manager can track the performance of a software development system.

If the quality of the output is low, the team will spend more time doing rework that does not enhance value, thus there is an incentive to send out high quality code to the users.

I would recommend reading the book to get a better description of the throughput of the software development team.

I recommend a couple of additions to this concept of throughput.
1) All features are not created equally: Have the customers assign a value rating (ie. 1-5) to a feature. This will help the team prioritize the backlog and work on the highest value items first. The throughput of the system will take into account the value generated by each feature.
2) Measure the complexity of a feature. Either by breaking a feature out into several sub-features of equivalent difficulty, or by rating each feature's complexity (ie 1-5) this helps a manager gauge the level of effort put out during an iteration.
3) Value in uncertainty: If a feature has a lot of uncertainty in either the functional requirements or technology risk, value is derived by gaining more information and reducing the uncertainty.
4) Option Value: Some features are dependent on the completion of others. The intermediate feature may not present a lot of value in itself, other than you know have the ability to execute another feature.

So the overall performance of a team is the value create over a period of time, considering the effort the team put into a feature set, and any uncertainty reduced or new features that can be put into future releases.

To make a team...

Next to getting customer involvement, putting together the right team is the most important component to a successful project. In my opinion, there are 2 traits that rise to the top when selecting team members. The ability to accomplish the tasks the job requires, and the ability to work as a member of the team.

In my travels, the failure of a team to 'gel' comes down to either of these points.

If anyone else has gone through business school, we now have to make a 2x2 grid.

The Stars: A person that's easy to work with that has the right expertise to fill the role on the team. You should obviously try to fill you team with these people. Stars are will to accept accountability for both good and bad are accept constructive criticism.

Some people have the aptitude and experience to get the job done, but working in a team environment they flounder because things must be done 'their way or the highway' I usually call these people the 'Lone Wolf'. They will be much more successful on single person projects than larger team oriented efforts. The Lone Wolf introduces risks to the project because they often fail to adhere to any development process you may have (Source Control, Unit Testing, etc, etc).

The Charlatan: The charlatan has neither the teamworking ability nor the expertise to get the job done. They will go to great lengths to try to hide their incompetence, often by alienating teammates and striving for attention when something gets accomplished. As a manager, it can be difficult to seperate your stars from the charlatans, since the charlatans are so good an managing up (That's why there are in the position they are in). However, the best test is how a person takes critcism and accept accountability.

The Bumbler: Usually an easy going person who while putting in an honest effort, often comes up short on results. The bumbler adds less risk, as opposed to the Charlatan or Lone Wolf, since they don't divide the team and usually adhere to the process used by the group.

My next rant is that a team needs roles and role players. My definition of a role is a position in a project with accountabilities. At the end of the day, if you are accountable for nothing that adds value to the customer, you have no business being part of the project team. Having the accountability within the team is very important to keeping a team motivated. If one person or group of people on the team have to burden all the responsibility, its terribly demotivating since everything gets dumped on them.

Finally, a team consists of role players. A football team would fare very poorly over a season if you selected the 11 best Running Backs available to make up your team. A team needs the offensive linemen who do the unglamorous grunt work that allows the RB to get to the end zone. In software development, its the same way. Having a team of the 5 best programmers available will not lead to success. You may need an architect who can see the big picture, you may need a project manager who knows agile development, you may need some analysts to codify the users requests in a clear fashion so the developers can focus on development. When planning a project, you need to put some real thought into these roles and how the role player is accountable for the output of each role.

We're Agile... Right?

"We don't do much requirements analysis because its too much overhead. "
"We really need to hit our deadline in a few months, so we will make the most with the resources we have."
"Testing is a luxury, we just don't have time for that."
"Our developers have really been letting us down and dogging the project"

These are all reasons I have heard that a team is developing with an Agile philosophy, however none of the projects were Agile at all.

Where do you draw the line between agile and chaos? All too often a project is managed under the assumption it is agile when it reality it is simply chaotic. In the wrong hands, the tenets of agile development can create an incredibly dysfunctional environment. Agile development requires more discipline to do the right things than a highly structured waterfall approach. Here is a list of questions I used to try to clear up the confusion.

1) Are you managing for VALUE? Are the management decisions (features/timeliness/resources) being made to enhance value, or are they serving another purpose. If the timeline is ultimately hard, then focus on providing the highest value items to the customers in allotted time and deliver a high-value working product. Do not feel you can just squeeze out some extra productivity out of the staff and expect everything to work out. It’s a mistake that's been made over and over again. Once the team is set, and the deadlines are made, then you can only manage an extra 10% or so out of the team.

2) Is the end goal Speed or Efficiency?
Most people confuse these two concepts. Getting a project done faster does not correlate well to the efficiency of the project. Some projects appear to move too slowly, but the lack of defects greatly reduces the cost of a project since most defects costs many more times than developing correctly the first time. The best analogy is driving to work. Do you want to be efficient or get home as fast as possible? You could choose to drive as fast as humanly possible, but you end up paying more for gas/speeding tickets/ etc and take on more risks, such as crashing head on into oncoming traffic. I choose efficiency.

3) Are the customers engaged?
Are your customers an integral part of the process? If not, how can you be sure you are developing the highest value items at any given time?

4) Do you have defined iterations?
I think have predefined iterations (Of a suitable length) are extremely important. Without any defined iterations, it’s too easy to drift into complacency or drift into chaos where the feature list is constantly updated and the development team has nothing to focus on.

5) Do you always have a working, production ready piece of software?
If you are managing the project in such a way that you are adding many partially tested features at one time you are not Agile. Every iteration needs a fully tested piece of software associated with it. If you are adding features that 'We'll clean up later' you are doing a great disservice to the project by introducing inefficiencies and uncertainty.

There are some more, but those covers most of the things that help me draw the distinction between chaos and Agile software development. If you can strongly say yes to all the above items, you need to address this issues ASAP. Being chaotic and explaining it as agility only hurts the agile software development community's credibility.