The 10 Minute MBA (Unaccredited)

After spending a significant amount of time getting an MBA, if find it interesting that the concepts that I thought were the most insightful, seem to be the ones least followed in the business world. Here’s my list in no particular order:

1) Be careful what you base the employee incentives on, because that is exactly what they will do.
2) Money now is worth more than money later. The degree of which is directly related to the likelihood that you get the money later.
3) For an individual investor, the stock market is pretty much efficient. A dartboard will pick stocks as better than an individual will ½ the time. The bigger the company, the truer this is.
4) When making a decision, only base the costs on the marginal costs affected by the decision, not allocated fixed costs that the decision does not impact.
5) The throughput (speed) of any process can ONLY go as fast as the throughput of its slowest component.
6) Marketing is not the same as sales. Marketing is creating demand for your product, and is focused on both your products and your customers. Sales is the last step in the process, getting the customer to make a decision.
That’s about it; everything else is just a detail you learn along the way. If you think about them, it’s pretty hard to argue. But these concepts seem to be very elusive in the corporate world, and especially IT.

Lets go make some Corollas!

I know, I know, I've been negligent in my blogging responsibility. The world wants to hear my voice, but I lay silent. I apologize.

Today's topic is performance.

Nearly every developer is obsessed with making their applications go as fast as possible. I guess its a badge of honor to say we are faster than our competitor. Mine is faster, so I must be smarter.

Here's something you won't hear many people admit: My apps aren't fast.

I rarely use caching. I don't optimize every line of code.

But I will say: My apps are fast enough!

Most software engineers are busy designing their customers a Jaguar. Fast fast fast.

I like designing the Corolla. Economical to buy, reliable, and easy to maintain. Simple design, fewer tricks to speed things up, test, test, test.

I was recently reviewing an application with a developer who noticed that we are loading the same data often from the database. This data rarely changes, and if it does, its a big deal. Its a great place to cache the data on the server. Additionally, the server is set up so its not as scalable as possible, that needs to be fixed to make it more scalable.

My response: No need. 1st, every user that going to use the app is currently using it. We have scaled and it works fine. Secondly, those things all add complexity. If a bug is found, you have another layer to determine where the bug resides - Did it cache right? Did the session server fail? Avoid that complexity. The application is as fast as the users need. If the customer desires speed, we can work on it. Until then, no request, no complexity.

The moral here is that while its fun to build with the fastest, coolest techniques possible, they have a short term and long term costs. The challenge is building what your customer needs, not what you want. If the requirement exists for blazing speed, then design a system around that need, however there are costs to that performance. 90% of the time, reliability and low maintenance costs trump the speed factor.

The Jag might be faster than the Toyota, but they both go the speed limit, and the Corolla costs half as much, gets 35 miles per gallon and spends a lot less time in the shop.

We're good... Right?

How do you know when you are performing at your best? What measure is appropriate for performance of software development?

With the waterfall mindset, a project is performing at peak efficiency when the project comes in on-time and under budget. However, this really says little about the efficiency of the software development process. Two identical projects may be greatly successful in one company, and a dismal failure in another simply because they started with different timelines and budgets.

I recently read a book by David Anderson which gives an interesting insight into this problem. The measurement should be the throughput of value through the software development process. Its an interesting concept which I've been thinking about lately. At first I was very skeptical of the 'Assembly' line metaphor for software development, since it is at its heart, still very innovative. However, as the author lays it out, I think several parallels can be made.

The theme is that the software development process should examined using many of the 'Just-In-Time' manufacturing concepts from operations management and research, namely the 'Theory of Constraints' (TOC) pioneered by Eli Goldratt. The idea is that the output of your system is governed by the bottleneck. All other components of the system need to operate at the same pace as the constraint, or you are just generating waste.

While the TOC is a valid observation in software development, and certainly should be considered when managing your process, I thought the measurement of productivity was the most interesting. In my experience, managers are looking at a timeline orient view of how effective their employees are. On some projects, I've been able to easily meet the expectations simply because the schedules were padded. Alternatively, I've been subject to missing deadlines that were completely unrealistic, and performance judged poorly.

Instead of relying on assumptions made at the beginning of the project with poor information, Anderson lays out the idea that throughput of value to the customer is the best measure of performance. Every feature that the customer wants in the system is kept on the project backlog, and after an iteration, the completed features are removed from the system and recorded. The number of features completed within an iteration represents the value created in the iteration. By keeping track of the features in 'Inventory' (The Features on the backlog) and the features in Finished Goods (In Production) the manager can track the performance of a software development system.

If the quality of the output is low, the team will spend more time doing rework that does not enhance value, thus there is an incentive to send out high quality code to the users.

I would recommend reading the book to get a better description of the throughput of the software development team.

I recommend a couple of additions to this concept of throughput.
1) All features are not created equally: Have the customers assign a value rating (ie. 1-5) to a feature. This will help the team prioritize the backlog and work on the highest value items first. The throughput of the system will take into account the value generated by each feature.
2) Measure the complexity of a feature. Either by breaking a feature out into several sub-features of equivalent difficulty, or by rating each feature's complexity (ie 1-5) this helps a manager gauge the level of effort put out during an iteration.
3) Value in uncertainty: If a feature has a lot of uncertainty in either the functional requirements or technology risk, value is derived by gaining more information and reducing the uncertainty.
4) Option Value: Some features are dependent on the completion of others. The intermediate feature may not present a lot of value in itself, other than you know have the ability to execute another feature.

So the overall performance of a team is the value create over a period of time, considering the effort the team put into a feature set, and any uncertainty reduced or new features that can be put into future releases.

To make a team...

Next to getting customer involvement, putting together the right team is the most important component to a successful project. In my opinion, there are 2 traits that rise to the top when selecting team members. The ability to accomplish the tasks the job requires, and the ability to work as a member of the team.

In my travels, the failure of a team to 'gel' comes down to either of these points.

If anyone else has gone through business school, we now have to make a 2x2 grid.

The Stars: A person that's easy to work with that has the right expertise to fill the role on the team. You should obviously try to fill you team with these people. Stars are will to accept accountability for both good and bad are accept constructive criticism.

Some people have the aptitude and experience to get the job done, but working in a team environment they flounder because things must be done 'their way or the highway' I usually call these people the 'Lone Wolf'. They will be much more successful on single person projects than larger team oriented efforts. The Lone Wolf introduces risks to the project because they often fail to adhere to any development process you may have (Source Control, Unit Testing, etc, etc).

The Charlatan: The charlatan has neither the teamworking ability nor the expertise to get the job done. They will go to great lengths to try to hide their incompetence, often by alienating teammates and striving for attention when something gets accomplished. As a manager, it can be difficult to seperate your stars from the charlatans, since the charlatans are so good an managing up (That's why there are in the position they are in). However, the best test is how a person takes critcism and accept accountability.

The Bumbler: Usually an easy going person who while putting in an honest effort, often comes up short on results. The bumbler adds less risk, as opposed to the Charlatan or Lone Wolf, since they don't divide the team and usually adhere to the process used by the group.

My next rant is that a team needs roles and role players. My definition of a role is a position in a project with accountabilities. At the end of the day, if you are accountable for nothing that adds value to the customer, you have no business being part of the project team. Having the accountability within the team is very important to keeping a team motivated. If one person or group of people on the team have to burden all the responsibility, its terribly demotivating since everything gets dumped on them.

Finally, a team consists of role players. A football team would fare very poorly over a season if you selected the 11 best Running Backs available to make up your team. A team needs the offensive linemen who do the unglamorous grunt work that allows the RB to get to the end zone. In software development, its the same way. Having a team of the 5 best programmers available will not lead to success. You may need an architect who can see the big picture, you may need a project manager who knows agile development, you may need some analysts to codify the users requests in a clear fashion so the developers can focus on development. When planning a project, you need to put some real thought into these roles and how the role player is accountable for the output of each role.

We're Agile... Right?

"We don't do much requirements analysis because its too much overhead. "
"We really need to hit our deadline in a few months, so we will make the most with the resources we have."
"Testing is a luxury, we just don't have time for that."
"Our developers have really been letting us down and dogging the project"

These are all reasons I have heard that a team is developing with an Agile philosophy, however none of the projects were Agile at all.

Where do you draw the line between agile and chaos? All too often a project is managed under the assumption it is agile when it reality it is simply chaotic. In the wrong hands, the tenets of agile development can create an incredibly dysfunctional environment. Agile development requires more discipline to do the right things than a highly structured waterfall approach. Here is a list of questions I used to try to clear up the confusion.

1) Are you managing for VALUE? Are the management decisions (features/timeliness/resources) being made to enhance value, or are they serving another purpose. If the timeline is ultimately hard, then focus on providing the highest value items to the customers in allotted time and deliver a high-value working product. Do not feel you can just squeeze out some extra productivity out of the staff and expect everything to work out. It’s a mistake that's been made over and over again. Once the team is set, and the deadlines are made, then you can only manage an extra 10% or so out of the team.

2) Is the end goal Speed or Efficiency?
Most people confuse these two concepts. Getting a project done faster does not correlate well to the efficiency of the project. Some projects appear to move too slowly, but the lack of defects greatly reduces the cost of a project since most defects costs many more times than developing correctly the first time. The best analogy is driving to work. Do you want to be efficient or get home as fast as possible? You could choose to drive as fast as humanly possible, but you end up paying more for gas/speeding tickets/ etc and take on more risks, such as crashing head on into oncoming traffic. I choose efficiency.

3) Are the customers engaged?
Are your customers an integral part of the process? If not, how can you be sure you are developing the highest value items at any given time?

4) Do you have defined iterations?
I think have predefined iterations (Of a suitable length) are extremely important. Without any defined iterations, it’s too easy to drift into complacency or drift into chaos where the feature list is constantly updated and the development team has nothing to focus on.

5) Do you always have a working, production ready piece of software?
If you are managing the project in such a way that you are adding many partially tested features at one time you are not Agile. Every iteration needs a fully tested piece of software associated with it. If you are adding features that 'We'll clean up later' you are doing a great disservice to the project by introducing inefficiencies and uncertainty.

There are some more, but those covers most of the things that help me draw the distinction between chaos and Agile software development. If you can strongly say yes to all the above items, you need to address this issues ASAP. Being chaotic and explaining it as agility only hurts the agile software development community's credibility.

How do we get there from here?

OK, last time I talked in vague terms about Agile software development, uncertainty, yada, yada, yada. But chances are if you are reading thing you already know what it means to be agile, and you even may know the benefits, but the one question I get asked the most is: How?

The first answer is that it is not easy. Many companies seem to be hardwired against agile methods from the start. Budgeting, bonuses, staffing all support a more linear waterfall project plan.

The best advice is to start small. Pick a project, preferrable a smaller project with large corporate visibility, and choose to be agile. Pick a method, such as Scrum, FDD, or even XP and a starting point and run the project using those methods as a guideline.

The first task after picking a project management guideline is picking an iteration length for the project. I would start out with 3 weeks and play it by ear. I like 2 week iterations, but they can start to feel fairly compressed, especially early on a project when you are ramping up.

From a technology side, I feel getting the environment set up to support agile methods is vital. I would highly recommend starting with a test-first development philosophy. Test-first development warrants is own topic, but in my opinion it drives 3 main benefits: Faster Development, Change with Confidence, and Quality Requirements.

The main drawback I hear to test-first development is that it requires more effort from the developers. If the metric for effort is lines of code, then naturally it requires more effort. However, if the metric is time, then I am not convinced. Based on my industry experience, code actually goes faster when unit testing coverage is higher, rather than lower. While this may initially seems counter-intuitive, the improvement in development time comes from catching bugs sooner rather than later. The further down the line a bug is discovered, the more effort is required to fix it. A good set of unit tests will catch 60% or so of the bugs as they are written, where the code is fresh in the mind of the developer.

One the philosphies of Agile Development is to embrace change. With a good set of unit tests, you can make changes to our code with confidence that the bugs will not cause problems in other parts of the system. These regression bugs, (As as I like to call them "Whack-a-Mole" bugs) can cause series headaches for developers and result in many lost hours. By investing in a set of unit tests, these bugs are caught sooner rather than later saving time throught development.

The last main advantage of the test-first technique is it helps to drive better requirements. By asking the question: "How can we test this?" it forces the analysts and customers to think about the feature at a deeper level. By going through the excercise, additional questions are usually raised that may not have been raised until much later in the process.

The next step is to set up a continuous integration server. There are several products available that accomplish this, including Cruise Control, which I am most familiar with. The idea behind continuous integration is that there is always a working copy of the build running at all times. Whenever a change is made to source control, a new build is made, unit tests are run, and it is deployed to the test area. If any of the steps fail, the entire team is notified through a tooltray icon, and the team works together to get the build up and running again.

The effect of continous integration is that integration related bugs are caught as soon as they are checked into source control. On many projects without Continuous Integration, there's a mad rush on build day to get changes into the build, and then a frantic effort to get the changes working together. By continously building the software, these frantic efforts are reduces and deployments can go smoothly.

The last thing I would recommend on your first Agile project is find a way to engage your customer. Getting regular face to face meetings is extremely important along with regular phone conferences, if necessary. Getting the face to face contact improves the communication, and just as important, it builds trust. Building trust allows the users to prioritize their requirements. All too often when a user hears "We'll but it on the next iteration" they interpret that to mean "You will never get that". After demonstrating in a couple iterations they will eventually see their lower priority items, they are much more willing to place items up the future task list and tackle them when they come up.

Welcome

I've dedicated this small corner of the web to spew forth my random thoughts on my chosen profession: Software Development.

First a little background:
I own a small consulting company, Agillence Software Inc, that specializes in helping companies utilize Agile Software development techniques and philosophies. I started in the field in this field right out of college with a degree in Computer Science. My studies were very much bits and bytes, which laid a nice foundation for a career in software development. Some years out of college I decided to round out my education and got an MBA. I will talk more about each of these experiences some other day, but for now I will stick with the introduction.

My consulting focuses on helping organizations embrace Agile Software development. Its an uphill battle, since companies seem to be wired to work against any progress. From budget battles, to timeliness, to corporate momentum, there are many factors that make this extremely difficult.

OK, now that you know a little reason why I'm spending time doing this, I'll answer a question some of you may have: What is agile software development? Well, to start go here http://agilemanifesto.org/

It outlines what the general philosophy of agile is, but I like think of it on even more abstract terms:

In its barest sense, Agile Development manages uncertainty. Software development projects are ripe with uncertainty. It is not like building a bridge. The laws of physics won't change at the end of quarter 2 where an alternate universe just released a new set of physical laws that give them a huge competitive advantage.

The change and uncertainty can be managed in a variety of ways, the simplest being to pretend the uncertainties do not exist. You can spend a lot of time planning the work, making very precise (but wildly inaccurate) estimates as to the level of effort needed. Once the plan is agreed to, make every effort to prevent change to the plan. Unfortunately, the uncertainty really isn't managed, it is ignored.

Agile development focuses efforts on the most certain component at any given time. Let the customers drive what they feel is the most valuable component of the project. Put off decisions that do not have to be made now, and make them when we have more information about the problem. When the decision is made with better information, it increases the value of the decision. That does not mean, in retrospect, that every decision will be the best, but taken as a whole, the system will run better.

"Managing uncertainty" That doesn't sound like the typical explanation of Agile Software Development. That doesn't sound like Software at all. In sounds like something more suited to the insurance or financial industry. Well, truth be told, when I went back to school for the MBA, I studied finance. But, like a moth to a light, I was drawn back to Software Development. Despite being drawn back into Software, Finance and economics are very interesting subjects that apply to a variety of fields because under all the math and theory, they are social sciences.

Which leads me to my first 'law' of software development:
Gillespie Fundamental Law #1 - "Its not the technology, silly."
Under all the bits and bytes, software development is a social activity. Ironically, an industry with the reputation of employing some of the most socially limited individuals (present company included), the primary success factor of any project is the ability to work together. Nearly all major problems I've seen an any given project can be traced back to people. Sure, a server may meltdown, or perfomance may be poor, but most of those problems are relatively easy to fix. The harder problems to fix are when are company cannot even get to the point of installing working software on a server. In the current business environment, if a company cannot utilize a software infrastructure, it will fail.
Corollary 1a - People are not interchangable parts. The output of a developer can vary my more than 10x between people of similar experience. If you get quality people on a project, the chances of success go up exponentially.
Corollary 1b - You cannot use process as a substitute for talent.

Gillespie Fundamental Law #2 - "There is always uncertainty."
You cannot plan away uncertainty. You can hedge against most of the types of uncertainty on software projects. The best you can do is accept it and manage your project accordingly.

Gillespie Fundamental Law #3 - "Everything comes at a cost."
Every activity comes at the cost of doing something else.
Corollary 3a - The value of every activity must be justifiable. Every action must generate value for the company. Creating volumes of documentation no one will ever read creates no value.

That's enough for now. There are more laws, but I will let you chew on those for a while.

-Mike