How do we get there from here?

OK, last time I talked in vague terms about Agile software development, uncertainty, yada, yada, yada. But chances are if you are reading thing you already know what it means to be agile, and you even may know the benefits, but the one question I get asked the most is: How?

The first answer is that it is not easy. Many companies seem to be hardwired against agile methods from the start. Budgeting, bonuses, staffing all support a more linear waterfall project plan.

The best advice is to start small. Pick a project, preferrable a smaller project with large corporate visibility, and choose to be agile. Pick a method, such as Scrum, FDD, or even XP and a starting point and run the project using those methods as a guideline.

The first task after picking a project management guideline is picking an iteration length for the project. I would start out with 3 weeks and play it by ear. I like 2 week iterations, but they can start to feel fairly compressed, especially early on a project when you are ramping up.

From a technology side, I feel getting the environment set up to support agile methods is vital. I would highly recommend starting with a test-first development philosophy. Test-first development warrants is own topic, but in my opinion it drives 3 main benefits: Faster Development, Change with Confidence, and Quality Requirements.

The main drawback I hear to test-first development is that it requires more effort from the developers. If the metric for effort is lines of code, then naturally it requires more effort. However, if the metric is time, then I am not convinced. Based on my industry experience, code actually goes faster when unit testing coverage is higher, rather than lower. While this may initially seems counter-intuitive, the improvement in development time comes from catching bugs sooner rather than later. The further down the line a bug is discovered, the more effort is required to fix it. A good set of unit tests will catch 60% or so of the bugs as they are written, where the code is fresh in the mind of the developer.

One the philosphies of Agile Development is to embrace change. With a good set of unit tests, you can make changes to our code with confidence that the bugs will not cause problems in other parts of the system. These regression bugs, (As as I like to call them "Whack-a-Mole" bugs) can cause series headaches for developers and result in many lost hours. By investing in a set of unit tests, these bugs are caught sooner rather than later saving time throught development.

The last main advantage of the test-first technique is it helps to drive better requirements. By asking the question: "How can we test this?" it forces the analysts and customers to think about the feature at a deeper level. By going through the excercise, additional questions are usually raised that may not have been raised until much later in the process.

The next step is to set up a continuous integration server. There are several products available that accomplish this, including Cruise Control, which I am most familiar with. The idea behind continuous integration is that there is always a working copy of the build running at all times. Whenever a change is made to source control, a new build is made, unit tests are run, and it is deployed to the test area. If any of the steps fail, the entire team is notified through a tooltray icon, and the team works together to get the build up and running again.

The effect of continous integration is that integration related bugs are caught as soon as they are checked into source control. On many projects without Continuous Integration, there's a mad rush on build day to get changes into the build, and then a frantic effort to get the changes working together. By continously building the software, these frantic efforts are reduces and deployments can go smoothly.

The last thing I would recommend on your first Agile project is find a way to engage your customer. Getting regular face to face meetings is extremely important along with regular phone conferences, if necessary. Getting the face to face contact improves the communication, and just as important, it builds trust. Building trust allows the users to prioritize their requirements. All too often when a user hears "We'll but it on the next iteration" they interpret that to mean "You will never get that". After demonstrating in a couple iterations they will eventually see their lower priority items, they are much more willing to place items up the future task list and tackle them when they come up.

Welcome

I've dedicated this small corner of the web to spew forth my random thoughts on my chosen profession: Software Development.

First a little background:
I own a small consulting company, Agillence Software Inc, that specializes in helping companies utilize Agile Software development techniques and philosophies. I started in the field in this field right out of college with a degree in Computer Science. My studies were very much bits and bytes, which laid a nice foundation for a career in software development. Some years out of college I decided to round out my education and got an MBA. I will talk more about each of these experiences some other day, but for now I will stick with the introduction.

My consulting focuses on helping organizations embrace Agile Software development. Its an uphill battle, since companies seem to be wired to work against any progress. From budget battles, to timeliness, to corporate momentum, there are many factors that make this extremely difficult.

OK, now that you know a little reason why I'm spending time doing this, I'll answer a question some of you may have: What is agile software development? Well, to start go here http://agilemanifesto.org/

It outlines what the general philosophy of agile is, but I like think of it on even more abstract terms:

In its barest sense, Agile Development manages uncertainty. Software development projects are ripe with uncertainty. It is not like building a bridge. The laws of physics won't change at the end of quarter 2 where an alternate universe just released a new set of physical laws that give them a huge competitive advantage.

The change and uncertainty can be managed in a variety of ways, the simplest being to pretend the uncertainties do not exist. You can spend a lot of time planning the work, making very precise (but wildly inaccurate) estimates as to the level of effort needed. Once the plan is agreed to, make every effort to prevent change to the plan. Unfortunately, the uncertainty really isn't managed, it is ignored.

Agile development focuses efforts on the most certain component at any given time. Let the customers drive what they feel is the most valuable component of the project. Put off decisions that do not have to be made now, and make them when we have more information about the problem. When the decision is made with better information, it increases the value of the decision. That does not mean, in retrospect, that every decision will be the best, but taken as a whole, the system will run better.

"Managing uncertainty" That doesn't sound like the typical explanation of Agile Software Development. That doesn't sound like Software at all. In sounds like something more suited to the insurance or financial industry. Well, truth be told, when I went back to school for the MBA, I studied finance. But, like a moth to a light, I was drawn back to Software Development. Despite being drawn back into Software, Finance and economics are very interesting subjects that apply to a variety of fields because under all the math and theory, they are social sciences.

Which leads me to my first 'law' of software development:
Gillespie Fundamental Law #1 - "Its not the technology, silly."
Under all the bits and bytes, software development is a social activity. Ironically, an industry with the reputation of employing some of the most socially limited individuals (present company included), the primary success factor of any project is the ability to work together. Nearly all major problems I've seen an any given project can be traced back to people. Sure, a server may meltdown, or perfomance may be poor, but most of those problems are relatively easy to fix. The harder problems to fix are when are company cannot even get to the point of installing working software on a server. In the current business environment, if a company cannot utilize a software infrastructure, it will fail.
Corollary 1a - People are not interchangable parts. The output of a developer can vary my more than 10x between people of similar experience. If you get quality people on a project, the chances of success go up exponentially.
Corollary 1b - You cannot use process as a substitute for talent.

Gillespie Fundamental Law #2 - "There is always uncertainty."
You cannot plan away uncertainty. You can hedge against most of the types of uncertainty on software projects. The best you can do is accept it and manage your project accordingly.

Gillespie Fundamental Law #3 - "Everything comes at a cost."
Every activity comes at the cost of doing something else.
Corollary 3a - The value of every activity must be justifiable. Every action must generate value for the company. Creating volumes of documentation no one will ever read creates no value.

That's enough for now. There are more laws, but I will let you chew on those for a while.

-Mike