Communication, Communication, Communication

In every team, communication is key. As a kid, I had this hammered into my skull time and time again. Playing ice hockey, you had a position. The position was well defined. If you were out of position, the coach was not impressed. I found the same thing in every team sport I played.

Sometimes, there was a play that was between positions, like a fly ball between the short stop and left field. In that situation, it was very important to call the ball, yelling off your teammates. This told your teammates that you intended to play the ball, keeping them from running into you. Especially important when you are running towards each other looking up. :)

I am surprised to find that even with all of this training as children, we fail to take this knowledge into the workforce. I see teams failing to "call the ball", stepping on each other's toes while working on the same task. I also see teams that forgetting tasks, assuming that someone else is taking care of it.

What can we do about this!? First, give people a position. Make them responsible for a single area of work. That area should be well reasonably defined! Make sure that you get complete coverage of the incoming work out of all of the positions on your team. Don't worry about a little overlap, we'll get to that. :) Taking the easy way out and making the entire team responsible for all of the work isn't going to work either. If you do this, you end up looking like a group of five year olds playing any sport (hockey, rugby, soccer)... Everyone ends up clustered around the most important piece of work.

Let's look at how this type of system would work.

When a piece of work arrives that is covered by a person's position, they cover it, no problem.

When something overlaps, people have to call ownership. They have to assign the work to themselves, and if they assign the work to themselves, they are accepting responsibility for that work - all the way to completion or hand-off to someone else. The converse is also true. They shouldn't work on something without calling it unless it has already been "called". Otherwise, they run the risk of confusing everyone. If you don't play "call ball", and start helping out, who is leading the work? No one knows, especially not a third party coming along later.

In order for any of this to work, everyone needs to know not only their position, but everyone else's. This is where communication comes in. Without it, people won't know when their positions overlap. The team needs to communicate not only who is responsible for what, but who is working on what.

If you start playing "call ball", you will have a lot fewer customers saying, "Hello? Is anyone working on this?", as well as a focus on the important work, instead of the most current.

As I read this post, it all seems too obvious. Of course I say, that's exactly how it should work! Yet, it seems to be a very rare occurrence in the marketplace.

Make it easy for me to READ IT!

If you want a response make it easy to read. Don't make me jump through hoops just to open your document. I make every effort to read them, so I am always surprised when people find new and interesting ways to prevent others from reading the documents they send out.

I was given a document today to review. First off, it was compressed in a rar. A RAR! Other than bittorrent movie downloaders, who uses RAR? Why would you use RAR? It's a tool that people have to search to find, and then is "Trial Only". Can you say, "Corporate License Hell"? I knew you could.

Next, it's a Microsoft Word document. I don't have Microsoft Word, but I do have Open Office. Too bad for me you used some fancy formatting, because Open Office crashes trying to import it.

So, not only do I have to find someone else to open the document, but you killed my word processor, losing some of my work.

Yep, that's the way to make me really want to review your work.

This happens to me all the time, with status reports that we are supposed to review, procedures that we are supposed to adhere to, everything. I just shake my head and go, "Oh well, I guess that's something else I don't need to do".

Hmmm maybe that was the purpose? Maybe you don't want me to review it?

The non-obvious benefits of outsourcing

All names changed to confuse the hell out of readers.

Outsourcing, the bugbear of modern IT. Companies all over the world are outsourcing software development jobs to cheaper locales, resulting in redundancies. This makes it a very, very emotional topic for some people.

I'm working for a company that is currently outsourcing it's support infrastructure. They aren't doing it for cost - the outsourcing company isn't any cheaper than their current cost structure (within 10%). They are doing it because the outsourcer is able to hire substantially more software developers than are available in the markets that my employer operates in.

I think that my employer will see substantially more benefits than just in scalability. They are actually starting to see some of them already.

Since they're outsourcing support and maintenance, the majority of work done by the outsourcee is self-inflicted: fixing bugs. All of a sudden, there's a new push to add quality to the software. Bugs have a definite bottom line effect. More customer discovered faults result in more charges from the outsourcing company.

This also shows the company what happens when they let projects slide. If a project delivers quickly, but with poor quality, it's visible. If they fail to meet customer requirements, it's visible. If they develop a system that can't be maintained, yep, it's visible. All these things were visible before, but now they're visible at the executive and board levels. It will be interesting to see if they start to track this.

It forces the company to also firm up their processes, procedures and communications. Since the outsourcing company is trying to do the least amount of work for the most money, if there is any reason to avoid doing work, they probably will. So, if they don't have ready access to an answer, or run into even the smallest problem, they will stop and ask a question. Since each question taking 24 hours to answer because of timezones, there is a lot of wasted time. The outsourcing company isn't a partner. They aren't going to say if the processes are bad or wrong, they are just going to charge you more.

It isn't yet apparent if the project will succeed. Depending on political power levels at the time, people could try and blame the outsourcing company for the increased costs, calling the outsourcing project a failure. This is, in fact, already starting to happen. We've even got people subtly changing development procedures to make it harder for the outsourcing company to do their work.

However, with a little effort, the project should help force the original company to grow and change. It's hard to ignore substantial amounts of money going out the door. Hopefully it's easier to find the root cause than blaming the person charging the money.

Time will tell.

Experiences with Pair Programming

One of the central tenets of Extreme Programming is that developers should never write code on their own, preferring to work in pairs. This is called Pair Programming. The basic concept is that the pair is better able to spot each others mistakes, and help find the simplest solution.

Pair programming has its sceptics. It really does depend on having an active conversation between the two developers. What it can't be is one person typing and the other watching. When I first started pair programming, that's what it was like. It was extremely boring. Boring to the point that I actually fell asleep watching this other developer type code. There's nothing more embarrassing than drooling on another person's desk.

But I kept trying. As we learned how to work together, we found several benefits. We did spot each other's faults. Coding/design oversights, off-by-one errors, etc. We caught a lot of them. However, the most telling point was when another developer and I were fixing a bug. He looked at me and asked "Should we write a unit test for this?" We both quickly came to a very important realisation. If it had been either of us on our own, we wouldn't have written the unit test, we would have tried to fix the bug without the test, and moved on. However, since we were sitting beside someone else, we couldn't talk ourselves out of it. We wrote the test, and solved the problem quicker for it.

That single incident convinced me that pair programming is a great way to enforce a "team value set". Peer pressure works really, really well in small groups.

The other thing that I noticed was that when it came time for me to leave employment with that team, I nothing to hand over. This didn't mean that I didn't achieve anything. It just meant that there was at least one other person out there who knew what I knew.

I'm now back contracting with another organisation and I see a lot of problems that might be made better by a little peer pressure. The build system is slow, agonisingly slow, and it's gotten slower over the past year. The systems aren't maintained in any semblance of order. If you use a machine, you can't expect that the system will be runnable, or even that there will be a set of development tools waiting for you. What does the team do? They ignore these "broken windows" moving on to other systems - to the point of avoiding faults reported by specific customers!

How would the peer pressure offered by Pair Programming help? Well, I would expect that it would become a lot harder for a developer to put something in the "too hard to do drawer" when there's someone sitting beside them. As each team left the systems better than they found them, the problems would gradually sort themselves out. This fits in with another mantra that I've learned recently, "No broken windows"[5].

I was very interested in understanding why pair programming helps peer pressure work. I did a quick google search and turned up some papers. It seems that peer pressure is an area of current research in economics. I didn't really try to understand the math. In [1], they say that the cost of punishment must be low enough for peer pressure to work. If it costs too much to sanction a person for slacking off, they will be left alone. The paper then goes on to demonstrate this using Nash equilibria. This fits in with my experience. Pair programming forces interaction, making it very obvious when an individual is not working to the standard set by the team, allowing immediate sanction by the other team members.

In [3], we have an actual experiment. In that paper, they compared the productivity of envelope stuffers alone, alone with an external indicator of performance, and in pairs. They found that the pairs were the most productive, primarily by improving the performance of less productive subjects.

So, it seems that pair programming works because it helps lower the cost of sanction, resulting in more efficient peer pressure. This may explain why some organisations are reporting that they see more productivity out of a pair than they do out of two individuals. Pair programming isn't a silver bullet. It isn't going to magically solve all of our development process issues. It just might help your developers take responsability for the systems around them. Something to think about.

References:

1. "Peer Pressure in Work Teams: The effects of Inequity Aversion", David Masclet, December 2002. Discusses how the cost of punishment must be sufficiently low enough to allow punishment. Pair programming lowers this drastically. Uses Nash equilibriums. Lowering the cost of punishment increases the frequency and potency of peer pressure.
2. "Social norms and optimal incentives in firms", Steffen Huck, Dorothea Kubler and Jorgen Weibull, 8 October 2001
3. "Clean Evidence on Peer Pressure", Armin Falk, Andrea Ichino, January 28, 2003.
4. "Team Size and Effort in Start-Up-Teams ­ another Consequence of Free-riding and Peer Pressure in Partnerships", Uschi Backes-Gellner, Alwine Mohnen, Arndt Werner, January 2004
5. "The Pragmatic Programmer: From Journeyman to Master", Andrew Hunt, David Thomas, 1999

What good software does for it's users

or Choice Sucks

I was reading Paul Graham's latest weblog [1] on how to make a lot of money writing software. Like any greedy person, I'm trying to come up with some software that's going to make me rich. Like Paul, I had been slowly coming to the realisation that developers didn't like purchasing software. So, I decided to take Paul's advice and try to think of what non-developers would want out of their computers.

This is harder than it looks. I don't really know many non-developers. All of my friends are software people. They tend to be married to other people in the IT industry. It is very hard to figure out what people want when you don't really have any experience. :) As I sat in bed mulling over the problem, my brain started to make associations. I realised that there was a connection between useful software, "The Long Tail" [2], and psychology.

I remember reading that people consider choice to be hard and stressful [3]. Everyone is very afraid of making the wrong choice. This is why we spend so long trying to figure out what breakfast cereal we want to buy. Check it out - next time you are in the grocery store, see how long you spend deciding. Now, think of how long we spend making more important decisions. How stressed we get worrying if we made the correct one or not.

The Long Tail [2] says that web businesses like Amazon and NetFlix do really well from being able to stock more titles than their bricks and mortar competitors. That they receive a substantial number of sales from the titles that they can stock but their non-on-line competitors cannot.

My epiphany was this. That the stress we feel over making a decision leads to the popularity of the long tail. Follow me in this. In order to better drive customers to other items, Amazon implements a review system. They then start to track purchases and make recommendations to their customers. This pushes people down into the tail. Why though? It works because people are afraid of doing the search themselves and making the wrong choice. They only really know that they want to purchase a book. They don't really know what they are looking for. So, Amazon's recommendations are as good a place to start as any, better than browsing the store themselves.

So, people willingly let the computer do the filtering for them. Does it always work? Nope. Is it a less stressful purchase? Definitely. The customer knows that someone out there, not completely unlike themselves, has also enjoyed this book. This means that it is probably not that bad.

This led me to my key realisation. Good software removes choice from the user. That doesn't mean that it doesn't make it possible for the user to see all the nitty gritty detail, just that it hides it as much as possible. Apple realised this, look at the interface on the iPod Shuffle. Mac software is all about hiding complexity.

So, truly useful software should help the user make decisions. It should remove grunt work from the user. For example, in the UK there is a web page that allows people to compare electricity providers. You enter your monthly usage and address, and out pops the cheapest provider for you. That helps the user make a decision. It means that the user no longer has to stress about which supplier to use, they just ask the system.

Good software should make (if possible) as many decisions for the user as possible. If the user ever answers a question, the software should never ask it again, and use it to answer other questions for itself. Software should be more about the assumptions it makes for the user. In most cases, the user won't care about the assumptions that are included, they will be the correct ones. Amazon's one click shopping is a good example of this. The user only has to setup the system once, after that all that they have to do is indicate that they are buying that item there.

This is a fine line. I'm not talking about autonomous agents that go off and do it all for you. Helpful software like Clippy is just plain annoying. Predictive text messaging isn't. It may sound like I'm saying that people are sheep. I'm not! I'm saying that the goal of a computer system should be to remove complexity from the problem. We shouldn't be adding decisions to a problem, we should be removing them.

References:

1. The Long Tail
http://www.wired.com/wired/archive/12.10/tail.html (original article) http://longtail.typepad.com/the_long_tail/ (web log)
2. The Paradox Of Choice: Why More Is Less
http://www.amazon.com/exec/obidos/tg/detail/-/0060005696/qid=1111411407/sr=8-1/ref=pd_csp_1/002-2296419-5280060?v=glance&s=books&n=507846 http://www.newyorker.com/critics/books/?040301crbo_books http://science.slashdot.org/article.pl?sid=04/03/29/1214225&tid=99&tid=14 http://www.healthpromoting.com/Articles/articles/major.htm http://www.lowellgeneral.org/features/health/foryourhealth/AIHspring99/AIHstress.htm
3. Paul Graham essay on startups
http://www.paulgraham.com/start.html

The Importance of Dates

Or: A project without an end date never finishes

The project I am working on is moving into the grey area where it will never finish. There are new releases every week, and while each release is better than the last, it isn't perfect.

The problems are caused by two things. The first is the lack of a project plan. Without a project plan, no one knows when the project is late. This was done intentionally by the customer's project management team to avoid internal political problems with not meeting a date.

However, this has a couple of flow on effects. Without a schedule, the development team (who is 12 timezones away) doesn't feel any pressure. They don't know if the software is good enough for delivery or not. They don't know if they should be pushing back on additional features in order to bring the project to completion.

Without a hard, or semi hard, end date, there is no common focus. There's no way to cause the customer's team to start making hard choices. They don't seem to be thinking "If I get one more feature, I'm delaying the project by 3 weeks". Without a date, they have nothing to fix their sight to, and there is no reason for them to not ask for more features. There's no weighing of the value of a feature, only a push to get everything included.

Finally, without a date, customers become frustrated. The parts of the customer's organisation not asking for new features begins to wonder why the project isn't finished. They look at the software and see it just as buggy between releases because the focus is still on additional, customer requested features. The people who are then waiting become frustrated, and express that frustration up their management chain. This then results in a disconnect between the customers view of the software and the development teams view. The developers look at the project and think they're doing well, they're responding to customer requests quickly. The customer looks at the software and sees something that isn't finished and they don't know why.

I believe that while there may be political reasons to not have a fixed, communicated end date, the various parts of the team do need to have one. Without that, there is no way to tell if things are achievable or not.

This could also be a failure of communication between the customer and the developer, or even internally at the customer. A harder line on change control would help with the creep, but it tends to result in frustration for the customer because they want their important changes implemented quickly. The trick is to get the customer to do the filtering, and accept the impacts of their decisions.

One final thing. Always make sure that your external milestones are not the same as your internal dates. Otherwise, if there's any slippage in your internal date, it becomes immediately visible externally. It is always best to have a little bit of room between those two dates to buffer the customer from the slips and recoveries in the dates.