Kris Kemper

I recently read an blog post[1] describing a cognitive justification for test driven development. I think the topic is interesting. In doing TDD, there are a variety of reasons for why we write tests first, and it isn’t always to ensure correctness over time.

Certainly, there are tests that are meant for ensure that changes don’t break earlier functionality. For instance, If I’m writing a sorting algorithm, unit tests will prevent it from not sorting at some point. If I use unit tests to assert the existence of explicit business rules against domain objects, I am trying to prevent those rules from being broken in the future.

However, there is a behavioral point to TDD where a developer breaks down a larger goal into quickly attainable and testable tasks, and then completes each task in order to complete the larger goal. The tests provide a clear stopping point for the given task. Usually, writing automated tests allows us to recheck our previous tasks to ensure that we haven’t undone them. The automation of tests leverages the speed of computers to make rechecking those assertions faster and more reliably correct than a human could do it.

There is an implied assumption here. The general assumption is that automation of those test will save us time. It is assumed that it will save us time either in rechecking our assertions, or save us time that would be lost if a defect occurs.

I have found that there are times where coding an automated test is too costly. It’s partially a personal judgment, but it is also influenced by other factors such as the culture of the team or the company that I am working with. I try to do a quick cost/benefit decision for testing to determine if the time spent on the test will really pay off. As much as I value automated tests, if the cost seems to significantly outweigh the value, I don’t automate.

So, I believe that the cognitive and behavioral benefits of testing first, and the automation of the tests to be very different considerations. Even when I don’t write automated tests, I still decompose my larger goal into achievable tasks, and test assertion manually. At certain points along the way, I will manually retest all or many of the previous assertions.

I find that this tends to happen with user interfaces. The user interface component might have behavior that is difficult to automatically test in a timely manner. For instance, a “drag and drop” feature where the user can drag items from one box to another. It’s such a user centric feature, that you really have to setup the UI widget, play with it, and verify that it’s behavior is acceptable.

Even in development environments where there aren’t any automated tests, you will still often see developers using printouts or some other immediate feedback mechanism to take “baby steps” through their work. I think that even developers that haven’t realized the benefit of automating tests are still essentially doing a rudimentary form of TDD, something more like “Confirmation Driven Development” perhaps.

Many developers do not develop with some kind of incremental confirmation that they are building the software correctly; They will spend days working on a feature but not test it at all. After a few days, they experiment with the application, see what breaks, and then will chase down the various defects. I would say that’s a mistake of a junior developer, but I’ve seen some very experienced people develop like this as well. Often, these people have a mistaken belief that they are optimizing their time. In some situations, this behavior comes from the lack of continuous integration, but I’ve also seen the behavior happen when feedback cycles are incredibly long (say, it takes a minute to get the application into a state that allows the developer to test the feature).

Beginning your work with an automated test is the best initial approach. However, if you find that you’re in a situation not conducive to automation, don’t give up on taking small steps and asserting that each step worked, somehow.

[1] http://www.m3p.co.uk/blog/2008/06/15/test-driven-development-a-cognitive-justification

I often think about the qualities that separate the “good” developers from the “bad”, the skilled from the unskilled. I’ve met a wide range of developers over the years, from skilled and experienced developers to developers that can barely write code. There are a lot of people somewhere in the middle. Whenever I pair-program with someone for any length of time, it’s a window into how they work, and I can see firsthand the strengths and weaknesses of their programming style and work ethic. I can judge people on very specific merits from that kind of close interaction.

Depending on the project (or even the team) there may not be a lot of pair programming going on, and I can only judge people through a higher level observation. It’s not a great way to judge a developer for the most part - it’s hard to tell what they do on a day to day basis that leads to them producing their portion of the software. At this higher level of observation, you can generally tell how fast someone delivers features, and how many bugs people generate. Even then, it takes a lot of time and reasonably repetitive or comparable tasks before such measurement can be even remotely reliable.

With such limited information, I tend to make a quick judgment initially about people based on a “gut feeling” that I get from talking to the person about work. I don’t base many decisions on that judgment, but I tend to start people off at a certain level in my mind and I wait until they prove or disprove my initial reaction.

It reminds me of when I was in college and the semester was in the first week. There were some project/group based programming classes where we could organize into our own teams. Over the first week or so, we would have to form teams that we would have for potentially the rest of the semester. Without much to go on, I would talk to people and try to get a feel for their ability and work ethic. If I had a good feeling about them, I would ask if they had formed a group yet. If they hadn’t, one of the next questions I would ask was what grade they wanted to get in the class. It’s a very direct question and it’s usually easy to determine if someone truthfully wants to make an “A” in the class. The next question was usually to ask them what grades they made in previous classes.

One of my earlier professors recommended this as a way to align goals when building a team. If your goal was to make an “A” in a class, then build a team with that as it’s stated goal and get them to commit to it. I wanted to make an “A” in every class, and I was willing to work for it. I was hoping to find other “A” grade oriented people like me. If that couldn’t work out, I would hope that I could get “B” grade people and occasionally “C” grade people to produce “A” quality work.

I labeled people with a grade based on the grades that they made in previous computer science classes, as well as my own observations. They got either an A, B, C, D, or F. I found that people could often move about one grade higher under the right conditions, but would never more than that. So, a “C” developer would not produce “A” quality work. This isn’t that surprising, getting an “A” in college is as much about learning how to get an “A” as it is about personal ability and work ethic. Learning how to get an “A” takes practice.

All characteristics aside, there was one quality that I found was the differentiating factor between an “A” grade developer and the other grades: Flawless delivery. In the classes that I took, getting an “A” in a class meant keeping a grade above a 90 for the entire semester (with a 100 being the max). In order to reliably accomplish this goal, you had to aim for a 100, and use the other 10 points as a buffer against minor mistakes.

There are 3 main factors to achieving flawless delivery of those projects: knowing what is expected from the graders (essentially, having clear acceptance criteria), having the ability to accomplish it, and having the motivation/desire to accomplish it. When your goal is flawless software, you get good at it after a while. When I was in school, I even found a deep sense of satisfaction when I submitted the code for my project knowing that I nailed it - that I was almost certainly going to get an “A,” and probably even a 100 point grade. I take it for granted, but one important activity that is worth mentioning is that that in order to accomplish flawless software, you have to personally and thoroughly test what you build to ensure that it works.

I kept that behavior after I got out of college. When I complete a feature, or a set of features, I like it to be as close to working perfectly as possible. I don’t want to see any bugs come out of it, even in the first round of QA. In fact, if I do get bugs in the first round, I make a mental note as to why It happened and consider how to prevent that in the future.

I generally don’t differentiate between missed requirements and technical bugs - as a developer, I play an important role in making sure that it is understood and agreed upon as to what I will be making. I’m not advocating that details specifications - I may build the feature and demo it to a product owner. Regardless, when I show that demo, I do my best to make sure that demo works flawlessly.

As a final note, striving to produce flawless software becomes significantly more important when a software team frequently delivers to production. As a developer, you have a great influence in whether a bug makes it to production. In some cases, you will know areas that are risk for breaking due to a change that non-developers (or even other developers) wouldn’t guess. It’s as important to ensure that you don’t break other parts of the system as it is make sure your code works; Often, it’s far more important.