Test before or test after?

What’s the point of programming? It’s to solve a problem, right? We work in a field that strives to attack real-world problems. Providing solutions that help make life easier for users. Whether that’s simplifying how you do taxes (TurboTax) or helping people stay in touch regardless of geography (Facebook.)  The main focus should be to make sure that we are delivering products that really solve the problem we’re trying to address.

One advantage that we have as developers today over developers from 20 years ago (or really, developers 10 years ago) is that we have a lot of really good tools that we can use. Our computers are fast enough and cheap enough that compilation time is essentially nothing. This means that we don’t have to spend a lot of time writing out flow charts, UML, pseudo-code and other preparatory material in order to write our program. We are able to be flexible and adjust as time goes on.

One of the best tools we have at our disposal in most modern languages, is the ability to write tests in code. This means we can build up an entire suite of tests that we know will run the same way every time. This means we can be certain that a block of code under test will reliably perform exactly the same based on the same inputs.

Not everyone agrees how to bridle this awesome power. There are two main ways of writing tests. The first is to write a test before you write the code that is to be tested. This is called Test Driven Development (TDD.)  The other end of the spectrum is to write the code, that is, the actual product, and then write the tests. In the end, both groups are trying to accomplish the same thing: automatically test their code and ensure that the code under test functions correctly.

So since both approaches have the same goal, and both approaches are writing tests that automatically execute code, are the same two approaches equal? No

How can these two approaches not be equal? In both, the user is using code to test code. What makes them different? There are a few differences between the two approaches.

Drawing the Blueprints After

You get the call from your general contractor, he’s ready to turn the keys over to your brand new house. The only thing left to do is walk through the house. You arrange to meet him the next morning at 8:00am at the new house.  As you pull up, everything on the outside looks great. The windows look perfect, the landscaping is nice and trim. The colors are exactly what you’d want.

You can’t wait to get in the house and look around. But that’s when the problems start. You walk in and look at the stairs and you notice they’re not all the same size. Some planks are so shallow you can’t get your whole foot on it, others are so deep you could take two steps on one plank. The heights aren’t equal either. Sometimes you can bound 3 steps at once, others make you wish you had some climbing rope to get to the next flat.

As you make it up to the top and start looking down the hall at all the bedrooms, you realize that the stairs aren’t your only problem. Each bedroom door is a different size. Not incredibly different, but it’s obvious that on some of the doors they had to cut large portions off to get them to fit in the frame.

By the time you walk all the way through the house you are fuming. You’re about to pay a large amount of money for this building and it looks horrible. In fact, it looks so bad you have concerns about the structure and if this house will be able to stand for the next 5 years.  As you approach the contractor you notice that he’s writing something down.  As you get closer you realize what he’s doing is drawing the blueprints to your house.  It’s no surprise that the house matches exactly the blueprints he hands you with your keys. After all, he didn’t draw the blueprints until the house was done.

That’s what writing your tests after is like. If you’re given a task to calculate a bill of materials (BOM) for an entire system. There are a lot of parts to this application. You have to calculate the length of the system, account for things like heat expansion, determine the exact number of brackets and fasteners. The way most of us will work is to write a method  (or a couple methods) that will calculate each item and add it to a list. In the end, it could be as small as 50 lines of code, or as many as 500 lines. After we sit down and write our BillOfMaterials method, we start to write our tests. What we know at this point is what the method does. We’ll end up writing tests for what we think the code does. If we’re lucky, that will also be what the code actually does. If we’re even more lucky, it will be what the code is supposed to do.

In contrast, when you write your tests first, you are writing code to make the test pass. You are drawing your blueprints first (the way real architects do it) and then building to the specification. So too with tests. The test is your blueprint, and you are building your code to make it meet the specification of your test.  When you get done, if there is problem with your code, it is highly likely that it’s because your specifications are wrong or lacking. But you know that your code meets your specification, instead of your specification meeting your code.

When does it Really Get Done?

Tell me if this scenario sounds familiar. You are working on a project for a customer. There is a long list of features that needs to get done before you can go to beta. You’re working on feature A. You estimated this feature would take 2 days, it ended up taking 3.5 days. When you get done you cross the feature off the list and hurriedly moved on to the next feature, feeling the crunch of time to get the entire project back on track.  It has happened to me more than once.

What gets lost in the process is writing tests. You write the business logic with every intention of writing your tests, but by the time you’re done you’ve got to move on to the next feature, and you don’t have time to write the tests. You tell yourself that if you get the next feature done early you’ll be able to circle back and write your tests. Or, worst case, you’ll have time to write them when the product is in user acceptance testing.

However, writing the tests first guarantees that the tests are actually written. After all, you write them before you write the code. You will not be releasing any code that doesn’t have tests.

In all honesty, writing tests first helps combat human nature. There’s an old joke that asks “What day to most people start their diet?” The answer is “Tomorrow.”  As true as that is about diets, it’s also true about tests. If you don’t write them first, when will you write them? Soon. But “soon” turns into “after lunch.” After lunch turns into tomorrow. Before you realize it, you haven’t actually written any tests and you have so many untested methods that you realize you’re at the foot of the technical debt mountain.

What am I Testing?

Closely related to the technical debt mountain is the problem you’ll have once you do decide to start writing tests. You’ll have written hundreds of lines of code since you wrote the method you’re testing. By the time you get back to the code you have to spend time reading it to remind yourself what it does in the first place. You probably didn’t use a lot of methods nor make it highly readable. After all, you were “in the zone” and pounded out this whole process in an hour or two. Now here you are 2 weeks later trying to remember what each line of MakeBom() actually does.

That’s not a problem you have when you write your tests first. If you write a test, and then write the smallest amount of code to make that test pass, you know what your code is doing, it’s passing the test specified. There’s not any guess work. You can always find the tests that you wrote for that method and see what it’s supposed to do.

Tl;dr

In summary, if you write your tests first you will realize numerous benefits, only a few of which have been mentioned here.

1. You write code to match your specs instead of specs to write your code.
2. You actually write your tests, instead of being woo’d by the siren song of the next feature.
3. Your tests help explain what it is the method is doing.

Writing tests first is the best way to guard against wrong specs and human nature. Writing tests after the code, just isn’t equal to writing tests first.