“Doing something that has never been done.”

Last week, there was a Quora post about why software estimates always seem to be wrong.  There was some decent discussion on the topic, but one quote in particular bugged me.  The quote in question was:

Developers are also the only group where they are asked to do something which has never been done before, and tell someone else how long it will take before they even know what actually needs to be done

Why did this bug me?  I think because at best it’s misguided, and at worst it’s flat out wrong. And in both cases, it’s unprofessional.

An Example from Engineering

As with many items, I go back to engineering.  I know a lot of people disagree with calling development software engineering, but I think there are a lot of parallels between them. An electrical engineer is asked to design a new board that will know it’s GPS location and send an SMS text message to a phone number to tell it where it’s at. For example, there is a model you can purchase for roughly $400.  Let’s look at what the engineer of the first one of these devices would have needed to do.  

• He’d either need to buy or program a chipset (and board) for the GPS functionality.  
• He’d need to either buy or program a chipset to send an SMS.  
• He’d need some form of input for the user to enter the SMS # the message should go to
• He’d need to design a circuit to power the entire device
• He’d need to make sure it fit within a specific form factor.

Have these things been done before?  Absolutely.  Even if the details (e.g. GPS) hadn’t been, it was still a basic pattern, a recipe so to speak. He probably pulled out his Motorola books that he had (or more likely went to the website) to find the chips he needed.  Looked at the schematics and added the necessary resistors.  I’m guessing he could design a power circuit in his sleep.  They’re all components, even if he’s never used them before.

At this point, he could factor in some time for integration, getting the GPS and SMS chips to play with each other, trimming of the final few milimeters to get it in the right packing etc.  But he should have a somewhat reliable estimate.  

How it Applies to Software

How is this fundamentally different than software development?  In all the software that I’ve written, there have been very few things that “have never been done before.”  I have a single patent, and that has more to do with my help designing the system and less to do with the code I actually wrote.  

In just about every single application I’ve written, here’s a summary of what I’ve had to do:

• Allow users to enter data
• Store data in some sort of persistant manner
• Retrieve data at a later date
• Display data to the user

In my more technical projects I’ve also had to communicate with another piece of hardware, whether that was an engine controller, or a little motor or what have you.

As I look at the list of the engineer and my list, I see some commonality.  I know just about every project I’m on right now will need some sort of persistance.  Regardless of if that is SQL Server, Mongo, XML, text files etc, it needs to be persisted.  That means I instantly know a few things:

• I need to connect to something external to my application
• I need to handle the case where the connection is lost
• I need to handle the case where the data doesn’t exist for the query I’m looking for
• I need to handle the possibility of data collision (e.g. inserting the same user twice in the database)

There’s nothing new there.  Sure, maybe on this project I’ll be using MySql instead of SQL Server.  Or maybe I’m using a new ORM to persist my data, but it’s not fundamentally new.

Application Specifics

While it’s true that we can rarely say the technology has been done, what about the business rules, the flow? 

Maybe it’s just me, but I’ve worked on very few ideas that were completely original.  At one job, my main duty was to re-write an application, updating several features a long the way — far from “original.” 

A lot of software today is the same drivel rewritten based on someone’s specific desires.  There are a ton of CRM software packages out there. Wikipedia lists over a dozen, and that’s only a small subset.  Countless companies have written their own internal CRM because those others were missing some function.

So while your particular requirements of how it flows from screen to screen, or what fields need to be checked, it’s still just a CRM.  There’s information out there, people that you can talk to, that will help you estimate.

Unprofessional?

Is it really unprofessional to make the statement that software developers are “doing something that has never been done”?  I believe it is.  Consider having a house built.  Most homes today seemed to be built according to this model:

• Pick a building company that you trust
• Pick one of their 4 communities in your town
• Pick a lot
• Pick one of their 5 floor plans/house styles
• Pick paint colors, carpet colors etc
• Move in

With this plan, you can have a reasonable expectation of when to move in.  But imagine you don’t want a cookie cutter house.  You sit down with an architect and work out your “dream home.”  It doesn’t look like any other house. It’s not really big, maybe 2000 square feet.  It’s not the Taj Mahal or anything like that, but it’s what you’ve always wanted.  Are you going to be ok with the general contractor if he tells you it will take 8 months to build and ends up taking 24 months?  Probably not.

After all, even though it’s not like anything else, it still is a house with walls perpendicular to the floor, a roof with a 4 in 12 pitch.  It has windows, doors, floor etc.  The configuration is different, maybe even some of the materials, but the laws of physics still apply.  So you might tolerate the timeline slipping a month, you’d be furious to let it be off by a factor of 2 or 3 (and you’d be right to be furious.)

So in part, it’s unprofessional because we are expecting people to hold software developers to a different standard than just about everyone else.  That comes across as elitest and whining, not professional.

Lack of Communication

There’s one more reason, though, why this is unprofessional.  I mentioned above that very little software is actually “new” or “original.”  Sadly, a lot of it now is rearranging pieces and components.  Maybe we arrange them in a new way, but I’m not convinced that’s fundamentally new.

With that in mind, then, there are people that have written a CRM in ColdFusion.  There are people that have written a CRM using Postgres as the persistence layer.  If you’re fortunate, there’s people that have written a CRM using CF AND Postgres.  But the problem is, almost no developers talk to each other.  Or if they do, it’s rarely about specifics, such as some gotcha the encountered when using CouchDB on their back end.

Doctors have journals where they detail some new procedure they’ve performed.  If they’re honest, they’ll outline some of the downsides, risks and caveats that must be accounted for.  This will allow another doctor to get a better understanding of how the procedure works.

Imagine if you needed surgery on your knee.  It’s not life threatening, but you’d like your knee to function after the surgery.  Would you choose a doctor who has never done the surgery, never seen the surgery, and has not even really read about the surgery before?  Probably not.  But as an industry, we have problem with developers who attempt to blaze new trails without talking to anyone about their experiences.

TL;DR

As software developers, we’re not as unique as we’d like to think.  The projects we work on aren’t as earth shattering as we hope.  As a result, we should do a better job with our estimates, considering that a large part of what we’re doing has already been done, possibly even by us.