Home > Epiphany > Measure always, optimize sparingly

Measure always, optimize sparingly


At work today, I found myself falling into a classic coder mental trap, one that’s worth sharing.

We have an automated test suite – we call them our “sanity tests” – that we’re expected to run prior to checking code into the repository. It’s a quick validation run that helps keep the code moving in a positive direction – we can know almost instantly if we’ve made changes that break the system, and back out bad code before anyone else is affected by it. It’s really the only sane way to code, I honestly don’t know how people and teams stay sane without… sanity tests.

Between each test run, we try to initialize the system back to a “known good” state, so failures from a previous test don’t result in false positives in later tests. The system “reset” routine destroys some intermediary data, and depending on the amount of data generated, it can take a while.

Or so I thought.

But I digress.

The total test time is very small, a few seconds on a nice Linux box and maybe half a minute on a slower Windows system. But for whatever reason, the time it takes to reset the system has been bugging me. I keep thinking to myself, if only I could shave the 8.37 seconds down to 4.83 seconds… that would be totally awesome.

So finally, I broke down and gave in to my obsessive compulsive nature and mentally geared up for a good half hour of debug/hack/burn to optimize the reset.

More than half an hour later, I put the finishing touches on the routine that destroys the intermediary data – because that was obviously the slowest, most annoying part.

Right?

Uh… no.

I literally smacked my own head in disgust.

I had fallen for the classic engineer “infinite optimization loop.” The special OCD place we all go where everything is open for tweaking, improving, and optimizing to death. The place where ROI is always infinite and there are no time constraints.

I went back and measured the reset routine, and the piece I had optimized accounted for only 1/3 of the reset time. My optimizations were about a 50% speedup, or 1/6 of the reset time, and only .3 seconds shaved off the best total system time.

Holy cow, I’m a flippin’ rockstar… move over Elvis, here comes dork boy.

When I was in the middle of this grand epiphany, I remembered the sage advice of my uncle – a carpenter of 30+ years: “measure twice, cut once.”

In software engineering, we’re lucky because many physical constraints just don’t apply. There is no board, no saw. So there’s no reason we can’t cut a hundred times, measure, and go back and cut some more.

But the problem is, sometimes we get so far removed from physical constraints, we can far too easily believe there are no constraints. But time, money, and schedules are at odds with the Utopian, ideal engineer’s optimization loop.

So I’d like to propose a motto for coders of all races, creeds, and compilers:

Measure always, optimize sparingly.

Atto

Advertisements
Categories: Epiphany Tags: , ,
  1. No comments yet.
  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: