Measurement and Precision

About 15 years ago, I worked for a large engineering software company. While I was attending one of their corporate conferences, I met John, who was instrumental in the development of one of their products. He told me a story about the beginning phases of their new product when they were making architectural decisions. Back then, memory wasn’t the commodity it is now. The product ran on UNIX workstations and you had to be far more conscious of memory consumption and data structure sizes than most applications do today (although we might all be better off if people gave it a bit more consideration). The product worked with Finite Element Analysis (FEA) for analyzing mechanical and structural designs. As you might imagine, this kind of mathematically intensive work could use a lot of numbers and the precision of the numbers would affect both speed and accuracy of the analysis. John was adamant that double precision was required for accuracy, but was overruled by a manager, who was not technically oriented. Months later, when the analysis results were proving to be inaccurate, someone asked who made the precision decision. Someone blamed John, who was understandably quite livid about being blamed for the decision he had fought against.

A few years later, I was working on a solid modeling product for architects. I was mentoring a colleague, Sharon, in the ways of solid modeling and general application development. In order to do some of our calculations, we were using a vector class created by our graphics guy. As such, the vector math was being done in single precision as that’s what was common practice for graphics. As you might guess, when some of our modeling operations came out poorly, I asked Sharon to convert our code to use a double precision vector class. She was sure this was not the problem, but as I was the project leader and her manager, she did it anyway. Lo and behold the problems disappeared and she learned a valuable lesson.

Perhaps you, too, will have to make some decisions like this in your software development. Before you jump to conclusions, especially when it comes to something that’s fundamental to your product, do some experiments on size, speed, and other measurable performance. Err on the side of flexibility, if all other parameters are equal. If your language supports creating your own data type (such as a C++ typedef), you can delay the decision and change it later. If you believe that someone in your group is making the wrong decision, say something. Be sure to say it nicely, but say it and record it for future reference. When the fecal material hits the rotating air circulation device later, and someone picks you as the scapegoat, you’ll be able to say you did what you could. Better still, however, is to prevent the mistake by proving that the decision is incorrect by backing it up with hard data and keeping it objective and not personal.