qualitas (lat.): a quality, property, nature, state, condition

This article shows you how to make a quantitative statement about the quality of a software product instead of a qualitative one. Uhm, what!? – Yes, this article is about measuring quality. It is a common misconception that the quality of a product is an additional ‘soft factor’ besides meeting all its requirements. There is a simple engineering approach to gauge every possible aspect of the quality of a product.

I once had a discussion with a colleague about the quality of our software product. The schedule had been tight, many extra hours had been spent, but we somehow managed to deliver something that had all the features in it that had been negotiated. Internally, we were not happy with its quality: Some modules would have needed extensive refactoring to lower the effort spent in debugging and feature development, while for others we felt the lack of additional automated test coverage for the exponential parameter space. The customer upset: Several corner cases he used the software for would not work anymore (guess what: they were from the exponential parameter space – we just did not know, foresee, or even expect they could be important), several buttons were misaligned on smaller screens than those of our developers and testers, and a bunch of calculations were drastically slower than in the previous build in the customer’s environment (because of a bug within the XML storage engine of that very specific build number of the database software that the customer provided, as we found out later). So much effort, so much commitment, so much frustration. Well, the average situation where you start general discussions about the quality of your product. 

We were discussing the question of quality in an attempt to write a business case for an invest in the overall quality of our product. The question was how to invest. What should we do and where should we start? And of course: How much will it cost to reach – well, something? What exactly would we want to reach? – First of all, the customer must be made happy, that’s for sure! He should ‘feel’ more quality in the next release. Likewise, we wanted to take steps so we could also ‘feel’ better about our codebase and our test universe. (I mean – we were the experts, right? If we felt bad about what we produce, how could we expect someone else to belief in its quality?) But how would we be measuring these ‘feelings’? How can we influence them on our own side and on the customers side? And how much does it cost to influence feelings? – Well, we are engineers and this is not Red Bull! We wouldn’t start with manipulating feelings, we needed a less emotional approach. I am not saying that either of both, the customers’ feelings or the developers’ feelings are neglectable. Quite the contrary, they are the key parameters at the end of the day. But we need some basic stuff beforehand. One step after the other. 

The key to systematic quality management lays in the Latin root of the word quality: qualitas. If you look it up in A Latin Dictionary, it translates to a quality, property, nature, state, condition. Translated to our problem domain that means there is no single quality of our software product, there are multiple – maybe hundreds, maybe thousands. Thousands of quality attributes determine the quality our software product currently has. Every property, state, or condition I start to control determines the quality of my product in a certain aspect. If I do not control it (or if I even did not yet think about it), my software product build might or might not have this property. If a property significantly contributes to the quality impression a person has about the product (may it be a customer, tester, or developer) then not having this property will give a lower overall quality impression and having this property will raise the overall quality impression – simple as that.

Let me give you an example: Here in Germany, Spaltmaß is an extremely important property. It can be translated to constant clearance or uniform all-around gap/space. We expect the engine hood of a car to have an exact 7mm distance to the car body all-around and to have that same 7mm space between the doors’ sheet and the body as well. It does not matter if it’s 7mm, or 8.5mm, or 9mm. That’s to the manufacturer’s discretion. It only has to be very, very uniform to adds to a German’s quality impression to a car. The point is: Every car manufacturer installs the engine hood and the doors in a way that there is enough gap all-around the component part so you can open and close it. The ‘obvious requirement’ is fulfilled. And every component part will have ‘a’ clearance. – But will it be uniform? Of course, if the robot has tolerances and installs the hood sometimes a little more to the left and sometimes a little more to the right, it can accidentally happen that the clearance is uniform. But for a high and constant quality impression you would have to control that property, at least for the German market.

But how would we identify all those properties that are important? And how on earth would we know them beforehand? Our customer/Product Owner does not tell us those things upfront, you might think. Well – yes, this is what iterative process models (e.g. Scrum) are made for. Mostly you don’t know these upfront, you learn them en route. Just treat them like any other insight you gather during your project (e.g. missing or ill-designed feature): collect the insight, learn, adapt, protect your progress (e.g. reviews, automated tests), and go on.

To control the property, the only step you have to take is adding them to your projects (internal) requirements base. Right next to all the explicit feature requests or functional requirements you get from your customer/Product Owner (cf. After the game is before the game). Provided that you can and that you do determine whether these properties are fulfilled with every delivery from now on, you will monotonically improve the quality (qualities) of your software. Quality does not start with testing, it starts with deciding, what our software should be like.