What is Software Quality?

“We offer 3 kinds of services: GOOD-CHEAP and FAST, but you can pick only two.” A well-known statement in Project Management. Meaning that you cannot have all three of them during your development and you will have to balance and make your choices.

GOOD & CHEAP won’t be FAST, FAST & GOOD won’t be CHEAP and CHEAP & FAST won’t be GOOD. True?

GOOD is clearly referring to quality, FAST is referring to speed and CHEAP is referring, of course, to money. If you want to develop something of high quality it will cost you either a lot of money and/or will need time to develop.

Let’s put the GOOD-FAST-CHEAP triangle in perspective of software development and see whether this triangle is true for software development. I would say, it is not. I would reformulate. The only way to be FAST and to be CHEAP is to be GOOD.

Being FAST.

Let’s have a closer look at being fast. Being fast is mainly determined by the level of complexity. The more complex the problem to be solved the longer it takes. Of course, this needs no further explanation. Complexity comes from two directions. The first one is external, meaning by requirements. Requirements will determine the complexity of the product and therefore are of high influence on costs and duration of the needed development. The second direction of complexity comes from, is internally. The level of the internal quality of software (What is Software Quality?) determines the complexity to deal with. Internal quality of software is the quality of the architecture/design and the quality of the code. Whenever you have a complex design, possibly caused by increasing technical debt (Help…, my software rots!), it is more complex to add features and adapt the software. If your code is not well readable it will take more time to understand to be able to adapt. As you can see, low quality will lead to higher complexity and thus being slower.

However, whenever you would start a green-field software development, from scratch, no existing software to be re-used, there would be no internal complexity yet. In such a case it is possible to be fast and cheap without being good (low quality). But, internal complexity will increase and speed will decrease accordingly. So even being fast without being good will only last for a short period as visualized in the picture below.

Since software is evolving and imperfections are accumulated during development your technical debt will grow and with that the internal complexity will grow resulting in becoming slow.

Being CHEAP.

Ok, now you might be convinced that you will need GOOD to be FAST. So according to the GOOD-FAST-CHEAP triangle you will not be CHEAP.

Hmmm……, is that the case? Of course software development is not cheap, but due to one of the properties of software development there is a direct relation between duration and costs in software development. Costs are determined mainly by labor in software development. Engineers developing and maintaining software are the main cost item of a software development project. This implies that there is a direct relation between duration and costs. Whenever it takes long to develop, it will cost you more. From this we can simply conclude that being FAST in software development equals being CHEAP for which, as explained above, low complexity and thus GOOD is needed.

Therefore, in software development the only way to be FAST and to be CHEAP is to be GOOD.

This week I wanted to create an user account with the NS (Nederlandse Spoorwegen, Dutch Railways) via their website. After entering the necessary details everything seemed to progress well until I received a so-called ‘confirmation-mail’ in which I had to click a link to finalize the process. This click directed me again to the website of the NS to fill out some additional information. However, I was confused they even needed more additional information and I closed the web window before filling it out. When clicking the link in the confirmation-mail again, I got an error message informing me that this was an invalid link. Multiple ‘re-clicks’ did not improve the situation and I realized I made a mistake in closing this web window. Frustrated as I was I decided to call the help desk, but after waiting for more than 15 minutes in the waiting queue I decided to search for an alternative flow, hoping I would find one.

As it is impossible to test every possible execution path through code, it is important to think about clever scenarios of testing. These scenarios need to be chosen to cover as much as possible to increase the chance of finding present defects as much as possible.

In defining these testcases, three possible kinds of scenarios can be considered:

• Happy Flows
• Alternative Flows
• Sad Flows

Happy flows

Test scenarios performing the intended usage of the software are called happy flows. Normally, these test scenarios would cover the majority of the software’s usage.

Alternative flows

An alternative flow is a test scenario testing a flow other than the intended usage of the software that, however, will result in the completion of the scenario’s goal. By means of an alternative flow, an alternative execution path through the code is taken to achieve the goal.

Sad flows

Sad flows are test scenarios testing error situations in which the intended goal of the flow is not achieved. An example would be to provide invalid input to the software. A sad flow tests how the software reacts to error situations.

Still, when defining different flows for testing a user story, it seems to be most difficult to think about alternative flows. Which flow of steps or actions, which are not specified, will still deliver the desired result to the customer? This brings me back to my, so far, failed user account request with the NS.

Apparently I was in the situation in which the happy flow in requesting a user account with the NS was not working anymore as I got the error message of an invalid link. And of course logging in using my e-mail address as user name, which I already entered in the first step of the registration, was also not feasible. Simply because I did not yet set a password.

When realizing I did not yet set a password, I thought to try to use the always present ‘forgotten password’ link; a savior whenever you cannot continue to login. Surprisingly,  clicking this ‘forgotten password’ link and entering my e-mail address resulted into receiving a new confirmation-mail with a link to click to finalize the user account creation process. And guess what….? Yes! This link worked and showed me the webpage in which the additional information for the user account was asked. After entering this additional information, including setting the password, my account was created….. I was relieved I had found an alternative flow to create my user account.

This alternative flow would look like:

1. Request creation of an user account.
2. Fill out needed information.
3. Wait for confirmation mail.
4. Click the link in the confirmation mail.
5. Do not fill out additional information but immediately close the ‘confirmation web window’.
6. Open login page.
7. Request to reset the password via the ‘forgotten password’ link.
8. Enter your e-mail address.
9. Wait for confirmation mail.
10. Click the link in the newly received confirmation mail.
11. Fill out additional requested information including a password.
12. Finalize the process.

I wonder whether the NS did consider this alternative flow in their testing…….

Last week an engineer, who I did not know, approached me for having a short talk. He explained to me that he is working on a piece of embedded software containing significant levels of technical debt. He would like to achieve that a certain percentage of the team effort would be spent on handling technical debt, only 10% would already be appreciated.

However, due to time pressure to deliver functionality the product owner does not want to grant the percentage for handling technical debt. The engineers question was whether I had some tricks and tips which could help him. He had read some parts of my book “What is Software Quality?”.

I pointed out that we, as a software community, need to stand for our profession. It is our responsibility to mitigate technical debt in a context of finding the right balance between short term and long term. We need to engage the dialogue on the subject with our stakeholders like product owners, project managers and management in general. We need to point out the consequences of technical debt and why it needs to be addressed.

First of all we need to have a look at ourselves; what can we do to mitigate technical debt. Like, developing software as it should be; applying good development practices with craftsmanship and producing clean code while demonstrating the needed discipline to do so despite time pressure. And even then, technical debt is inevitable and will creep into our software. So, we need to do something additional.

Considering technical debt itself we can distinguish between small TD and big TD. An example of small TD is this compiler warning which is not yet fixed. Or this function or method with a too high cyclomatic complexity. Small TD can be handled by the boy-scout rule; leaving code cleaner than you encountered it. Whenever altering a piece of code, get rid of small TD in this piece of code. We always should apply the boy scout rule and, in my opinion, we do not need to ask for ‘permission’ to do so. It is part of our craftsmanship.

An example of big TD is a required redesign of a module which will take a significant amount of effort. In this case the technical debt is so big that it cannot be solved instantly. We need to register big TD on e.g. a Technical Debt Backlog (TDB). This TDB then needs to be considered on a regular basis in the context of the features to be planned for the next release. Which TDB-items are needed to be addressed for the implementation of the prioritized User Stories? Preferably TDB-items will be ‘connected’ to one or multiple User Stories, thus whenever the User Story is prioritized the ‘connected’ TDB-items will be as well.

To be able to discuss and prioritize big TD with the stakeholders it is important the stakeholders do understand what technical debt is and which consequences it has. Therefore, they need to be educated by us; by the software professionals. In my book I try to explain technical debt in such a way that it can be understood by people who do not have software knowledge as well.

Engaging with the stakeholders and explaining and discussing the consequences of technical debt is necessary; using metaphors, explaining the complexity of execution paths, visualizing the size of software and showing the vast diversity of technical debt and pointing out the long term consequences of technical debt on development speed and efficiency. Personally, I like to talk about ‘a sustainable pace of development’ instead of ‘development speed’, in accordance with the Formula-1 in which they talk about ‘race-pace’ instead of ‘race-speed’. This has reason, focus on speed in Formula-1 will increase tire-wear like focus on speed in software development will increase software-wear. In both cases velocity declines. Let’s take our responsibility and start discussing technical debt with our stakeholders, using metaphors like tire-wear in Formula-1.

Software is everywhere and its relevance is growing at a phenomenal pace. It would not be an exaggeration to say that software runs the world. And still, as expressed in a previous blog of mine, “Trial and Error Programming”, programming skills need to improve significantly. Besides programming skills, software engineering skills also need to improve. As an example, how is that not many software engineers are using UML for modeling requirements and design?

Low threshold to become a software engineer

Let’s have a closer look at the issues related to becoming a software engineer. If there is a lack of professionals in a certain profession, people will jump on it. There will be jobs all over the place. Combine this with the very low threshold to produce any software and it will be very easy to jump on the possibilities we have due to the lack of software engineers.

How easy is it to download a compiler and write and execute your first program? How easy is it to search the internet for any piece of code and copy it into your program to get a result? How easy is it to learn Python and start programming? Unlikely in other engineering disciplines, everybody can start programming easily. Tools are available and downloadable from the internet; on the other hand, production is done by compiling. With partial understanding of a programming language, first results can be easily achieved. How different is it from building your first electronics circuitry? Building electronic circuitry without understanding voltage, current, resistors, coils, capacitors, transistors, or digital circuitry is not even possible.

The low threshold to becoming a software engineer coupled with the high need of software engineers in the market is the reason why we do have high numbers of software engineers who are not adequately educated.

Software engineering is more than programming alone.

If we take software development seriously, we should make sure that our software engineers are well trained in necessary software engineering practices, next to “only” knowing a programming language. Engineering practices such as requirements engineering, modeling, design methodologies and design patterns, Clean Design, algorithms, Clean Code, and testing assume even greater importance than knowing the programming language itself. Additionally, our software engineers should be educated in computer architectures, computer networks and security, databases, operating systems, communication protocols, and so on.

In order to build high quality software, it is not sufficient to “only” know a programming language. Therefore, should we certify our software engineers to ensure they are well educated in relevant topics associated with software engineering? This is especially important for determinative software in our society, like software in aerospace or automotive or in our financial systems. Certification of our software engineers would help us ensure that only well-educated software engineers are working on our most critical software!

by Ger Cloudt, author of “What is Software Quality?”

One of the struggles of software development is the assurance of quality. Sure, if you talk about quality assurance, people will refer to testing. Or, one might want to assure quality by applying proper processes and best practices. Process improvement and reference models like ASPICE and CMMi are used in the industry to increase quality or to determine the capability of the organization to develop high quality software. However, when using reference models like ASPICE or CMMi there is a pitfall, using ASPICE or CMMi might disturb the balance between process and skill.

Process and Skill

According to Wikipedia.org a process is defined as “a series or set of activities that interact to produce a result”.
Additionally, to execute a process one needs to have a skill. According to Wikipedia.org a skill is “the learned ability to perform an action with determined results”.

Process and skill are complementary, so you will need both. To execute actions or activities you need a certain level of skill, to achieve a result multiple actions need to be structured by a process. However, the importance of process compared to the importance of skill is dependent on the kind of activities to be performed.

In figure below it is tried to visualize the relevance of process versus the relevance of skill to perform a certain task. On the left side of the figure tasks are positioned for which process is highly relevant and the needed skill-level is relatively low. An example is assembly-line work. Many people are able to perform these tasks as the needed skill-level is low, however the order of activities to be performed are very strict and therefore the process relevance is high. The same applies to building this furniture bought at IKEA. A manual describing how to assemble (process) the cupboard or chair is important whilst the needed skill-level is quite modest.

At the other side of the scale we have activities for which process relevance is low but a very high skill-level is needed which only a few people master. An example would be painting the Mona Lisa as Leonardo da Vinci did.

Software Engineer

The question is where to position the software engineer in the figure? If you draw a line reflecting the position at which process relevance and needed skill-level are equal, would the software engineer be placed on the left or right of this line? In my humble opinion the software engineer would be positioned on the right of this line, meaning that I think the needed skill-level is more important than the process for performing the role of software engineer. To put it differently: a process is a tool that helps you apply your skills. If the skill isn’t available, a process doesn’t help you. However, if the skill is available, a process can help to apply this skill.

ASPICE or CMMI level

Process improvement models like ASPICE and CMMI define different maturity levels which are used in some industries as a minimum requirement for software or system suppliers. This requirement or target setting to be process compliant within a certain process reference model disturbs the balance between process and skill.

By setting a target to be process compliant and achieve a certain ASPICE or CMMI maturity level the focus is directed to the less important part in the process-relevance versus needed skill-level balance. Achieving the target-level of process compliance does not tell the full story and therefore there is the danger that it is assumed that the process compliance is equal to achieving high-quality software which it is not. Process compliance assures that defined and necessary activities are performed, however it does not say anything about how well these activities are performed. Therefore, additionally to checking process compliance, one should check the content, the result of the activities, as well. Simply, because for performing the activities well, skills are needed. Good skills will lead to good results, bad skills will lead to bad results.