Once a test colleague shared a memorable quote “in god we trust, the rest we test”. Besides embracing this quote, I discovered its relevance extends to software development where Artificial Intelligence (AI) is involved. Just for the simple reason that AI is made by men and therefore it is fallible. In my presentation, I will reflect on two relevant cases of AI implementations that underestimated the critical role of software testing together with a proper risk analysis.

The first example is about a company that perhaps overly confident or oblivious to potential risks, conducted no testing when AI was implemented to scan new customers’ driver’s licenses. This had server consequences, allowing individuals with invalid licenses to enter as customers. What was the reason they did no risk assessment and no testing and what test and risk approach should have been applicable to dealing with such a technology next time? The second example is about one of the Netherlands’ most significant cases of injustice, where algorithms, fueled by ‘polluted’ data, compelled AI to make unethical decisions. The aftermath included personal tragedies, and the Dutch government is still actively rectifying the situation. What should have been the role of testing (and quality management) to prevent such situations in the future?

These examples together with smaller, examples will be used to determine the different facets that be used for risk analysis and test approaches when AI is implemented.  Facets that will be discussed are for example biases in data, non-representative data, unclear decision-making and algorithms, privacy concerns, legal impact and scope, ethics, and operational processes.

The examples made clear that testing should inevitably be part of an AI implementation. Testers should be able to have a holistic understanding of the context in which the product is developed. If a tester knows and understands better how to apply those specific AI facets, e.g. during refinements where risk sessions and test approaches are determined. During testing e.g. focusing on using the ‘right’ test data. And know “how to shift right”, because they can point out at which specific risk areas monitoring has the highest value. In my presentation, I will tell more about the approaches I used in my projects which can be seen as interesting practices.

To conclude, the specific facets and approaches should be part of the tester’s skill set these days. Of course, I realize it is unexplored territory for many testers. Still, deep thinking and curiosity (and going to a relevant test conference, like Testing United) will also help them to cover this new area in software development where we learned that AI is not a goddess that we can trust.

Key takeaways: 

1. For what reasons are risk assessments needed while using AI (based on real experiences)
2. What are the angles (or heuristics) can be used to perform a risk assessment while dealing with AI implementations
3. How to set up a test- and quality approach with the outcome of the risk assessment
4. Point of attention while executing the test- and quality approach