When AI became a widely used technology, its main focus was accuracy, and the business perception around AI today is still very much accuracy-driven. While it is important, accuracy should not be the be-all and end-all KPI for AI solutions. Too much focus on accuracy can hinder the true added value of AI for companies.
Accurate model, bad results
Here’s an example: we worked with FPS BOSA, a Belgian public agency. BOSA was in the process of making government websites more accessible, following an EU directive. They used an internal accessibility checker that was mostly rule-based. Superlinear was tasked with developing a flexible, open-source machine learning model that could automate most of the accessibility checks (for example, checking if the contrast between text and background is high enough, for maximum readability).
This is where accuracy comes into play. BOSA wanted the model to be accurate, but not perfect. Striving for 100% accuracy could lead to false positives becoming an issue. These are accessibility issues wrongly detected by the checker. Having to account for false positives would have created a lot of friction in the process and therefore having a too high model accuracy would have been counterproductive. This example shows that focusing too much on the accuracy of the models and algorithms can only get you so far. Accuracy is just one part of the story.
AI algorithm saturation
The algorithms - AI’s architecture, and the formulas that we use - are becoming more or less saturated. This means that every new AI application (like AI vision or reading) is only marginally better than its predecessors, which leads to only slight improvements rather than marked jumps.
If you have the right team, the algorithm is not the hard part anymore, and there are multiple other dimensions to AI solutions that present greater challenges, especially in today’s world. They go way beyond accuracy, and even sometimes in their way. These bigger challenges include 5 different angles.
These dimensions above often fall under the “responsible AI” moniker and this is why accuracy should never be the only metric for AI models.
Beyond accuracy - do it like Tesla
To go beyond accuracy, it all starts with the basics: the experts you will be working with, the business case, and the UX/UI. Responsible AI, the impact of AI on decisions and on people's lives, should also be a priority. This is because of the additional complexity that comes with AI structures, compared to other types of software.
These dimensions that look further than just the accuracy of the algorithm, are the next step in AI becoming more mature. Furthermore, the current nature of AI is very much data-driven. To achieve great results, you need enough data that represents the complexity of a problem or situation. Your model can be the best in the world, but if the world isn’t represented in your data, it is bound to underdeliver.
Algorithms are basically tied up to that data. This is why Tesla sees a strategic advantage in what they call “train, evaluate, label”.
That cycle - and getting it to work fast and efficiently, is where differences can be made. Not in the model or in the accuracy, but in these types of frameworks. This is where forward-thinking companies should invest.
A pathway to new breakthroughs
In many ways and fields, breakthroughs are a natural extension of “looking beyond”. The relative saturation of algorithms we touched on earlier will have an impact on new, groundbreaking applications within the AI field. Achievements like GPT-3 are impressive, but they are not (yet) real game-changers within AI. New breakthroughs will require new approaches and entirely different AI architectures. Learn more in our work on this topic.
Are you thinking beyond accuracy, with a holistic view on your AI projects? Book a meeting with our specialist to discover how you can achieve impact today.
