Cracking the black box : the quest to understand the machines that run our lives

Abstract

Artificial intelligence (AI) is now part of everyday life. It recommends what we watch online, helps banks approve loans, assists doctors in hospitals and even acts as a digital gatekeeper for who gets hired. Many people enjoy the convenience of these systems, yet, few truly understand how they work. That is where the ‘Explainable AI’ notion comes in. Essentially, it is a growing movement that is aimed at increasing AI transparency, to earn user trust.

For years, AI systems were treated like mysterious ‘black boxes’. You feed information into them and they produce an answer. However, at times, it proves hard to clearly explain how they have reached their conclusions. Even the engineers who have built these systems sometimes struggle to fully understand the internal reasoning behind complex AI models.

This becomes worrying when AI is used in areas such as healthcare, education, banking, policing or public services. Imagine applying for a loan and being rejected by an AI system without any explanation. Alternatively, consider a hospital using AI to help doctors diagnose patients without anyone being able to explain why the system recommended a particular treatment. In such situations, people may naturally ask: Why did the machine decide this? Explainable AI (XAI) tries to answer that question.