Explainable AI (XAI) has been growing in popularity over the past few years as the adoption of AI has increased among companies. One of the biggest challenges that businesses face is explaining the complexities behind an AI model to a non-technical audience, which just means that AI is something of a black box for them. For this reason XAI is also called the “glass-box approach” because it aims to cut through the confusion and provide users with techniques that can be leveraged to break down these complex models.
This blog serves as an end-to-end guide regarding the importance of XAI, some practical techniques to implement XAI, and common challenges faced by companies and individuals throughout the process of implementing XAI.
Overview of XAI
XAI is a framework that can be integrated with an existing ML model to understand the output of AI or ML algorithms. This framework is not only used to explain the results behind an ML algorithm but also to get feedback on results and re-train the model based on that feedback. Feedback can be received in terms of suggestions from business stakeholders to detect bias, and then one can tweak the hyperparameters and re-train the model.
XAI is a rule-based approach of early AI that exists in stark contrast with the concept of the “black box” in machine learning. It attempts to answer the transparency issues in the decision-making rationale of the system.
The importance of XAI
There are numerous benefits that arise out of implementing XAI; the top six are listed below:
Reduces errors: The more visibility one has over the ML framework, the less space there is for errors that have the potential to cost a company thousands of dollars. Errors are not limited to choosing the wrong algorithm; they can also look like a billion-dollar lawsuit!
Curbs model bias: XAI allows you to capture bias before your model is deployed and eventually act on it. Bias can be found in all shapes and forms; your model might be biased (favorable) to one sex, company, nationality, etc. It is essential to treat bias so that one of the target populations is included in your model and predictions.
Confidence and compliance: Seeking compliance approvals is an essential step in highly regulated industries. XAI frameworks allow you to explain AI models thoroughly and get the required approvals.
Model performance: One of the benefits of XAI is that it gives you the ability to gather feedback from stakeholders and re-train your model. This allows you to attain optimal model performance.
Informed decision-making and transparency: One of the goals of XAI is to make AI more collaborative. This means all AI-related decisions have the potential to include everyone.
Increased brand value: Statistically, customers are more likely to invest in your brand if they find it trustworthy. Using explainable AI frameworks strengthens your users’ trust and makes them more comfortable handing over their data.
Practical techniques to implement XAI
1. LIME (Local Interpretable Model Agnostic Explanations)
LIME is a model-agnostic technique, meaning it can be applied to any model. The goal of this model is to provide a local approximation. The local approximation can be found by training the model on a small perturbation of the original instance.
How to implement LIME:
1. Choose the instance for which you want to have an explanation of its black box prediction.
2. Discompose your dataset to create fake data points to produce the black box predictions.
3. Weight the new samples according to their proximity to the instance of interest.
4. Train a weighted, interpretable model (such as linear/logistic regression and decision trees) on the dataset with the variations.
5. Explain the prediction by interpreting the local model.
The output of LIME is a list of features alongside their respective contribution to the model’s prediction; this gives you a clear understanding of each contributing feature.
2. Fairness and bias testing
Bias often creeps into data, so it is essential to check the fairness and bias within the data first. Bias can come in all shapes and forms, but having biased data will lead to the exclusion of certain populations and thus an incomplete model.
Below are a few simple ways you can check forbias in your model:
- A considerable number of missing values could be an indicator of missing an entire population, which means that the predictions might be favorable for one population exclusively.
- Outliers could be another indication pointing toward bias.
- Data skewness is another critical factor that should be analyzed because it is often an indicator that one population is more favorable than the other.
3. SHAP (Shapley Additive Explanations)
Shapley Values are a common technique used to assess feature importance. The output is an excellent plot with each of the feature’s importance. For example, if you were trying to predict how many people will test positive for the coronavirus each year, you can assess the influence of removing or adding a feature to the overall predictions
Let’s say the average prediction is 50,000 people. How much has each feature value contributed to the prediction compared to the average prediction?
Comparing it to Game Theory, the “game” is the prediction task for the given dataset. The “players” are the feature values of the instance that collaborate to predict a value.
In our coronavirus prediction example, features such as has_vaccinated, access_to_masks, and underlying_conditions came together to achieve the prediction of 52,000. Our goal is to explain the difference between the actual prediction (52,000) and the average prediction (50,000)—a difference of 2,000.
A possible explanation could be that has_vaccinated contributed to 500, access to masks contributed to 500, and underlying_conditions contributed to 1,000. The contributions add up to 2,000– the final prediction minus the average predicted coronavirus cases
Shapley Values for each variable are trying to find the correct weight such that the sum of all Shapley values is the difference between predictions and the average value of the model.
Challenges of XAI
Limited models availability: Though a few model-agnostic approaches, such as LIME, are available, there are still limited frameworks for complex ML methods.
Accuracy vs. interpretability: XAI seems tempting for an interpretable model, but one may have to compromise the model’s accuracy. This is a tough choice that any data scientist must make when opting for interpretable models.
XAI has proven itself to be a glimmer of hope for companies struggling to implement AI solutions as a result of pushback from compliance enforcement and other authorities. It is being adopted by doctors to predict the mortality rates and suitability of treatment options, as well as insurance industries to predict fraudulent claims.
All in all, XAI represents a way for AI ML models to solve impactful problems for the greater good. It will be interesting to observe the adaptability curve for these frameworks in the coming years and across industries.
