Explainable boosting machine

You can have data without information, but you cannot have information without data.

Daniel Keys Moran, science fiction writer

The business impact of machine learning models is becoming increasingly significant whether it is in marketing campaign budget allocation or credit risk management. However, the lack of interpretability and explainability of many machine learning models makes them not only difficult to trust but also difficult to debug. When the end user can comprehend the decision-making process of a machine learning model, they are more likely to accept, trust, and adopt the model’s outputs, giving rise to reliable model-based decisions.

The easiest way to ensure interpretability is to use interpretable models. Consequently, there is a growing interest in the development and utilization of interpretable machine learning models.

Explainable Boosting Machine (EBM) originally developed at Microsoft Research is an inherently interpretable machine learning model that gained increasing popularity in a variety of business applications due to its predictive performance and low computational cost. It falls into a category of ML models known as “Glass box”. Not only it can capture non-linear relationships between dependent and independent variables, but it also has a built-in mechanism to capture interactions among independent variables. at its heart, it uses some of the well-known ML techniques like bagging and gradient boosting.

Explainable Boosting Machine is a special case of ensemble models, which involve training multiple very simple decision trees and combining their predictions to obtain a final prediction.

Imagine we are given a dataset with n features. The first decision tree is trained only on the first feature. The residuals are then calculated and we continue by training another model but this time only using the second feature. The process continues until the last model is trained in a boosting fashion using the last feature. The order of the features will not impact the results since we ensure to keep the learning rate very small. Now we have the first iteration completed. We continue the process for many many iterations, let’s say 10,000 iterations. At the end of the cycle, we have 10,000 small trees trained on feature 1 that can be summarized as a graph. We can extract a graph for all the features. These series of graphs with size n construct our model.

The ability to model complex relationships and interactions while maintaining explainability makes the Explainable Boosting Machine a powerful tool for a wide range of predictive modeling tasks.

Conclusion:

You probably heard about the trade-off between explainability and accuracy of machine learning models. The more accurate the model is, the more complex it is and less explainable. EBMs violate this trade-off. With EBMs you will get high accuracy while getting a highly explainable model with an added bonus of editability. This makes EBMs the perfect candidate of choice for many real-world applications that support model-based decisions.

Inherently interpretable ML models

Interested in this topic? Read more on “How to design inherently interpretable machine learning models” by Sudjianto and Zhang (2021)

In my next post, I should follow up on the fairness, weakness detection, reliability, robustness, and resilience of XAI.