Interpretability Open Source Projects

A game theoretic approach to explain the output of any machine learning model.

A curated list of awesome open source libraries to deploy, monitor, version and scale your machine learning

Advanced AI Explainability for computer vision. Support for CNNs, Vision Transformers, Classification, Object detection, Segmentation, Image similarity and more.

Fit interpretable models. Explain blackbox machine learning.

Model interpretability and understanding for PyTorch

A collection of infrastructure and tools for research in neural network interpretability.

A curated list of awesome responsible machine learning resources.

🔅 Shapash: User-friendly Explainability and Interpretability to Develop Reliable and Transparent Machine Learning Models

StellarGraph - Machine Learning on Graphs

Algorithms for explaining machine learning models

Class activation maps for your PyTorch models (CAM, Grad-CAM, Grad-CAM++, Smooth Grad-CAM++, Score-CAM, SS-CAM, IS-CAM, XGrad-CAM, Layer-CAM)

FedML - The Research and Production Integrated Federated Learning Library: https://fedml.ai

Responsible AI Toolbox is a suite of tools providing model and data exploration and assessment user interfaces and libraries that enable a better understanding of AI systems. These interfaces and libraries empower developers and stakeholders of AI systems to develop and monitor AI more responsibly, and take better data-driven actions.

[ICCV 2017] Torch code for Grad-CAM

A collection of research materials on explainable AI/ML

Interpretable ML package 🔍 for concise, transparent, and accurate predictive modeling (sklearn-compatible).

moDel Agnostic Language for Exploration and eXplanation

Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model in just 2 lines of code.

XAI - An eXplainability toolbox for machine learning

Interpretability Methods for tf.keras models with Tensorflow 2.x

[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Stanford NLP Python library for understanding and improving PyTorch models via interventions

Public facing deeplift repo

A collection of anomaly detection methods (iid/point-based, graph and time series) including active learning for anomaly detection/discovery, bayesian rule-mining, description for diversity/explanation/interpretability. Analysis of incorporating label feedback with ensemble and tree-based detectors. Includes adversarial attacks with Graph Convolutional Network.

Interesting resources related to XAI (Explainable Artificial Intelligence)

Locating and editing factual associations in GPT (NeurIPS 2022)

深度学习近年来关于神经网络模型解释性的相关高引用/顶会论文(附带代码)

👋 Xplique is a Neural Networks Explainability Toolbox

Visualization toolkit for neural networks in PyTorch! Demo -->

A collection of state-of-the-art algorithms for the training, serving and interpretation of Decision Forest models in Keras.

Examples of techniques for training interpretable ML models, explaining ML models, and debugging ML models for accuracy, discrimination, and security.

[JMLR 2023] Quantus is an eXplainable AI toolkit for responsible evaluation of neural network explanations

A library to train, evaluate, interpret, and productionize decision forest models such as Random Forest and Gradient Boosted Decision Trees.

Code for the TCAV ML interpretability project

Making decision trees competitive with neural networks on CIFAR10, CIFAR100, TinyImagenet200, Imagenet

💭 Aspect-Based-Sentiment-Analysis: Transformer & Explainable ML (TensorFlow)

Fast SHAP value computation for interpreting tree-based models

Human-explainable AI.

H2O.ai Machine Learning Interpretability Resources

[Pattern Recognition 25] CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks

Interpretability for sequence generation models 🐛 🔍

Explain & debug any blackbox machine learning model with a single line of code.

The Truth Is In There: Improving Reasoning in Language Models with Layer-Selective Rank Reduction

A Python library for Interpretable Machine Learning in Text Classification using the SS3 model, with easy-to-use visualization tools for Explainable AI :octocat:

💡 Adversarial attacks on explanations and how to defend them

A curated list of awesome Fairness in AI resources

📍 Interactive Studio for Explanatory Model Analysis

This repository introduces MentaLLaMA, the first open-source instruction following large language model for interpretable mental health analysis.

Wanna know what your model sees? Here's a package for applying EigenCAM on the new YOLO V8 model

For calculating global feature importance using Shapley values.

Diffusers-Interpret 🤗🧨🕵️‍♀️: Model explainability for 🤗 Diffusers. Get explanations for your generated images.

OpenXAI : Towards a Transparent Evaluation of Model Explanations

Concept Bottleneck Models, ICML 2020

Awesome Resources for Advanced Computer Vision Topics

Zennit is a high-level framework in Python using PyTorch for explaining/exploring neural networks using attribution methods like LRP.

Layer-wise Relevance Propagation (LRP) for LSTMs.

[ECCV 2020] QAConv: Interpretable and Generalizable Person Re-Identification with Query-Adaptive Convolution and Temporal Lifting, and [CVPR 2022] GS: Graph Sampling Based Deep Metric Learning

A python package for benchmarking interpretability techniques on Transformers.

GraphXAI: Resource to support the development and evaluation of GNN explainers

A Python library for Secure and Explainable Machine Learning