[2011.09903] Impact of Accuracy on Model Interpretationsopen searchopen navigation menucontact arXivsubscribe to arXiv mailings

Model interpretations are often used in practice to extract real world insights from machine learning models. These interpretations have a wide range of applications; they can be presented as business recommendations or used to evaluate model bias. It is vital for a data scientist to choose trustworthy interpretations to drive real world impact. Doing so requires an understanding of how the accuracy of a model impacts the quality of standard interpretation tools. In this paper, we will explore how a model's predictive accuracy affects interpretation quality. We propose two metrics to quantify the quality of an interpretation and design an experiment to test how these metrics vary with model accuracy. We find that for datasets that can be modeled accurately by a variety of methods, simpler methods yield higher quality interpretations. We also identify which interpretation method works the best for lower levels of model accuracy.

1 mentions: @rasbt
Date: 2020/11/20 17:22

Referring Tweets

@rasbt "Impact of Accuracy on Model Interpretations" -- was just reading this interesting preprint by Liu and Udell. As always, simulations on real data can be messy, but the conclusions are interesting t.co/1Ei4GFt2og 1/2

Related Entries

Read more Stat453: Intro to Deep Learning (SS20) -- L12 Intro to Convolutional Neural Networks (Part 1) - You...
0 users, 1 mentions 2020/04/02 15:51
Read more [2002.07162] AIBench: An Agile Domain-specific Benchmarking Methodology and an AI Benchmark Suiteope...
0 users, 1 mentions 2020/05/16 15:51
Read more [2009.01564] Can AutoML outperform humans? An evaluation on popular OpenML datasets using AutoML Ben...
0 users, 1 mentions 2020/09/04 23:21
Read more StackingCVClassifier - mlxtend
0 users, 1 mentions 2020/11/10 15:51
Read more OneRClassifier -- "One Rule" for Classification - mlxtend
0 users, 1 mentions 2020/11/26 05:28