Você está na página 1de 29

Tutorial on Methods for Interpreting and

Understanding Deep Neural Networks

Wojciech Samek
Grégoire Montavon
Klaus-Robert Müller

(Fraunhofer HHI) (TU Berlin) (TU Berlin)

1:30 - 2:00 Part 1: Introduction

2:00 - 3:00 Part 2a: Making Deep Neural Networks Transparent

3:00 - 3:30 Break

3:30 - 4:00 Part 2b: Making Deep Neural Networks Transparent

4:00 - 5:00 Part 3: Applications & Discussion

ICASSP 2017 Tutorial — W. Samek, G. Montavon & K.-R. Müller

Before we start

We thank our collaborators !

Alexander Binder
Sebastian Lapuschkin

(SUTD) (Fraunhofer HHI)

Lecture notes will be online soon at:

Please ask questions at any time !

ICASSP 2017 Tutorial — W. Samek, G. Montavon & K.-R. Müller /29

-R. Samek. Montavon.Tutorial on Methods for Interpreting and Understanding Deep Neural Networks W. G.-R. Müller . Müller Part 1: Introduction ICASSP 2017 Tutorial — W. G. Samek. Montavon & K. K.

Montavon & K. Müller 4 /29 . Samek.-R. G.Recent ML Systems achieve superhuman Performance Deep Net outperforms humans AlphaGo beats Go in image classification DeepStack beats human champ professional poker players Autonomous search-and-rescue drones outperform humans Computer out-plays Deep Net beats human at humans in "doom" recognizing traffic signs IBM's Watson destroys humans in jeopardy ICASSP 2017 Tutorial — W.

Müller 5 /29 . G.-R. Samek. Montavon & K.From Data to Information Huge volumes of data Solve task Interpretable Information ex tra Computing power ct Deep Nets / Kernel Machines / … Information (implicit) ICASSP 2017 Tutorial — W.

From Data to Information Interpretability AlexNet Clarifai VGG GoogleNet ResNet (16. Müller 6 /29 .3%) (6.1%) (7. G.4%) (11. Samek.7%) (3.57%) Performance Interpretable Data Information for human Crucial in many applications (industry. sciences …) ICASSP 2017 Tutorial — W. Montavon & K.-R.

but easily interpretable Can be very complex “global explanation” “individual explanation” ICASSP 2017 Tutorial — W. G. Müller 7 /29 . Poor fit. Samek.Interpretable vs.-R. Montavon & K. Powerful Models ? Linear model Non-linear model vs.

G. Poor fit. Samek. Müller 8 /29 .000 neurons We have techniques to interpret and explain such complex models ! ICASSP 2017 Tutorial — W.-R. Montavon & K. Powerful Models ? Linear model Non-linear model vs. but easily interpretable Can be very complex “global explanation” “individual explanation” 60 million parameters 650. Interpretable vs.

-R. Müller 9 /29 . G.Interpretable vs. Samek. sparsity …) ICASSP 2017 Tutorial — W. Powerful Models ? train best train interpretable interpret it vs. model model suboptimal or biased due to assumptions (linearity. Montavon & K.

G. Montavon & K.Dimensions of Interpretability Different dimensions prediction of “interpretability” “Explain why a certain pattern x has been classified in a certain way f(x).” ICASSP 2017 Tutorial — W. Müller 10 /29 . Samek.-R.” data “Which dimensions of the data are most relevant for the task.” model “What would a pattern belonging to a certain category typically look like according to the model.

Müller 11 /29 .Why Interpretability ? 1) Verify that classifier works as expected Wrong decisions can be costly and dangerous “Autonomous car crashes. Samek. “AI medical diagnosis system because it wrongly recognizes …” misclassifies patient’s disease …” ICASSP 2017 Tutorial — W. G. Montavon & K.-R.

Samek. Müller 12 /29 .Why Interpretability ? 2) Improve classifier Generalization error Generalization error + human experience ICASSP 2017 Tutorial — W.-R. Montavon & K. G.

Müller 13 /29 .” ICASSP 2017 Tutorial — W. G.-R.” (Fan Hui) “Learn about the human brain. I've never seen a human play this Old promise: move. Samek. Montavon & K.Why Interpretability ? 3) Learn from the learning machine “It's not a human move.

G.” survive with probability __” What to do with this Great !!! information ? ICASSP 2017 Tutorial — W. Montavon & K.Why Interpretability ? 4) Interpretability in the sciences Stock market analysis: In medical diagnosis: “Model predicts share value “Model predicts that X will with __% accuracy.-R. Samek. Müller 14 /29 .

find genes linked to cancer.Why Interpretability ? 4) Interpretability in the sciences Learn about the physical / biological / chemical mechanisms. Montavon & K. G.g. Müller 15 /29 . identify binding sites …) ICASSP 2017 Tutorial — W.-R. (e. Samek.

Why Interpretability ? 5) Compliance to legislation European Union’s new General “right to explanation” Data Protection Regulation Retain human decision in order to assign responsibility. Montavon & K. G. Samek. “With interpretability we can ensure that ML models work in compliance to proposed legislation.” ICASSP 2017 Tutorial — W. Müller 16 /29 .-R.

Samek. G. Montavon & K. • Retain human decision in order to assign responsibility.-R. • “Right to explanation” Interpretability as powerful engineering tool • Optimize models / architectures • Detect flaws / biases in the data • Gain new insights about the problem • Make sure that ML models behave “correctly” ICASSP 2017 Tutorial — W.Why Interpretability ? Interpretability as a gateway between ML and society • Make complex models acceptable for certain applications. Müller 17 /29 .

Techniques of Interpretation ICASSP 2017 Tutorial — W.-R. Samek. Müller 18 /29 . Montavon & K. G.

-R. G.find pattern maximizing activity of a neuron Explaining decisions crucial for many (individual) practical applications . Samek. Müller 19 /29 .Techniques of Interpretation Interpreting models better understand (ensemble) internal representation .verify that model behaves as expected ICASSP 2017 Tutorial — W. Montavon & K.find prototypical example of a category .“why” does the model arrive at this particular prediction .

Montavon & K. doctor. patient). G. Samek. ICASSP 2017 Tutorial — W.-R. Müller 20 /29 .Techniques of Interpretation In medical context • Population view (ensemble) • Which symptoms are most common for the disease • Which drugs are most helpful for patients • Patient’s view (individual) • Which particular symptoms does the patient have • Which drugs does he need to take in order to recover Both aspects can be important depending on who you are (FDA.

Techniques of Interpretation Interpreting models .find pattern maximizing activity of a neuron cheeseburger goose car ICASSP 2017 Tutorial — W. Müller 21 /29 .find prototypical example of a category . Montavon & K.-R. G. Samek.

G. Samek.find pattern maximizing activity of a neuron cheeseburger goose car simple regularizer (Simonyan et al. 2013) ICASSP 2017 Tutorial — W.-R.find prototypical example of a category .Techniques of Interpretation Interpreting models . Montavon & K. Müller 22 /29 .

Samek. Montavon & K.Techniques of Interpretation Interpreting models .-R.find pattern maximizing activity of a neuron cheeseburger goose car complex regularizer (Nguyen et al. G.find prototypical example of a category . Müller 23 /29 . 2016) ICASSP 2017 Tutorial — W.

Techniques of Interpretation Explaining decisions . G.-R.verify that model behaves as expected ICASSP 2017 Tutorial — W.“why” does the model arrive at a certain prediction . Montavon & K. Müller 24 /29 . Samek.

Techniques of Interpretation Explaining decisions .“why” does the model arrive at a certain prediction . Samek.-R.verify that model behaves as expected .Sensitivity Analysis .Layer-wise Relevance Propagation (LRP) ICASSP 2017 Tutorial — W. G. Müller 25 /29 . Montavon & K.

G.Techniques of Interpretation Sensitivity Analysis (Simonyan et al. Samek. Müller 26 /29 .-R. Montavon & K. 2014) ICASSP 2017 Tutorial — W.

G. 2015) “every neuron gets it’s share of relevance depending on activation and strength of connection..” Theoretical interpretation Deep Taylor Decomposition (Montavon et al. 2017) ICASSP 2017 Tutorial — W. Samek.Techniques of Interpretation Layer-wise Relevance Propagation (LRP) (Bach et al.-R. Müller 27 /29 . Montavon & K.

G.Techniques of Interpretation Sensitivity Analysis: LRP / Taylor Decomposition: “what makes this image “what makes this image less / more ‘scooter’ ?” ‘scooter’ at all ?” ICASSP 2017 Tutorial — W. Müller 28 /29 . Samek.-R. Montavon & K.

applications. interpretability in the sciences. Samek. Müller 29 /29 .-R. G. More to come Part 2 Part 2 Part 3 quality of explanations. Montavon & K. discussion ICASSP 2017 Tutorial — W.