Black box and white box models for interpretable AI

Use model properties 、 The local logical representation and the global logical representation generate explanations from the black box model

Quickly review ：XAI and NSC

Interpretable AI (XAI) Committed to the development of human （ Include users 、 Developer 、 Policy makers and auditors ） In essence, it is easier to understand the artificial intelligence model . Neural symbolic computation (NSC) The process combines the sub symbol learning algorithm with the symbol reasoning method . therefore , We can assert that neural symbolic computing is a sub field of interpretable artificial intelligence .NSC It is also one of the most applicable methods , Because it relies on combining existing methods and models .

If explicability refers to the ability to describe things meaningfully in human language . Then it can be said that , It is the original information ( data ) Mapping to meaningful symbolic representations of humans ( for example , English text ) The possibility of .

By extracting symbols from sub symbols , We can make these sub symbols interpretable .XAI and NSC Are trying to make the sub symbol system easier to explain .NSC More about the mapping of sub symbols to symbols , Through the interpretability of logical design : Symbolic reasoning on the learning representation of sub symbols .XAI Not that specific , More about the explicability of all the nuances , Even if interpretability is wrapped in an unexplained model . If extracting symbols from sub symbols means interpretability , that XAI Contains NSC.

Neuro-Symbolic Concept Learner

Mao Et al. Proposed a new NSC Model ,Neuro-Symbolic Concept Learner, It follows these steps :

• Image classifiers learn to extract sub symbols from images or text segments ( Numbers ) Express .
• then , Each semiotic representation is associated with a symbol that human beings can understand .
• then , The symbol reasoner checks the embedded similarity of the symbol representation
• Training continues , Until the output accuracy of the inference engine is maximized by updating the representation .

White box and black box models

AI The model can be (i) White box or (ii) Black box .

The white box model can be explained by design . therefore , It does not require additional functionality to explain .

The black box model itself cannot be explained . therefore , In order to make the black box model interpretable , We must use several techniques to extract interpretations from the internal logic or output of the model .

The black box model can be explained with the following

Model properties : Show the specific properties of the model or its predictions , Such as (a) Sensitivity to attribute changes , or (b) The model component responsible for a given decision ( Such as neurons or nodes ) The identification of .

Local logic : A representation of the internal logic behind a single decision or forecast .

Global logic : The representation of the entire internal logic .

therefore , The image below shows AI Subcategories of interpretability of models :

Rule based interpretability and case-based interpretability

In addition to the logical distinction of interpretable models , We also identified two common types of interpretation , All the above models can be used to provide explanations :

Rule based interpretation : Rule - based interpretability depends on generating “ A set of formal logic rules , These rules form the internal logic of a given model .”

Case based explanation : Rule - based interpretability depends on providing valuable input - Output pair ( Both positive and negative ), To provide an intuitive understanding of the internal logic of the model . Case based explanations depend on the human ability to infer logic from these pairings .

Comparison of rule-based and case-based learning algorithms

Suppose our model needs to learn how to make Apple Pie Recipes . We have blueberry pie 、 Cheese Cake 、 Recipes for shepherd pie and plain cake . Rule based learning attempts to come up with a common set of rules to make all types of desserts ( The urgent method ), The case-based learning method summarizes the information required for a specific task according to the needs . therefore , It will look for the most similar dessert to apple pie in the available data . then , It will try to make small changes in similar recipes to customize .

XAI: Design white box model

Including rule-based and case-based learning systems , We have four main white box designs :

Handmade expert system ;

Rule based learning system : From inductive logic programming 、 Algorithm for learning logic rules from data such as decision tree ;

Case study system : Case based reasoning algorithm . They use examples 、 Case study 、 Precedents and / Or counter examples to explain the system output ; and

Embedded symbol and extraction system : More biologically inspired algorithms , Such as neural symbol calculation .

The final summary

In this paper , We :

1. Brief introduction XAI And NSC Similarities and differences ;
2. Define and compare black box and white box models ;
3. Ways to make the black box model explicable ( Model properties , Local logic , Global logic );
4. Compare rule-based interpretation with case-based interpretation , And give an example .

author ：Orhan G. Yalçı

Original address ：https://towardsdatascience.com/black-box-and-white-box-models-towards-explainable-ai-172d45bfc512