https://www.nytimes.com/2017/11/21/magazine/can-ai-be-taught-to-explain-itself.html?_r=0
article-can-ai-be-taught-to-explain-itself#explainable-ai1"The disconnect between how we make decisions and how machines make them, and the fact that machines are making more and more decisions for us, has birthed a new push for transparency and a field of research called explainable A.I., or X.A.I" article-can-ai-be-taught-to-explain-itself#explainable-ai1
article-can-ai-be-taught-to-explain-itself#correlation-example1But some of these inferences could be terrifically wrong. Caruana was particularly concerned by something another graduate student noticed about the data they were handling: It seemed to show that asthmatics with pneumonia fared better than the typical patient. This correlation was real, but the data masked its true cause. Asthmatic patients who contract pneumonia are immediately flagged as dangerous cases; if they tended to fare better, it was because they got the best care the hospital could offer. A dumb algorithm, looking at this data, would have simply assumed asthma meant a patient was likely to get better — and thus concluded that they were in less need of urgent care. article-can-ai-be-taught-to-explain-itself#correlation-example1
article-can-ai-be-taught-to-explain-itself#different-details-matter-to-different-people1Caruana may have brought clarity to his own project, but his solution only underscored the fact the explainability is a kaleidoscopic problem. The explanation a doctor needs from a machine isn't the same as the one a fighter pilot might need or the one an N.S.A. analyst sniffing out a financial fraud might need. Different details will matter, and different technical means will be needed for finding them. You couldn't, for example, simply use Caruana's techniques on facial data, because they don't apply to image recognition. There may, in other words, eventually have to be as many approaches to explainability as there are approaches to machine learning itself. article-can-ai-be-taught-to-explain-itself#different-details-matter-to-different-people1