Project

Wearable Reasoner: AI Agents for Intelligence Augmentation

Fluid Interfaces

Groups

Wearable Reasoner: Towards Enhanced Human Rationality through a Wearable AI Assistant

We present "Wearable Reasoner," a proof-of-concept wearable system capable of analyzing if an argument is stated with supporting evidence or not to prompt people to question and reflect on the justification of their own beliefs and the arguments of others.

In an experimental study, we explored the impact of argumentation mining and explainability of the AI feedback on the user through a verbal statement evaluation task. The results demonstrate that the device with explainable feedback is effective in enhancing rationality by helping users differentiate between statements supported by evidence and those without. When assisted by an AI system with explainable feedback, users significantly consider claims given with reasons or evidence more reasonable than those without. Qualitative interviews demonstrate users' internal processes of reflection and integration of the new information in their judgment and decision making, stating that they were happy to have a second opinion present, and emphasizing the improved evaluation of presented arguments.

Published in ACM AHs '20

Based on recent advances in artificial intelligence (AI), argument mining, and computational linguistics, we envision the possibility of having an AI assistant as a symbiotic counterpart to the biological human brain. As a "second brain," the AI serves as an extended, rational reasoning organ that assists the individual and can teach them to become more rational over time by making them aware of biased and fallacious information through just-in-time feedback. To ensure the transparency of the AI system, and prevent it from becoming an AI "black box,'' it is important for the AI to be able to explain how it generates its classifications. This Explainable AI additionally allows the person to speculate, internalize and learn from the AI system, and prevents an over-reliance on the technology.

In order to explore how different types of real-time, AI-based feedback might enhance the user's reasoning in argument based judgment and decision making tasks, we present a prototype device; "Wearable Reasoner,'' a wearable system capable of identifying whether an argument is stated with evidence or not. We conducted a closed environment experimental study where we compared two types of interventions on user judgement and decision making: Explainable AI versus Non-Explainable AI through a device capable of telling the user if an argument is stated with evidence or without.

Using a verbal statement evaluation task, we presented the user with various arguments on socially divisive issues and asked them to evaluate them along three dimensions: 1) level of agreement, indicating their opinion leanings, 2) level of reasonableness, indicating the perceived argumentation quality, and 3) level of willingness to donate for an organization that backs the given claim, indicating the decision making tendency.

Specifically, the following research questions are explored:

RQ1. How will different types of feedback affect participants' judgment and decision making?
RQ2. How will the ability of the AI system to explain its thinking have an effect on user judgment and decision making?
RQ3. How will users evaluate their experience with the Wearable Reasoner?

Our findings demonstrate that our prototype with Explainable AI feedback has a significant effect on users' level of agreement as well as on how reasonable they find the presented arguments to be. When assisted with such feedback, users tend to agree more with claims supported by evidence and consider them more reasonable compared to those without. In qualitative interviews users report their internal processes of speculating and integrating the information from the AI system in their own judgment and decision making, resulting in improved evaluation of presented arguments.

For more results and analysis, please visit our paper below

via ACM AHs '20