In an effort to design alternatives to opinion-based discourse at scale online that exploded with polarization, toxicity, and misinformation in the last two US election cycles, the Local Voices Network hosts small group conversations in partnership with grassroots and public organizations, where the emphasis is on the respectful sharing of personal experiences rather than opinion. Facilitation by trusted community leaders and organizers, as well as active recruitment of voices that are traditionally underheard or actively excluded in civic processes yields intimate conversations that often dip into powerful personal narrative, shedding light on a constellation of public issues in the process. Quantitative and qualitative insights from these conversations have been used to inform questions in the mayoral candidate debate last year in Boston, as well as the hiring of a police chief in Madison, Wisconsin.
To date, the LVN platform has largely used human sensemakers to qualitatively analyze the transcripts and summarize findings. We present methods for automatically analyzing the conversations at scale. Checking if facilitators followed the conversation guide, extracting interesting passages that address questions of interest, and coding speaker turns for topical themes are all processes that can take months for human sensemakers who work on these conversations. Similar challenges face organizations and researchers who make sense of focus group transcripts or semi-structured interviews en masse for research, so this problem has far wider reach than just this data set of LVN conversations.
We use sentence embeddings to check for facilitator adherence to the conversation guide, then fine-tune a GPT-3 model to identify structural labels for speaker turns with remarkable accuracy. Bringing humans together with AI methods can save human experts immense time!