Increasing understanding of how to categorize patient symptoms for efficient diagnosis has led to structured patient interviews and diagnostic flowcharts that can provide diagnostic accuracy and save valuable physician time. But the rigidity of predefined questions and controlled vocabulary for answers can leave patients feeling over-constrained, as if the doctor (or computer system) is not really attending to them. I'm Listening is a system for automatically conducting patient pre-visit interviews. It does not replace a human doctor, but can be used before an office visit to prepare the patient, deliver educational materials or triage care, and preorder appropriate tests, making better use of both doctor and patient time. It uses an on-screen avatar and natural language processing to (partially) understand the patient's response. Key is a common-sense reasoning system that lets patients express themselves in unconstrained natural language, even using metaphor, and that maps the language to medically relevant categories.