Designing Systems for Cognitive Support

MIT Media Lab / Fluid Interfaces

Publication

Image classification and reconstruction from low-density EEG

Guenther, S., Kosmyna, N. & Maes, P. Image classification and reconstruction from low-density EEG. Sci Rep 14, 16436 (2024). https://doi.org/10.1038/s41598-024-66228-1

Publication

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Wazeer Deen Zulfikar, Samantha Chan, and Pattie Maes. 2024. Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). Association for Computing Machinery, New York, NY, USA, Article 450, 1–18. https://doi.org/10.1145/3613904.3642450

Publication

Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation

Wazeer Deen Zulfikar, Samantha Chan, and Pattie Maes. 2024. Memoro: Using Large Language Models to Realize a Concise Interface for Real-Time Memory Augmentation. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI '24). Association for Computing Machinery, New York, NY, USA, Article 450, 1–18. https://doi.org/10.1145/3613904.3642450

Publication

Putting Things into Context: Generative AI-Enabled Context Personalization for Vocabulary Learning Improves Learning Motivation

Joanne Leong, Pat Pataranutaporn, Valdemar Danry, Florian Perteneder, Yaoli Mao, and Pattie Maes. 2024. Putting Things into Context: Generative AI-Enabled Context Personalization for Vocabulary Learning Improves Learning Motivation. In CHI'24. May 11-16, 2024, Honolulu, HI, USA. ACM, New York NY, USA, 15 pages. https://doi.org/10.1145/3544548.3581039

Publication

LLMR: Real-time Prompting of Interactive Worlds using Large Language Models

Publication

An Accessible, Three-Axis Plotter for Enhancing Calligraphy Learning through Generated Motion

Publication

Enabling Waypoint Generation for Collaborative Robots using LLMs and Mixed Reality

Publication

Synthetic Visual Sensations: Augmenting Human Spatial Awareness with a Wearable Retinal Electric Stimulation Device

Valdemar Danry, Laura Chicos, Matheus Fonseca, Ishraki Kazi, and Pattie Maes. 2024. Synthetic Visual Sensations: Augmenting Human Spatial Awareness with a Wearable Retinal Electric Stimulation Device. In The Augmented Humans International Conference (AHs 2024), April 4–6, 2024, Melbourne, VIC, Australia. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3652920.3652932

Publication

Are you still watching? Assessing internal and external attention in AR using wearable brain sensing glasses

Nataliya Kosmyna and Eugene Hauptmann "Are you still watching? Assessing internal and external attention in AR using wearable brain sensing glasses", Proc. SPIE 12913, Optical Architectures for Displays and Sensing in Augmented, Virtual, and Mixed Reality (AR, VR, MR) V, 1291310 (12 March 2024); https://doi.org/10.1117/12.3023316

Publication

Human-AI Co-Dancing: Evolving Cultural Heritage through Collaborative Choreography with Generative Virtual Characters

Pataranutaporn, P., Mano, P., Bhongse-Tong, P., Chongchadklang, T., Archiwaranguprok, C., Hantrakul, L., Eaimsa-ard, J., Maes, P., & Klunchun, P. (2024). Human-AI co-dancing: Evolving cultural heritage through collaborative choreography with generative virtual characters. In Proceedings of the 9th International Conference on Movement and Computing (Article 14, pp. 1-10). Association for Computing

Publication

Improving Attention Using Wearables via Haptic and Multimodal Rhythmic Stimuli

Nathan Whitmore*, Samantha Chan*, Jingru Zhang, Patrick Chwalek, Sam Chin, and Pattie Maes. 2024. Improving Attention Using Wearables via Haptic and Multimodal Rhythmic Stimuli. In Proceedings of the CHI Conference on Human Factors in Computing Systems (CHI ’24), May 11–16, 2024, Hon- olulu, HI, USA. ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/ 3613904.3642256 (*equal contribution)

Publication

A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration

Kosmyna N, Hauptmann E, Hmaidan Y. A Brain-Controlled Quadruped Robot: A Proof-of-Concept Demonstration. Sensors. 2024; 24(1):80. https://doi.org/10.3390/s24010080

Publication

AI comes out of the closet: Using AI-Generated Virtual Characters to Help Individuals Practice LGBTQIA+ Advocacy

Pillis, D., Pataranutaporn, P., Maes, P., & Sra, M. (2024). AI comes out of the closet: Using AI-Generated Virtual Characters to Help Individuals Practice LGBTQIA+ Advocacy. In Proceedings of the 29th International Conference on Intelligent User Interfaces

Publication

Using Wearable Brain Sensing Glasses during Zero-G Flight for Hyperscanning: Preliminary Study

Nataliya Kosmyna, Daniel Hails, Eugene Hauptmann, Christopher Markus, Zoe Lee, Gun Bolukbasi and Minsol Kim. "Using Wearable Brain Sensing Glasses during Zero-G Flight for Hyperscanning: Preliminary Study," AIAA 2023-4656. ASCEND 2023. October 2023.

Publication

Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social Connections

Leong, Joanne, et al. "Social Wormholes: Exploring Preferences and Opportunities for Distributed and Physically-Grounded Social Connections." Proceedings of the ACM on Human-Computer Interaction 7.CSCW2 (2023): 1-29.

Publication

Influencing human–AI interaction by priming beliefs about AI can increase perceived trustworthiness, empathy and effectiveness

Pataranutaporn, P., Liu, R., Finn, E. et al. Influencing human–AI interaction by priming beliefs about AI can increase perceived trustworthiness, empathy and effectiveness. Nat Mach Intell (2023). https://doi.org/10.1038/s42256-023-00720-7

Publication

Joie: a Joy-based Brain-Computer Interface (BCI)

Vujic, Angela, Shreyas Nisal, and Pattie Maes. "Joie: a Joy-based Brain-Computer Interface (BCI)." Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 2023.

Publication

Fascia Ecosystem: A Step Forward in Sleep Engineering and Research

Bernal, Guillermo, Malika Chhibber, Mradul Bhatnagar, Urvil Jivani, Nelson Hidalgo, Anthony Levasseur, and Pattie Maes. 2023. Fascia Ecosystem: A Step Forward in Sleep Engineering and Research. 45th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’23), no. Sydney Australia, (July )

Publication

Pioneering Remote Sleep Research: A Comparative Study of Sleep Stage Parameters in Diverse Datasets for Enhanced Interventions.

Bernal, Guillermo, Malika Chhibber, Mradul Bhatnagar, Urvil Jivani, Nelson Hidalgo, Anthony Levasseur, and Pattie Maes. Pioneering Remote Sleep Research: A Comparative Study of Sleep Stage Parameters in Diverse Datasets for Enhanced Interventions. SLEEP 2023, the 37th Annual Meeting of the Associated Professional Sleep Societies, LLC (APSS), June 6, 2023.

Publication

InExChange: Fostering Genuine Social Connection through Embodied Breath Sharing in Mixed Reality

Morris, C., Liu, P., Riecke, B. E., & Maes, P. (2023). InExChange: Fostering Genuine Social Connection through Embodied Breath Sharing in Mixed Reality. Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems, Article Article 459. https://doi.org/10.1145/3544549.3583917

Publication

Don’t Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy

Valdemar Danry, Pat Pataranutaporn, Yaoli Mao, and Pattie Maes. 2023. Don’t Just Tell Me, Ask Me: AI Systems that Intelligently Frame Explanations as Questions Improve Human Logical Discernment Accuracy over Causal AI explanations. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI '23). Association for Computing Machinery, New York, NY, USA, Article 352, 1–13.

Publication

Using Generative AI to Cultivate Positive Emotions and Mindsets for Self-Development and Learning

Leong, Joanne. "Using Generative AI to Cultivate Positive Emotions and Mindsets for Self-Development and Learning." XRDS: Crossroads, The ACM Magazine for Students 29.3 (2023): 52-56.

Publication

Olfactory Wearables for Mobile Targeted Memory Reactivation

Amores Fernandez, J., Mehra, N., Rasch, B., & Maes, P. (2023). Olfactory Wearables for Mobile Targeted Memory Reactivation. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery.

Publication

Living memories: Ai-generated characters as digital mementos

Pataranutaporn, P., Danry, V., Blanchard, L., Thakral, L., Ohsugi, N., Maes, P., & Sra, M. (2023, March). Living memories: Ai-generated characters as digital mementos. In Proceedings of the 28th International Conference on Intelligent User Interfaces (pp. 889-901).

Publication

"Picture the Audience...": Exploring Private AR Face Filters for Online Public Speaking

Joanne Leong, Florian Perteneder, Muhender Raj Rajvee, and Pattie Maes. 2023. “Picture the Audience...”: Exploring Private AR Face Filters for Online Public Speaking. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (CHI ’23), April 23–28, 2023, Hamburg, Germany. ACM, New York, NY, USA, 13 pages. https://doi.org/10.1145/3544548.3581039

Publication

Targeted dream incubation at sleep onset increases post-sleep creative performance

Horowitz*, A.H., Esfahany*, K., Gálvez, T.V. et al. Targeted dream incubation at sleep onset increases post-sleep creative performance. Sci Rep 13, 7319 (2023). https://doi.org/10.1038/s41598-023-31361-w

Publication

Modulating Interoceptive Signals for Influencing the Conscious Experience

Publication

Aesthetic chills cause an emotional drift in valence and arousal

Jain A, Schoeller F, Horowitz A, Hu X, Yan G, Salomon R and Maes P (2023) Aesthetic chills cause an emotional drift in valence and arousal. Front. Neurosci. 16:1013117. doi: 10.3389/fnins.2022.1013117

Publication

ChillsDB, A gold standard for aesthetic chills stimuli

Publication

Adaptive Virtual Neuroarchitecture

Jain, A., Maes, P., Sra, M. (2023). Adaptive Virtual Neuroarchitecture. In: Simeone, A., Weyers, B., Bialkova, S., Lindeman, R.W. (eds) Everyday Virtual and Augmented Reality. Human–Computer Interaction Series. Springer, Cham. https://doi.org/10.1007/978-3-031-05804-2_9

Publication

Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text

P. Tandon et al., "Txt2Vid: Ultra-Low Bitrate Compression of Talking-Head Videos via Text," in IEEE Journal on Selected Areas in Communications, 2022, doi: 10.1109/JSAC.2022.3221953.

Publication

Frisson: Leveraging Metasomatic Interactions for Generating Aesthetic Chills

Jain, A., Schoeller, F., Zhang, E., & Maes, P. (2022, November). Frisson: Leveraging Metasomatic Interactions for Generating Aesthetic Chills. In Proceedings of the 2022 International Conference on Multimodal Interaction (pp. 148-158).

Publication

Wemoji: Towards Designing Complementary Communication Systems in Augmented Reality

Joanne Leong, Olivia Seow, Cathy Mengying Fang, Benny J. Tang, Rajan Vaish, and Pattie Maes. 2022. Wemoji: Towards Designing Complementary Communication Systems in Augmented Reality. In The Adjunct Publication of the 35th Annual ACM Symposium on User Interface Software and Technology (UIST ’22 Adjunct), October 29-November 2, 2022, Bend, OR, USA. ACM, New York, NY, USA, 3 pages.

Publication

Decoding Visual Imagery Using EEG/EOG Glasses: A Pilot Study

Kosmyna, N., Balyan, A., Hauptmann, E. (2023). Decoding Visual Imagery Using EEG/EOG Glasses: A Pilot Study. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 2. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 560. Springer, Cham. https://doi.org/10.1007/978-3-031-18458-1_29

Publication

Target Speaker Detection with EEG/EOG Glasses: A Pilot Study

Kosmyna, N., Balyan, A., Hauptmann, E. (2023). Target Speaker Detection with EEG/EOG Glasses: A Pilot Study. In: Arai, K. (eds) Proceedings of the Future Technologies Conference (FTC) 2022, Volume 2. FTC 2022 2022. Lecture Notes in Networks and Systems, vol 560. Springer, Cham. https://doi.org/10.1007/978-3-031-18458-1_30

Publication

AI-Generated Virtual Instructors Based on Liked or Admired People Can Improve Motivation and Foster Positive Emotions for Learning

P. Pataranutaporn* J. Leong*, V. Danry*, A. Lawson, P. Maes, M. Sra. 2022. *Co-first authorship ”AI-Generated Virtual Instructors Based on Liked or Admired People Can Improve Motivation and Foster Positive Emotions for Learning” to appear in FIE 2022, October 8-11, Uppsala, Sweden.

Publication

Improving Memory via Automated Targeted Memory Reactivation during Sleep

Whitmore, N. W., Harris, J. C., Kovach, T. & Paller, K. A. Improving Memory via Automated Targeted Memory Reactivation during Sleep. Journal of Sleep Research 2022.06.28.497977 (2022) doi:10.1101/2022.06.28.497977.

Publication

Interfacing With Dreams: Novel Technologies and Protocols for Targeted Dream Incubation

Haar Horowitz, Adam Jedidiah. Interfacing With Dreams: Novel Technologies and Protocols for Targeted Dream Incubation. Diss. Massachusetts Institute of Technology, 2022.

Publication

EmbER: A System for Transfer of Interoceptive Sensations to Improve Social Perception

Morris, C., Danry, V., & Maes, P. (2022). EmbER: A System for Transfer of Interoceptive Sensations to Improve Social Perception. Designing Interactive Systems Conference, 277–287. https://doi.org/10.1145/3532106.3533550

Publication

Towards Enhancing Empathy Through Emotion Augmented Remote Communication

Rojas, C., Patekar, G., Zuccarelli, E., Chin, A., Esquivel, D., Maes, P. Towards Enhancing Empathy Through Emotion Augmented Remote. 2022. Proc. ACM Conference on Human Factors in Computing Systems - CHI (Accepted)

Publication

SpaceCHI 2.0: Advancing Human-Computer Interaction Systems for Space Exploration

Pat Pataranutaporn, Valentina Sumini, Melodie Yashar, Susanna Testa, Marianna Obrist, Scott Davidoff, Amber M. Paul, Dorit Donoviel, Jimmy Wu, Sands A Fish, Ariel Ekblaw, Albrecht Schmidt, Joe Paradiso, and Pattie Maes. 2022. SpaceCHI 2.0: Advancing Human-Computer Interaction Systems for Space Exploration. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI EA '22).

Publication

AI-Generated Characters: Putting Deepfakes to Good Use

Valdemar Danry, Joanne Leong, Pat Pataranutaporn, Pulkit Tandon, Yimeng Liu, Roy Shilkrot, Parinya Punpongsanon, Tsachy Weissman, Pattie Maes, and Misha Sra. 2022. AI-Generated Characters: Putting Deepfakes to Good Use. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. Association for Computing Machinery, New York, NY, USA, Article 119, 1–5.

Publication

What We Talk About When We Talk About Human-Computer Integration

Nathan Arthur Semertzidis, Zoe Xiao Fang, Pedro Lopes, Kai Kunze, Paul Pangaro, Florian Floyd Mueller, and Pattie Maes. 2022. What We Talk About When We Talk About Human-Computer Integration. In CHI Conference on Human Factors in Computing Systems Extended Abstracts. Association for Computing Machinery, New York, NY, USA, Article 149, 1–4.

Publication

Deceptive AI systems that give explanations are just as convincing as honest AI systems in human-machine decision making

Danry, Valdemar, Pat Pataranutaporn, Ziv Epstein, Matthew Groh, and Pattie Maes (Under review). “Deceptive AI systems that give explanations are just as convincing as honest AI systems in human-machine decision making.” Extended Abstract. Presented at the International Conference on Computational Social Science (IC2S2) 2022.

Publication

On Eliciting a Sense of Self when Integrating with Computers

Danry, Valdemar, et al. "On Eliciting a Sense of Self when Integrating with Computers." Augmented Humans 2022. 2022. 68-81.

Publication

Galea: A physiological sensing system for behavioral research in Virtual Environments

Bernal, G., Hidalgo, N., Russomanno, C., & Maes, P. (2022, March). Galea: A physiological sensing system for behavioral research in Virtual Environments. In 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR) (pp. 66-76). IEEE.

Publication

Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence

Singh, N., Bernal, G., Savchenko, D., & Glassman, E. L. (2022). Where to Hide a Stolen Elephant: Leaps in Creative Writing with Multimodal Machine Intelligence. ACM Transactions on Computer-Human Interaction.

Publication

Situated VR: Toward a Congruent Hybrid Reality Without Experiential Artifacts

Sra, Misha, Valdemar Danry, and Pattie Maes. "Situated VR: Toward a Congruent Hybrid Reality Without Experiential Artifacts." IEEE Computer Graphics and Applications 42.3 (2022): 7-18.

Publication

AI-generated characters for supporting personalized learning and well-being

Pataranutaporn, P., Danry, V., Leong, J. et al. AI-generated characters for supporting personalized learning and well-being. Nat Mach Intell (2021). https://doi.org/10.1038/s42256-021-00417-9

Publication

KinVoices: Using Voices of Friends and Family in Voice Interfaces

Chan, S. W. T., Gunasekaran, T. S., Pai, Y. S., Zhang, H., & Nanayakkara, S. (2021). KinVoices: Using Voices of Friends and Family in Voice Interfaces. Proceedings of the ACM on Human-Computer Interaction, 5(CSCW2), 1-25.

Publication

A Pilot Study using Covert Visuospatial Attention as an EEG-based Brain Computer Interface to Enhance AR Interaction

Nataliya Kosmyna, Chi-Yun Hu, Yujie Wang, Qiuxuan Wu, Cassandra Scheirer, and Pattie Maes. 2021. A Pilot Study using Covert Visuospatial Attention as an EEG-based Brain Computer Interface to Enhance AR Interaction. In 2021 International Symposium on Wearable Computers (ISWC '21). Association for Computing Machinery, New York, NY, USA, 43–47. DOI:https://doi.org/10.1145/3460421.3480420

Publication

Assessing Internal and External Attention in AR using Brain Computer Interfaces: A Pilot Study

N. Kosmyna, Q. Wu, C. -Y. Hu, Y. Wang, C. Scheirer and P. Maes, "Assessing Internal and External Attention in AR using Brain Computer Interfaces: A Pilot Study," 2021 IEEE 17th International Conference on Wearable and Implantable Body Sensor Networks (BSN), 2021, pp. 1-6, doi: 10.1109/BSN51625.2021.9507034.

Publication

Transfer Learning with Real-World Nonverbal Vocalizations from Minimally Speaking Individuals

Narain, J., Johnson, K., Quatieri, T., Picard, R., and Maes, P. “Transfer Learning with Real-World Vocalizations from Minimally Speaking Individuals”. Workshop in Interpretable ML in Healthcare at International Conference on Machine Learning. July 2021.

Publication

Machinoia, Machine of Multiple Me: Integrating with Past, Future, and Alternative Selves

Pataranutaporn, Pat, Valdemar Danry, and Pattie Maes. "Machinoia, Machine of Multiple Me: Integrating with Past, Future and Alternative Selves." Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.

Publication

Exploring the Use of Real-Time Camera Filters on Embodiment and Creativity

Leong, Joanne, et al. "Exploring the Use of Real-Time Camera Filters on Embodiment and Creativity." Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. 2021.

Publication

“The Thinking Cap”: Fostering Growth Mindset Of Children By Means Of Electroencephalography And Perceived Magic of Harry Potter Universe

Nataliya Kosmyna, Cassandra Scheirer, and Pattie Maes. 2021. The Thinking Cap: Fostering Growth Mindset Of Children By Means Of Electroencephalography And Perceived Magic of Harry Potter Universe. In CHI Conference on Human Factors in Computing Systems Extended Abstracts (CHI ’21 Extended Abstracts), May 08–13, 2021, Yokohama, Japan. ACM, New York, NY, USA, 6 pages. https://doi.org/10.1145/34

Publication

Brain-Computer Interfaces, Open Source and Democratizing the Future of Augmented Consciousness

Bernal, Guillermo, Sean Montgomery, and Pattie Maes. "Brain-Computer Interfaces, Open Source and Democratizing the Future of Augmented Consciousness." Frontiers in Computer Science 3 (2021): 23.

Publication

Increasing Embodiment for Desktop-Based Participants in Mixed Desktop and Immersive Collaborative Virtual Environments

Simonson, Aubrey and Pattie Maes. “Increasing Embodiment for Desktop-Based Participants in Mixed Desktop and Immersive Collaborative Virtual Environments.” (May 7, 2020).

Publication

A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens

Camilo Rojas, Niels Poulsen, Mileva Van Tuyl, Daniel Vargas, Zipporah Cohen, Joe Paradiso, Pattie Maes, Kevin Esvelt, and Fadel Adib. 2021. A Scalable Solution for Signaling Face Touches to Reduce the Spread of Surface-based Pathogens.. Proc. ACM IMWUT

Publication

Investigating the Use of Synthetic Media and Real-Time Virtual Camera Filters for Supporting Communication and Creativity

Leong, Joanne. Investigating the Use of Synthetic Media and Real-Time Virtual Camera Filters for Supporting Communication and Creativity. 2021. Massachusetts Institute of Technology, MS Thesis.

Publication

Augmenting aesthetic chills using a wearable prosthesis improves their downstream effects on reward and social cognition

Haar, A.J.H., Jain, A., Schoeller, F. et al. Augmenting aesthetic chills using a wearable prosthesis improves their downstream effects on reward and social cognition. Sci Rep 10, 21603 (2020). https://doi.org/10.1038/s41598-020-77951-w

Publication

Contrastive Representation Learning for Electroencephalogram Classification

Mohsenvand, M.N., Izadi, M.R. & Maes, P.. (2020). Contrastive Representation Learning for Electroencephalogram Classification. Proceedings of the Machine Learning for Health NeurIPS Workshop, in Proceedings of Machine Learning Research 136:238-253 Available from http://proceedings.mlr.press/v136/mohsenvand20a.html.

Publication

Going with our Guts: Potentials of Wearable Electrogastrography (EGG) for Affect Detection

Vujic, A., Tong, S., Picard, R., & Maes, P. (2020). Going with our Guts: Potentials of Wearable Electrogastrography (EGG) for Affect Detection. Proceedings of the 2020 International Conference on Multimodal Interaction.

Publication

Personalized Modeling of Real-World Vocalizations from Nonverbal Individuals

Narain, J.*, Johnson, K.T.*, Ferguson, C., O’Brien, A., Talkar, T., Zhang, Y., Wofford, P., Quatieri, T., Picard, R.W.,Maes, P., "Personalized Modeling of Real-World Vocalizations from Nonverbal Individuals," Proceedings of the International Conference on Multimodal Interaction (ICMI), Utrecht, Netherlands, October 2020. (*Co-first authors/Equal contribution)