• Login
  • Register

Work for a Member company and need a Member Portal account? Register here with your company email address.

Project

Your Brain on ChatGPT

Copyright

Nataliya Kosmyna

 Nataliya Kosmyna

Check project's website: https://www.brainonllm.com

With today's wide adoption of LLM products like ChatGPT from OpenAI, humans and businesses engage and use LLMs on a daily basis. Like any other tool, it carries its own set of advantages and limitations. This study focuses on finding out the cognitive cost of using an LLM in the educational context of writing an essay.

We assigned participants to three groups: LLM group, Search Engine group, and Brain-only group, where each participant used a designated tool (or no tool in the latter) to write an essay. We conducted 3 sessions with the same group assignment for each participant. In the 4th session we asked LLM group participants to use no tools (we refer to them as LLM-to-Brain), and the Brain-only group participants were asked to use LLM (Brain-to-LLM). We recruited a total of 54 participants for Sessions 1, 2, 3, and 18 participants among them completed session 4.

We used electroencephalography (EEG) to record participants' brain activity in order to assess their cognitive engagement and cognitive load, and to gain a deeper understanding of neural activations during the essay writing task. We performed NLP analysis, and we interviewed each participant after each session. We performed scoring with the help from the human teachers and an AI judge (a specially built AI agent).

We discovered a consistent homogeneity across the Named Entities Recognition (NERs), n-grams, ontology of topics within each group. EEG analysis presented robust evidence that LLM, Search Engine and Brain-only groups had significantly different neural connectivity patterns, reflecting divergent cognitive strategies. Brain connectivity systematically scaled down with the amount of external support: the Brain‑only group exhibited the strongest, widest‑ranging networks, Search Engine group showed intermediate engagement, and LLM assistance elicited the weakest overall coupling. In session 4, LLM-to-Brain participants showed weaker neural connectivity and under-engagement of alpha and beta networks; and the Brain-to-LLM participants demonstrated higher memory recall, and re‑engagement of widespread occipito-parietal and prefrontal nodes, likely supporting the visual processing, similar to the one frequently perceived in the Search Engine group. The reported ownership of LLM group's essays in the interviews was low. The Search Engine group had strong ownership, but lesser than the Brain-only group. The LLM group also fell behind in their ability to quote from the essays they wrote just minutes prior.

As the educational impact of LLM use only begins to settle with the general population, in this study we demonstrate the pressing matter of exploring a possible decrease in learning skills based on the preliminary results of our study. The use of LLM had a measurable impact on our participants, and while the benefits were initially apparent, as we demonstrated over the course of 4 sessions, which took place over 4 months, the LLM group's participants performed worse than their counterparts in the Brain-only group at all levels: neural, linguistic, scoring.

We hope this study serves as a preliminary guide to encourage better understanding of the cognitive and practical impacts of AI on learning environments.

Note, that as of June 2025, when the first paper related to the project, was uploaded to Arxiv, the preprint service, it has not yet been peer-reviewed, thus all the conclusions are to be treated with caution and as preliminary.

Additionally, there are several limitations and important avenues for future work, which will need to be addressed in the next or similar studies:

1. In this study we had a limited number of participants recruited from a specific geographical area, several large academic institutions, located very close to each other. For future work it will be important to include a larger number of participants coming with diverse backgrounds like professionals in different areas, age groups, as well as ensuring that the study is more gender balanced.

2. This study was performed using ChatGPT, and though we do not believe that as of the time of this paper publication in June 2025, there are any significant breakthroughs in any of the commercially available models to grant a significantly different result, we cannot directly generalize the obtained results to other LLM models. Thus, for future work it will be important to include several LLMs and/or offer users a choice to use their preferred one, if any.

3.  Future work may also include the use of LLMs with other modalities beyond the text, like audio modality.

4.  We did not divide our essay writing task into subtasks like idea generation, writing, and so on, which is often done in prior work. This labeling can be useful to understand what happens at each stage of essay writing and have more in-depth analysis.

5. In our current EEG analysis we focused on reporting connectivity patterns without examining spectral power changes, which could provide additional insights into neural efficiency. EEG's spatial resolution limits precise localization of deep cortical or subcortical contributors (e.g. hippocampus), thus fMRI use is the next step for our future work.

6. Our findings are context-dependent and are focused on writing an essay in an educational setting and may not generalize across tasks.

7. Future studies should also consider exploring longitudinal impacts of tool usage on memory retention, creativity, and writing fluency.

#cognitivedebt #brainonllm #yourbrainonchatgpt 

Copyright

Nataliya Kosmyna

FAQ

  1. Is it safe to say that LLMs are, in essence, making us "dumber"?
  2. Do you have a peer review timeline for this project?
  3. Are you planning any additional studies in the near future?
  4. Anything else to add that you feel is relevant to a story about this project?
  5. Additional vocabulary to avoid using when talking about the paper
  6. Was this study funded?
  1. Is it safe to say that LLMs are, in essence, making us "dumber"?

    No! Please do not use the words like “stupid”, “dumb”, “brain rot”, "harm", "damage", "brain damage", "passivity", "trimming" , "collapse" and so on. It does a huge disservice to this work, as we did not use this vocabulary in the paper, especially if you are a journalist reporting on it.

  2. Do you have a peer review timeline for this project?

    Currently this paper is a preprint. We decided to release this preprint now to collect a wider feedback faster as the topic is pressing and the speed of development and integration of LLMs in everyday lives is unmatched and not something we have truly seen before.

    Peer review process has been already started, but we are in the very beginning of this process, and it will probably take months. As you might know/have heard/read – peer reviews take usually a long period of time, anytime between 4 months and up two years. 

  3. Are you planning any additional studies in the near future?

    Yes,  the next one is about the "vibe coding". We have already collected the data and are currently working on the analysis and draft. It adds to why it is important to get general public's feedback now.

  4. Anything else to add that you feel is relevant to a story about this project?

    Lots of media and people used LLMs to summarize the paper. It adds to the noise. Your HUMAN feedback is very welcome, if you read the paper or parts of it.

    Also, as a reminder, the study has a list of limitations we list very clearly both in the paper and on the webpage. And to the best of our knowledge, it is one of the first protocols, thus we do expect more papers/studies (from ourselves and other researchers!) with different protocols, populations, tasks, methodologies, that will add to the general understanding of the use of this technology in different aspects of our lives.

  5. Additional vocabulary to avoid using when talking about the paper

    In addition to the vocabulary from Question 1 in this FAQ - please avoid using "brain scans",  "LLMs make you stop thinking", "impact negatively", "brain damage", "terrifying findings". 

  6. Was this study funded?

     No, no funding to report for this study or the next one. We will add this line to the final paper draft!