Recognizing and interpreting group behaviors is much more challenging than that of individual behaviors. Firstly, the system must perform well in recognizing individual behavioral cues. Secondly, it must do so simultaneously, while keeping track of every individual in the group. Finally, it must also recognize the subtle interactions that take place between group members as it can provide more insights into what is being communicated. Natural human conversations are interactively contingent, where people act and react in a coordinated fashion in turns. Consequently, understanding group behavior in multiparty conversations requires recognizing contingent behaviors between group members.
To address these challenges, we propose a new model , the Multiparty-Transformer (Multipar-T), which is able to handle multiple streams of input data for all of the members of the group.
Work has been accepted at IJCAI 2023, and can be found here: https://arxiv.org/pdf/2304.12204.pdf