For best experience please turn on javascript and use a modern browser!
You are using a browser that is no longer supported by Microsoft. Please upgrade your browser. The site may not present itself correctly if you continue browsing.

Dr. J.P. (James) Trujillo

Faculty of Science
ILLC
Visiting address
  • Science Park 107
Postal address
  • Postbus 94242
    1090 GE Amsterdam
Contact details
  • Research Interests

    Overall, my research aims to understand how we use the ensemble of visual and vocal modalities available to us in order to communicate. How do we orchestrate, face, head, body, hands, and speech together into one multimodal whole, and how do others make sense of this? How do we adapt this complex array of behaviors to different contexts? How is neurodiversity reflected in different styles of communication, and how does this impact social interaction?

    To this end, I use motion tracking, acoustic analysis, qualitative coding, and virtual agents to study both what people do in naturalistic settings, and to design experiments to test the hypotheses generated from studying more unconstrained behavior.

  • Teaching and Supervision

    I currently teach Language, Speech and Dialogue Processing in the Artificial Intelligence bachelor programme, and regularly co-organize (largely methods-oriented) workshops on multimodal communication, such as the Practical Approaches to Human Multimodal Behavior summer school in Nijmegen, and the Behavioral Dynamics in Social Interaction workshop in Krakow.  

    I currently supervise PhD candidate Anna Palmann, as well as Master and Bachelor students on projects relating to cognition, language, and AI.

     

  • Publications

    2023

    • Nota, N., Trujillo, J. P., & Holler, J. (2023). Specific facial signals associate with categories of social actions conveyed through questions. PLoS ONE, 18(7 July), Article e0288104. https://doi.org/10.1371/journal.pone.0288104
    • Trujillo, J. P., & Holler, J. (2023). Interactionally Embedded Gestalt Principles of Multimodal Human Communication. Perspectives on Psychological Science, 18(5), 1136-1159. https://doi.org/10.1177/17456916221141422
    • Trujillo, J. P., Dideriksen, C., Tylén, K., Christiansen, M. H., & Fusaroli, R. (2023). The Dynamic Interplay of Kinetic and Linguistic Coordination in Danish and Norwegian Conversation. Cognitive Science, 47(6), Article e13298. https://doi.org/10.1111/cogs.13298

    2022

    • Eijk, L., Rasenberg, M., Arnese, F., Blokpoel, M., Dingemanse, M., Doeller, C. F., Ernestus, M., Holler, J., Milivojevic, B., Özyürek, A., Pouw, W., van Rooij, I., Schriefers, H., Toni, I., Trujillo, J., & Bögels, S. (2022). The CABB dataset: A multimodal corpus of communicative interactions for behavioural and neural analyses. NeuroImage, 264, Article 119734. https://doi.org/10.1016/j.neuroimage.2022.119734
    • Owoyele, B., Trujillo, J., de Melo, G., & Pouw, W. (2022). Masked-Piper: Masking personal identities in visual recordings while preserving multimodal information. SoftwareX, 20, Article 101236. https://doi.org/10.1016/j.softx.2022.101236
    • Trujillo, J. P., Levinson, S. C., & Holler, J. (2022). A multi-scale investigation of the human communication system's response to visual disruption. Royal Society Open Science, 9(4), Article 211489. https://doi.org/10.1098/rsos.211489
    • Trujillo, J. P., Özyürek, A., Kan, C. C., Sheftel-Simanova, I., & Bekkering, H. (2022). Differences in functional brain organization during gesture recognition between autistic and neurotypical individuals. Social Cognitive and Affective Neuroscience, 17(11), 1021-1034. https://doi.org/10.1093/scan/nsac026
    • van der Meer, H. A., Sheftel-Simanova, I., Kan, C. C., & Trujillo, J. P. (2022). Translation, Cross-Cultural Adaptation, and Validation of a Dutch Version of the Actions and Feelings Questionnaire in Autistic and Neurotypical Adults. Journal of Autism and Developmental Disorders, 52(4), 1771-1777. https://doi.org/10.1007/s10803-021-05082-w

    2021

    • Nota, N., Trujillo, J. P., & Holler, J. (2021). Facial signals and social actions in multimodal face-to-face interaction. Brain Sciences, 11(8), Article 1017. https://doi.org/10.3390/brainsci11081017
    • Trujillo, J. P., & Holler, J. (2021). The kinematics of social action: Visual signals provide cues for what interlocutors do in conversation. Brain Sciences, 11(8), Article 996. https://doi.org/10.3390/brainsci11080996
    • Trujillo, J. P., Levinson, S. C., & Holler, J. (2021). Visual Information in Computer-Mediated Interaction Matters: Investigating the Association Between the Availability of Gesture and Turn Transition Timing in Conversation. In M. Kurosu (Ed.), Human-Computer Interaction. Design and User Experience Case Studies - Thematic Area, HCI 2021, Held as Part of the 23rd HCI International Conference, HCII 2021, Proceedings (pp. 643-657). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12764 LNCS). Springer Science and Business Media Deutschland GmbH. https://doi.org/10.1007/978-3-030-78468-3_44
    • Trujillo, J. P., Özyürek, A., Kan, C. C., Sheftel-Simanova, I., & Bekkering, H. (2021). Differences in the production and perception of communicative kinematics in autism. Autism Research, 14(12), 2640-2653. https://doi.org/10.1002/aur.2611
    • Trujillo, J., Özyürek, A., Holler, J., & Drijvers, L. (2021). Speakers exhibit a multimodal Lombard effect in noise. Scientific Reports, 11(1), Article 16721. https://doi.org/10.1038/s41598-021-95791-0

    2020

    • Pouw, W., Trujillo, J. P., & Dixon, J. A. (2020). The quantification of gesture–speech synchrony: A tutorial and validation of multimodal data acquisition using device-based and video-based motion tracking. Behavior Research Methods, 52(2), 723-740. https://doi.org/10.3758/s13428-019-01271-9
    • Trujillo, J. P., Simanova, I., Bekkering, H., & Özyürek, A. (2020). The communicative advantage: how kinematic signaling supports semantic comprehension. Psychological Research, 84(7), 1897-1911. https://doi.org/10.1007/s00426-019-01198-y
    • Trujillo, J. P., Simanova, I., Özyürek, A., & Bekkering, H. (2020). Seeing the Unexpected: How Brains Read Communicative Intent through Kinematics. Cerebral Cortex, 30(3), 1056-1067. https://doi.org/10.1093/cercor/bhz148

    2019

    • Trujillo, J. P., Vaitonyte, J., Simanova, I., & Özyürek, A. (2019). Toward the markerless and automatic analysis of kinematic features: A toolkit for gesture and movement research. Behavior Research Methods, 51(2), 769-777. https://doi.org/10.3758/s13428-018-1086-8

    2018

    • Trujillo, J. P., Simanova, I., Bekkering, H., & Özyürek, A. (2018). Communicative intent modulates production and comprehension of actions and gestures: A Kinect study. Cognition, 180, 38-51. https://doi.org/10.1016/j.cognition.2018.04.003

    2018

    • Drijvers, L., & Trujillo, J. P. (2018). Commentary: Transcranial magnetic stimulation over left inferior frontal and posterior temporal cortex disrupts gesture-speech integration. Frontiers in Human Neuroscience, 12, Article 256. https://doi.org/10.3389/fnhum.2018.00256
    This list of publications is extracted from the UvA-Current Research Information System. Questions? Ask the library or the Pure staff of your faculty / institute. Log in to Pure to edit your publications. Log in to Personal Page Publication Selection tool to manage the visibility of your publications on this list.
  • Ancillary activities
    No ancillary activities
edit contact information   edit profile page