Affiliated Faculty, Faculty
My primary research interests are in robotics and natural language processing; in my work, I combine these interests to support research in human-robot interaction, or HRI. My work focuses on the problem of grounded language acquisition: extracting semantically meaningful representations of human language by mapping those representations to the noisy, unpredictable physical world in which robots operate. More specifically, I work on combining probabilistic, grammar-based natural language processing with machine learning to transform human communication into a formal language that a robot can understand. I have looked at using this kind of language learning to learn how to follow navigation instructions or learn more about the world from human users by learning to extend a world model in tandem with learning a language parsing model.
- G. Y. Kebe, L. E. Richards, E. Raff, F. Ferraro, and C. Matuszek, "Bridging the Gap: Using Deep Acoustic Representations to Learn Grounded Language from Percepts and Raw Speech", InProceedings, Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence (AAAI-22), June 2022, 26 downloads.
- N. Pillai, C. Matuszek, and F. Ferraro, "Neural Variational Learning for Grounded Language Acquisition", InProceedings, IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), August 2021, 129 downloads.
- N. Pillai, C. Matuszek, and F. Ferraro, "Measuring Perceptual and Linguistic Complexity in Multilingual Grounded Language Data", InProceedings, 34th International FLAIRS Conference (FLAIRS-34), May 2021, 301 downloads.
- M. Murnane, P. Higgins, M. Saraf, F. Ferraro, C. Matuszek, and D. Engel, "A Simulator for Human-Robot Interaction in Virtual Reality", InProceedings, Conference on Virtual Reality and 3D User Interfaces, Abstracts and Workshops (VRW), March 2021, 397 downloads.
- P. Higgins, G. Y. Kebe, K. Darvish, D. Engel, F. Ferraro, and C. Matuszek, "Towards Making Virtual Human-Robot Interaction a Reality", InProceedings, 3rd International Workshop on Virtual, Augmented, and Mixed-Reality for Human-Robot Interactions (VAM-HRI), March 2021, 495 downloads.
- N. Pillai, E. Raff, F. Ferraro, and C. Matuszek, "Sampling Approach Matters: Active Learning for Robotic Language Acquisition", InProceedings, IEEE BigData BDML, December 2020, 530 downloads.
- A. T. Nguyen, L. E. Richards, G. Y. Kebe, E. Raff, K. Darvish, F. Ferraro, and C. Matuszek, "Practical Cross-modal Manifold Alignment for Grounded Language", Article, arXiv:2009.05147 [cs.CV], September 2020, 203 downloads.
- P. Jenkins, R. Sachdeva, G. Y. Kebe, P. Higgins, K. Darvish, E. Raff, D. Engel, J. Winder, F. Ferraro, and C. Matuszek, "Presentation and Analysis of a Multimodal Dataset for Grounded Language Learning", Article, arXiv:2007.14987 [cs.RO], July 2020, 210 downloads.
- C. Kery, N. Pillai, C. Matuszek, and F. Ferraro, "Building Language-Agnostic Grounded Language Learning Systems", InProceedings, 28th International Conference on Robot and Human Interactive Communication (Ro-Man), October 2019, 321 downloads.
- M. Murnane, M. Breitmeyer, F. Ferraro, C. Matuszek, and D. Engel, "Learning from Human-Robot Interactions in Modeled Scenes", InProceedings, ACM SIGGRAPH 2019 Posters, July 2019, 218 downloads.
- M. Murnane, M. Breitmeyer, F. Ferraro, and C. Matuszek, "Learning from Human-Robot Interactions in Modeled Scenes", InProceedings, ACM SIGGRAPH 2019 Posters, July 2019, 221 downloads.
- C. Kery, F. Ferraro, and C. Matuszek, "¿Es un plátano? Exploring the Application of a Physically Grounded Language Acquisition System to Spanish", NAACL Combined Workshop on Spatial Language Understanding and Grounded Communication for Robotics, June 2019, 346 downloads.
- N. Pillai, F. Ferraro, and C. Matuszek, "Deep Learning for Category-Free Grounded Language Acquisition", NAACL Workshop on Spatial Language Understanding and Grounded Communication for Robotics, June 2019, 265 downloads.
- N. Pillai, F. Ferraro, and C. Matuszek, "Optimal Semantic Distance for Negative Example Selection in Grounded Language Acquisition", Workshop on Models and Representations for Natural Human-Robot Communication (Robotics: Science and Systems), June 2018, 330 downloads.
- P. K. Das, A. L. Kashyap, G. Singh, C. Matuszek, T. Finin, and A. Joshi, "Semantic knowledge and privacy in the physical web", InProceedings, Proceedings of the 4th Workshop on Society, Privacy and the Semantic Web - Policy and Technology (PrivOn2016) co-located with 15th International Semantic Web Conference (ISWC 2016), October 2016, 2031 downloads.