NAACL Workshop on Spatial Language Understanding and Grounded Communication for Robotics

Deep Learning for Category-Free Grounded Language Acquisition

, , and

We propose a learning system in which language is grounded in visual percepts without pre-defined category constraints. We present a unified generative method to acquire a shared semantic/visual embedding that enables a more general language grounding acquisition system. We evaluate the efficacy of this learning by predicting the semantics of ground truth objects and comparing the performance with each of a predefined category classifier and a simple logistic regression classifier. Our preliminary results suggest that this generative approach exhibits promising results in language grounding without pre-specifying visual categories such as color and shape.


  • 2038073 bytes

natural language processing, robotics

Downloads: 14 downloads

UMBC ebiquity