IEEE International Conference on Robot & Human Interactive Communication (RO-MAN)

Neural Variational Learning for Grounded Language Acquisition

, , and

We propose a learning system in which language is grounded in visual percepts without specific pre-defined categories of terms. We present a unified generative method to acquire a shared semantic/visual embedding that enables the learning of language about a wide range of real-world objects. We evaluate the efficacy of this learning by predicting the semantics of objects and comparing the performance with neural and non-neural inputs. We show that this generative approach exhibits promising results in language grounding without pre-specifying visual categories under low resource settings. Our experiments demonstrate that this approach is generalizable to multilingual, highly varied datasets.


  • 1524578 bytes

robot nlp learning

InProceedings

IEEE

Downloads: 67 downloads

UMBC ebiquity