Teaching AI to Understand and Use Images in Dialogue
Researchers from South Korea have developed a dataset designed to aid research into AI’s understanding of the way that humans use images in dialogue, and to help natural language models to participate in this very recent development in human communications.
The paper, from KAIST at Daedeok Innopolis, notes that research into such multi-modal dialogue systems over the last ten years has been hamstrung by datasets and methodologies centering on disciplines that are peripheral to the topic, such as visual question answering and image captioning.
In these older approaches, images are evaluated out of the lexical context of a conversation, with no understanding of the way that the dialogue is enhanced and developed by image responses, and no cross-domain schema for decoding the contributions of visual contributions to discourse.
Many of the aforementioned approaches to date have been initiatives or developments from Microsoft’s AI research arm, which in 2017 also examined the topic of multimodal conversations that are begun by an image, rather than freely using images as dialogue components.
