Situated Task-Driven Multimodal Intent Modeling and Applications

Adriana Kovashka, Diane Litman & Malihe Alikhani
How should AI systems support multimodal task-driven interactions? What cues do they use to gauge the role of each communicative modality, i.e. what objective each verbal utterance, naming, pointing gesture, or visual demonstration plays in accomplishing goals? How does the meaning of each action change with the actor’s/speaker’s goal? How do actions vary depending on physical context? We will first capture the purpose of multimodal interactions by asking humans to collaborate to complete a set...
This data repository is not currently reporting usage information. For information on how your repository can submit usage information, please see our documentation.