From visual perception to multimodal communication: Incremental route descriptions
Journal
Artificial Intelligence Review Journal
ISSN
0269-2821
Type
journal article
Date Issued
1994
Author(s)
Abstract
In the last few years in cognitive science there has been a growing interest in the connection between visual perception and natual language. The question of interest is: How ca we discuss what we see? With this question in mind in this article we will look at the area of incremental route descriptions. Here, a speaker step-by-step presents the relevant rout information in a 3D-environment. The speaker must adjust his/her descriptions to the currently visible objects. Two major questions arise in this context: 1. How is visually obtained informaiton used in natural language generation? and 2. How are the modalities coordniated? We will present a computational framework for the interaction of visual perception and natural language descriptions which integrates several processes and representations. Especially discussed is the interaction between the spatial representation and the presentation representation used for natural language descriptions. We have implemented a prototypical version of the proposed model, called MOSES.
Funding(s)
Language
English
HSG Classification
contribution to scientific community
Refereed
Yes
Publisher
Springer
Publisher place
Dordrecht
Volume
8
Number
5/6
Start page
68
End page
82
Pages
15
Subject(s)
Division(s)
Eprints ID
12840
File(s)![Thumbnail Image]()
Name
b108.pdf
Size
303.82 KB
Format
Adobe PDF
Checksum (MD5)
7dfe0e3a2a8626a4a7532754bf954c2a