MRTranslate: Bridging Language Barriers in the Physical World Using a Mixed Reality Point-and-Translate System
Type
conference paper
Date Issued
2024-06
Author(s)
Research Team
Human-computer Interaction (HCI)
Abstract
Language barriers pose significant challenges in our increasingly globalized world, hindering effective communication and access to information. Existing translation tools often disrupt the current activity flow and fail to provide seamless user experiences. In this paper, we contribute the design, implementation, and evaluation of \MRTranslate, an assistive Mixed Reality (MR) prototype that enables seamless translations of real-world text. We instructed 12 participants to translate items on a food menu using MRTranslate, which we compared to state-of-the-art translation apps, including Google Translate and Google Lens. Findings from our user study reveal that when utilising a fully functional implementation of MRTranslate, participants achieve success in up to 91.67% of their translations whilst also enjoying the visual translation of the unfamiliar text. Although the current translation apps were well perceived, participants particularly appreciated the convenience of not having to grab a smartphone and manually input the text for translation when using MRTranslate. We believe that MRTranslate, along with the empirical insights we have gained, presents a valuable step towards a future where MR transforms language translation and holds the potential to assist individuals in various day-to-day experiences.
Language
English
Keywords
Mixed Reality
Assistive MR Artefacts
Empirical Research
HSG Classification
contribution to practical use / society
Book title
International Conference on Advanced Visual Interfaces
Publisher
ACM
Event Title
AVI 2024
Event Location
International Conference on Advanced Visual Interfaces (AVI 2024)
Official URL
Subject(s)
Contact Email Address
florian.mathis@unisg.ch
File(s)![Thumbnail Image]()
open.access
Name
Manuscript_AVI2024_MathisFlorian.pdf
Type
Main Article
Size
3.1 MB
Format
Adobe PDF
Checksum (MD5)
c29a60e786a9208661e0006fdc53d3b7