Mobile Guides


The project will investigate theoretical and technical aspects in the design of full-scale mobile guides in different application domains. The project will propose an integrated framework for the development of context-aware mobile guides, characterized by:
  1. a standard and shared format for the representation of data allowing interoperability between different devices and applications
  2. ability to derive information on the state and behaviour of the user from common sensors data (e.g., GPS data)
  3. automatic generation of natural language presentations
  4. device-dependent generation and adaptation of the user interface
  5. usability-centered input/output modalities based on visual interfaces and multimedia and multimodal techniques
  6. multi-user features (mobile communication, community)
Three applications for three different domains (tourism, culture and sport) will be developed to prove the effectiveness and generality of the proposed solution.

Short presentation


Università di Udine, Luca Chittaro (Coordinatore)

Università di Torino, Ilaria Torre

Università di Pisa, Maria Simi

Goals of our research unit

The research activity of the Pisa unit will focus on two essential aspects of the integrated framework for context-aware mobile guides that will be jointly designed by the three research units: the first one concerns the support of user cooperation that will be deployed in collaborative learning solutions during group visits; the second one concerns the implementation of a module for the generation of adaptive natural language comments.
The solutions developed will be experimentally evaluated with proper user studies. In addition, the application domain of museum guides will be considered as a benchmark for experimentation of the proposed solutions and a prototype guide will be developed for a specific museum (the Marble Museum of Carrara).

This research will be performed in collaboration by researchers of the University of Pisa and researchers of the group of human-computer interaction of ISTI-CNR (main investigator Fabio Paternò).
The university researchers have competences in the fields of knowledge representation, implementation of Web applications and services, search engines, natural language analysis and generation. Of special relevance for this project is a research activity performed in collaboration with the Accademia Navale di Livorno (Esuli, Cisternino, Pacini, Simi, 2002), where a system was developed for teaching about sail racing and regattas rules. The system features a component for the analysis of dynamic scenarios (an expert system in sail rules and strategies), a tridimensional visualization of the evolving scenarios and automatic generation of comments in real-time. The system is able to adjust the audio comments to the level of competence of the student and to the 3D visualization times.

The ISTI researchers bring to the project complementary competences in the fields of human-conputer interaction and usability evaluation of man-machine interfaces.
In the field of investigation of the project, they have experiences in the automatic generation of adaptive comments for virtual guides in Web based museums (Marucci, Paternò, 2002). In this previous work, comments are generated dynamically according to the user profile and to the information accesses effectively performed during the navigation. These comments are created by enriching standard descriptions with additional information of various types (introductive, comparison, difference, summary, curiosity) according to a style which is typical of human guides. In another project they developed a supporting software for a handheld device (Ciavarella, Paternò, 2004) able to provide multimedial information and services which depend on the room where the user is located. The result is a palm-top guide, now available to the visitors of the Marble Museum in Carrara, which is able to detect in which room the visitor is located by means of infrared devices. This guide now offers predefined audio comments, as well as text, graphics, images and videos.

The proposal is to extend this previous experience along two directions:
- Dynamic adaptive comment generation, taking into account users movements and their accesses to information; along the lines of real time comments for regattas scenarios, a new dimension could be the visit style and the time available which imposes constraints on the length of the audio comment.
- Support for multi-user cooperative games, in order to make the learning experience more effective and enjoyable, by introducing the possibility of group cooperation.

Thanks to the existing cooperation with the Marble Museum in Carrara, it will be possible to experiment the results of the project within the museum, and to verify their effectiveness with real visitors.


Esuli A., A. Cisternino, G. Pacini, M. Simi, Adaptive Real Time Comment Generation for Sail Racing Scenarios, Adaptive Hypermedia and Adaptive Web-Based Systems, Second International Conference, AH 2002, vol. LNCS 2347, pp. 392-396, Malaga, Spain 2002.

Ciavarella C., F.Paternò, The design of a handheld, location-aware guide for indoor environments, Personal and Ubiquitous Computing, Vol.8 N.2, pp.82-81, Sprinter Verlag, May 2004.

Marucci L., F. Paternò, “Design and evaluation of an adaptive virtual guide for Web applications”, Universal Access in the Information Society, Springer Verlag, Vol.1, N.3, June 2002, pp.163-175.

First year

The unit of Pisa will participate with the other units to the definition of the requirements for the shared architecture of mobile guides, and will implement in particular the modules dedicated to multi-user functionalities and for community/group support.

During the first year of the project, the Pisa unit will study the part of the software infrastructure concerned with multi-user management for supporting collaborative group visits.
Pisa will study in particular the domain of museum/cultural guides and the requirements for a mobile guide in such a context. In order to guarantee the generality of the solutions with respect to different kinds of applications, they will acquire requisites from the other research units of Torino and Udine, which investigate tourism and sport related applications respectively.
In order to support cooperative learning by means of interactions between visitors, it will be necessary to study the software architecture and the infrastructure for communication and interaction among participants, devise the best techniques and state-of-the-art technologies for localization and wireless communication, investigate how to deliver intuitive interfaces on mobile devices, and the support for context-awareness. Regarding the user localization, we will adopt solutions characterised by a good balance between installation cost and localization precision.
This task will be performed in collaboration with the other units; it will be a specific task of the Pisa unit to devise how to integrate in the software architecture detection and communication of the activities performed by other users, coordination aspects and the support for group cooperation, which are not usually required by mobile guides for individual use.

For supporting cooperation, the software infrastructure will have to include the following aspects:
- distance communication by means of mobile devices mediated by a server;
- management of data related to the accesses to information (art works, artists, ...) and to specific activities (for instance game related taks) performed by users, as well as data related to user localization;
- the generation of multimedia interfaces specialized according to the user and the device, which in general could differ in the technology used;
- support for group cooperation;
- the way to exploit guide related information in the context of collaborative activities.

Moreover, during the first year of the project, the Pisa unit will explore in particular the application of the multi-user support component in the context of museum visits. They will investigate the most suitable educational cooperative games to support the museum experience and will devise application scenarios for the guides in collaborative educational games within the museum. Two classes of games will be considered: those where cooperation is explicit and those where cooperation is implicit and is carried on by means of information concerning individual activities performed by other users. In the former kind of activities users are requested to share and discuss what they have learned during the visit and are called to cooperate in the solution of a shared game. As an alternative, one can think of different forms of implicit cooperation where users are invited to explore the museum individually in order to solve distinct parts of a shared game, such as a puzzle. Differently from what happens in the Nottingham project, where the collaboration among visitors takes place only when they are in front of interactive displays, the collaboration will be supported by mobile devices during the all visit, or until visitors agree to participate in the game and have a common goal.

Still in relation to museum guides, the different content categories to be provided by the mobile guide and the different dimensions in adaptivity will be analyzed and the requisites for the natural language comment generation module for this application domain defined.

Finally, the unit of Pisa will study and define the specifications for a museum mobile guide with cultural contents and multi-user edutainment functionalities.
The application field will be the Marble Museum in Carrara, a multi-disciplinary museum containing a number of sections, related in various ways to the marble (sculpture, roman archeology, working techniques, crafts. ...). This material has been used in many artworks famous worldwide and for this reason the museum has a wide foreign audience. It is located a few kilometres away from the marble quarries, which are often the destination of tourist visits.
The final goal is to create a guide supporting groups of visitors (some of them could be inside the museum, some at the quarries site) in a way that makes their experience stimulating and interesting.

Expected results at the end of the first year

- Software module of the architecture to support communication and cooperation among users by means of mobile devices.
- Document on the implementation of the software module to support communication and cooperation among users.
- Analysis and requirement specification document for the implementation of a museum guide with natural language adaptive comments, and support for multi-user visits and group games.

Second year

During the second year of the project the unit of Pisa will study and take care of implementing the part of the framework concerning the generation of adaptive comments, used to provide natural languages commentaries on mobile devices, also in audio form.
The general idea is to develop a methodology and software tools to deploy virtual guides which can act very much like real world guides, by using similar techniques in order to engage visitors and make their visit interesting and enjoyable. The virtual guide application will be available in addition to the application supporting access to information related to the artworks and related aspects, in such a way as to add additional information depending on the user behaviour and profile. The comments will be mainly provided through the vocal channel in order to avoid interfering the real museum fruition mainly performed through the visual channel.
In order to support a dynamic comment generation functionality reusable in different application contexts, the generation software will rely on a interoperable knowledge representation layer (which is the task of Torino to produce); moreover it will be necessary to clearly separate, in the comment generation strategy, the context-invariant part, which could be reused in different applications, from the variable part.
The first step towards defining and implementing the software module for comment generation will be extending to the other application domains the analysis about the categories of information which are being used effectively in the context of museum guides (introductory information, comparison information, summary ... ) as well as the dimensions in adaptivity, necessary for flexible comment generation.
This will be done with generality in mind, by considering the result of parallel analysis carried on in cooperation with the other units in their respective application domains and with the goal of devising the basic discourse structures characterizing this kind of communication and their main variants, as well as the rules for their composition.
This process will result in the definition of an XML based language for describing contents so as to render the interface generation system independent of the application.
Next, a strategy of dynamic comment generation will be defined and a module for adaptive and dynamic comment generation implemented, as general as possible or at least easily adaptable to different context of use.

In this second year, the unit of Pisa will also implement a prototype of mobile museum guide with cultural contents and multi-user edutainment functionalities.
The technologies developed inside the project will be tested on the field, by extending the existing experimental setting already in place at the Marble Museum in Carrara. The new guide will extend the functionalities of the previous one by introducing adaptive comments, generated dynamically in real time, and will include exploration experiments by means of collaborative games to be played by visitors.
On-field experiments will allow evaluating the effectiveness of the adopted solutions with real visitors. The evaluation will be performed with logging tools that provide quantitative information regarding visit time, games performed, and the tasks accomplished. In addition, we will ask users to fill in questionnaires, which can be useful to gather information related to their satisfaction and appreciation of the museum experience.

Expected results at the end of second year

- Software module for the dynamic generation of natural language comments
- Engineered prototype of a guide for the Marble Museum in Carrara, with edutainment functionalities and dynamic comment generation.
- Report on the comment generation module and guidelines for contents editing.
- Report on usability evaluation of the guide prototype, on the basis of users tests with visitors of the museum.
Dipartimento di Informatica, Pisa, Italy