All projects

or select a theme:    Accessibility Affordances Data Visualization Digital Instruments Document Engineering Experience analysis and modelling Hypervideos Immersive Analytics Interpretation Systems Knowledge Engineering Learning Analytics Patient Experience Progressive analytics Reflective Systems Technology-Enhanced Learning Trace-based Activity Analysis Trace-based Systems VR and Psychotherapy Video Annotations Virtual Reality

• 

  ACAV: Collaborative Annotation for Video Accessibility

  Accessibility Document Engineering Video Annotations

  ACAV was dedicated to improving web video accessibility through annotations, mainly aimed at hard-hearing, deaf and blind people. We proposed a modeled approach to video enrichment, and studied various renderings with blind subjects.

  Details

  Partners: LIRIS, EURECOM, Dailymotion
  Funding: French Industry Ministry funding, "Innovative web"
  Beginning: 2009-10-01
  End: 2011-10-30
  

  Related publications

  • Benoît Encelle, Magali Ollagnier-Beldame, Yannick Prié. (2013) Towards the usage of pauses in audio-described videos in W4A 10th International Cross-Disciplinary Conference on Web Accessibility, may 2013. doi Show abstract Classical audiodescription process for improving video accessibility sometimes finds its limits. Depending on the video, required descriptions can be omitted because these may not fit in the durations of "gaps" in the video soundtrack (i.e. "void" spaces between dialogues or important sound elements). To address this issue, we present an exploratory work that focuses on the usage of "artificial" pauses in audio-described videos. Such pauses occur during the playing of the video so as to transmit more audio-descriptions. Our results show artificial pauses offer a good acceptability level as well as a low disturbing effect.
  • Benoît Encelle, Magali Ollagnier-Beldame, Stéphanie Pouchot, Yannick Prié. (2011) Annotation-based video enrichment for blind people: A pilot study on the use of earcons and speech synthesis in 13th International ACM SIGACCESS Conference on Computers and Accessibility. Dundee, Scotland. pp. 123-130. Oct 2011. doi Show abstract Our approach to address the question of online video accessibility for people with sensory disabilities is based on video annotations that are rendered as video enrichments during the playing of the video. We present an exploratory work that focuses on video accessibility for blind people with audio enrichments composed of speech synthesis and earcons (i.e. nonverbal audio messages). Our main results are that earcons can be used together with speech synthesis to enhance understanding of videos; that earcons should be accompanied with explanations; and that a potential side effect of earcons is related to video rhythm perception.
  • J.F. Saray Villamizar, B. Encelle, Y. Prié, P-A. Champin. (2011) An Adaptive Videos Enrichment System Based On Decision Trees For People With Sensory Disabilities in 8th International Cross-Disciplinary Conference on Web Accessibility, Hyderabad, India Show abstract The ACAV project aims to improve videos accessibility on the Web for people with sensory disabilities. For this purpose, additional descriptions of key visual/audio information of the video, that cannot be perceived, are presented using accessible output modalities. However, personalization mechanisms are necessary to adapt these descriptions and their presentations according to user interests and cognitive/physical capabilities. In this article, we introduce the concepts needed for personalization and an adaptive personalization mechanism of descriptions and associated presentations is proposed and evaluated.
  • Pierre-Antoine Champin, Benoît Encelle, N.W.D. Evans, Magali Ollagnier-Beldame, Yannick Prié, Raphaël Troncy. (2010) Towards Collaborative Annotation for Video Accessibility in 7th International Cross-Disciplinary Conference on Web Accessibility (W4A 2010), Raleigh, USA. 2010. doi Show abstract The ACAV project aims to explore how the accessibility of web videos can be improved by providing rich descriptions of video content in order to personalize the rendering of the content according to user disabilities. We present a motivating scenario, the results of a preliminary study as well as the different technologies that will be developed.
  • Benoît Encelle, Yannick Prié, Olivier Aubert.. (2010) Annotations pour l'accessibilité des vidéos dans le cas du handicap visuel in Handicap 2010, Jun 2010, Paris, France. IFRATH, pp.130-136, 2010. Show abstract L'accessibilité des contenus vidéo aux personnes en situation de handicap représente un enjeu important pour l'égalité de l'accès aux ressources numériques. Cet article expose une approche à base de métadonnées visant à améliorer l'accessibilité des vidéos aux personnes mal et non-voyantes. On ajoute dans un premier temps à une vidéo des annotations qui décrivent textuellement des éléments visuels clefs de son contenu. Dans un deuxième temps, on construit, à l'aide de modes de présentation de ces annotations et de la vidéo, des rendus enrichis de celle-ci, accessibles. Après le recueil et l'analyses des besoins des non-voyants, nous avons mené une étude préliminaire permettant d'évaluer notre approche et les mécanismes associés. Le développement de ces propositions et leur application à d'autres types de handicaps sont discutés et des perspectives, en cours d'étude au sein du projet ACAV (Annotation Collaborative pour l'Accessibilité Vidéo), sont exposées.

  Additional comments

  ACAV means “Annotation collaborative pour l’accessibilité vidéo”.

  We had already began playing with Advene on video annotations and accessibility, and the ACAV project allowed us to get funding to get further.

  Additional illustrations

  
• 

  ADVENE: Annotate Digital Video, Exchange on the NEt

  Document Engineering Hypervideos Interpretation Systems Reflective Systems Video Annotations

  Project on video annotations, hypervideos, concept, models and tools for video active reading. The Open-Source Advene software runs on Windows, Linux and Mac OS. It features the Cinelab / Advene2 model for Hypervideo description and Exchange. A lot of information can be found on the website.

  Details

  Partners: Numerous
  Funding: at first no funding (2003-2007), then Cinelab project on film annotations (ANR RIAM, 2007-2008, with LIRIS, IRI Centre Pompidou, Forum des Images, Antenna Audio), then Bertrand Richard PhD thesis, and other projects (notably Cinecast on Film Digital Libraries).
  Beginning: 2002-10-01
  End: false
  Website: http://advene.org
  

  Related publications

  • Olivier Aubert, Yannick Prié, Daniel Schmitt. (2012) Advene as a Tailorable Hypervideo Authoring Tool: a Case Study in ACM Symposium on Document Engineering, Paris, sept 2012, pp. 79-82 . doi Show abstract Audiovisual documents provide a great primary material for analysis in multiple domains, such as sociology or interaction studies. Video annotation tools offer new ways of analysing these documents, beyond the conventional transcription. However, these tools are often dedicated to specific domains, putting constraints on the data model or interfaces that may not be convenient for alternative uses. Moreover, most tools serve as exploratory and analysis instruments only, not proposing export formats suitable for publication. We describe in this paper a usage of the Advene software, a versatile video annotation tool that can be tailored for various kinds of analyses: users can define their own analysis structure and visualizations, and share their analyses either as structured annotations with visualization templates, or published on the Web as hypervideo documents. We explain how users can customize the software through the definition of their own data structures and visualizations. We illustrate this adaptability through an actual usage for interview analysis.
  • Yannick Prié. (2009) Interacting with Annotated Videos in In Workshop Interacting with temporal data at CHI 2009. Boston, MA, April 2009. 4pp. Show abstract We present Advene, a project and a prototype dedicated to video active reading, i.e. inscribing annotations on a video document and taking advantage of these annotations for navigating it. Both the video and its annotations are temporal data that need to be presented and interacted with. We present some illustrative interfaces, and discuss some of the challenges related to such a complex activity.
  • Olivier Aubert, Pierre-Antoine Champin, Yannick Prié, Bertrand Richard. (2008) Canonical processes in active reading and hypervideo production in Multimedia Systems Journal, Special Issue on Canonical Processes of Media Production. 14(6):427-433, 2008 doi Show abstract Active reading of audiovisual documents is an iterative activity, dedicated to the analysis of the audiovisual source through its enrichment with structured metadata and the definition of appropriate visualisation means for this metadata, producing new multimedia objects called hypervideos. We will describe in this article the general decomposition of active reading and how it is put into practice in the Advene framework, analysing how its activities fit into the Canonical Media Processes model.
  • Bertrand Richard, Yannick Prié, Sylvie Calabretto. (2008) Towards a unified model for audiovisual active reading in Tenth IEEE International Symposium on Multimedia, Berkeley, CA, USA, dec 2008. [AR:24%] doi Show abstract Audiovisual active reading is a commonly practised activity in different fields. This creative process consists in describing, criticizing and building an analysis during the reading of a document. More and more tools aim at providing ways to perform part of this activity, built on diferent description models. However, the whole activity is lacking a full support. In this paper, we present our work on a unified model for active reading of audiovisual documents, taking into account each part of this activity. Our approach is based on active reading data and activity modeling so as to propose a pertinent, generic and adaptive model.
  • Olivier Aubert, Pierre-Antoine Champin, Yannick Prié. (2008) Advene, une plate-forme ouverte pour la construction d'hypervidéos in Ludovia 2008, Ax-les-Thermes, aug 2008, 10 pp. Show abstract Nous présentons le projet Advene et les enjeux abordés : 1/ la question du public visé et apte à s'emparer d'un nouvel outil pour construire de nouvelles pratiques ; 2/ la souplesse nécessaire pour permettre de développer des pratiques innovantes (qui pourra entrer en conflit avec la nécessaire convivialité d'un outil s'adressant à un large public) ; 3/ les enjeux soulevés par le projet qui dépassent largement des aspects techniques ou ergonomiques et posent également des questions sur le droit d'auteur.
  • Yannick Prié, Ilias Yocaris. (2007) Informatique et analyse filmique : indécidabilité et contagion sémiotique dans Epidemic de Lars von Trier in Semio 2007, 8pp. Show abstract D’un point de vue strictement technique, notre exposé vise à esquisser une méthode de travail permettant de créer de nouveaux outils stylistiques destinés à l’analyse filmique. Pour ce faire, nous nous proposons d’analyser le film de Lars Von Trier et Niels Vørsel Epidemic (1988 ; tourné en noir et blanc). Partant d’une étude détaillée du fonctionnement sémio-stylistique de ce film, nous entendons montrer : (1) Qu’il se présente comme un « système textuel » (cf. Metz 1971 : 107-108 et passim) découlant d’une série d’interactions extrêmement complexes entre différents codes sémiotiques (discursifs, picturaux, narratifs, génériques …) ; (2) Que sa dimension « pluri-codique » (Metz 1971 : 90) doit être décrite dans le cadre d’une approche résolument interdisciplinaire, permettant de conjoindre aux acquis de la sémiotique filmique une série de concepts empruntés à la stylistique textuelle (le concept riffaterrien de « convergence »), à la narratologie (le concept genettien de « métalepse narrative »), à la philosophie du langage (le concept goodmanien d’« exemplification », le concept derridien d’« indécidabilité ») etc. ; (3) Que l’efficacité d’une telle démarche d’analyse est démultipliée par l’utilisation d’outils informatiques adéquats, qui peuvent être développés ad hoc.
  • Pierre-Antoine Champin, Yannick Prié. (2007) Models for sustaining emergence of practices for hypervideo in International Workshop On Semantically Aware Document Processing And Indexing (SADPI'07), May 2007, 10pp. Show abstract The work presented in this paper aims at covering several domains: hypervideo modelling, document annotation, and practices sharing and emergence. It is based on the Advene project, providing a model and a prototype for creating, rendering and sharing annotations of audiovisual documents. After a presentation of the notion of multi-structurality in documents and a definition of hypervideos, we present the original Advene model. We then discuss some limitations observed in our model, and introduce a new model for hypervideos. We finally discuss related work with respect to our model, and raise the problem of the emergence of semantics in videos and hypervideos.
  • Olivier Aubert, Yannick Prié. (2007) Annotations de documents audiovisuels - Temporalisation et spatialisation in Atelier Interfaces pour l'annotation et la manipulation d'objets temporels : une comparaison des outils et des paradigmes dans le domaine musical et cinématographique, nov 2007, Paris. Show abstract Le prototype open-source Advene offre une plateforme pour l’intégration, la visualisation et l’échange de métadonnées sur des documents audiovisuels. Il permet aux utilisateurs de définir eux-mêmes, en fonction des tâches à effectuer, les structures des métadonnées et la manière de les visualiser. Il permet ainsi de mettre en ½uvre des pratiques de lecture active de documents audiovisuels. Nous présentons ici quelques éléments de réflexion sur la pratique d’annotation de documents temporalisés.
  • Bertrand Richard, Yannick Prié, Sylvie Calabretto. (2007) Lecture active de documents audiovisuels : organisation de connaissances personnelles par la structuration d’annotations in Ingénierie des Connaissances 2007, Grenoble, July 2007, 12 pp. Show abstract Au cours de la pratique de la lecture active, un lecteur manipule une quantité importante de connaissances, liées à sa pratique et au document étudié. Ces connaissances au coeur de l’activité sont amenées à évoluer afin de correspondre au mieux à la pratique du lecteur. Nous présentons dans cet article nos réflexions relatives à la construction et la manipulation de connaissances personnelles au cours de cette activité et à l’évolution de ces connaissances au fil de la pratique. Après étude des modèles existants, nous proposons un modèle de structuration de connaissances adapté aux contraintes de la lecture active et permettant de diffuser les différentes pratiques de la lecture active.
  • Olivier Aubert, Yannick Prié. (2007) Advene: an open-source framework for integrating and visualising audiovisual metadata in Open Source Competition: Technical presentation and overview paper, Proc. of ACM Multimedia Conference, Augsburg, sept. 2007, pp. 1005-1008. doi Show abstract The open-source Advene prototype offers a framework for integrating and visualising audiovisual metadata. It allows users to define by themselves, according to their specific tasks, the structure of the metadata as well as the different ways in which it should be visualised. By storing metadata and visualisation specifications independently from the audiovisual document, it allows to share analyses and comments on any audiovisual document. Its open nature and simple principles make it an ideal testbed for experimentation with new audiovisual metadata interaction modalities.
  • Olivier Aubert, Yannick Prié. (2006) Des vidéos aux hypervidéos : vers d'autres interactions avec les médias audiovisuels in Technique et science informatique, Vol 25, num 4, 2006, pp. 409-436, 2006. doi Show abstract La numérisation des documents permet d'en envisager des exploitations novatrices, comme la démontré le succès du web. La capacité des systèmes informatiques permet maintenant d'exploiter les documents audiovisuels, mais tous les concepts et outils ne sont pas encore présents. Afin de fournir un support à la réflexion sur les utilisations innovantes des documents audiovisuels numériques, nous présentons et discutons ici la notion d'hypervidéo, qui permet d'une part, d'analyser des instances existantes d'hypervidéos, et d'autre part, de donner un guide de construction pour des systèmes mettant en oeuvre les hypervidéos. Nous décrivons ensuite le projet Advene, qui vise à fournir une plate-forme permettant 1) la création d'hypervidéos (visualisations dynamiques et statiques de données associées à un médium audiovisuel) ; 2) l'échange des données et des vues associées, afin de permettre un travail collaboratif sur ce type de médium.
  • Olivier Aubert, Pierre-Antoine Champin and Yannick Prié. (2006) Integration of Semantic Web Technology in an Annotation-based Hypervideo System in SWAMM 2006, 1st International Workshop on Semantic Web Annotations for Multimedia, Held as part of the 15th World Wide Web Conference, May 2006, 12 pp. Show abstract This article discusses the integration of semantic web technologies (ontology and inference) into audiovisual annotation based models and systems. The Advene project, aimed at all purpose hypervideo generation from annotated audiovisual documents, is used as a testbed. Advene principles and the Advene prototype are first presented, before a discussion on how ontology and reasoning have been easily integrated into the Advene framework. Some motivating examples are proposed, and our proposals and related works are discussed.
  • Olivier Aubert, Yannick Prié. (2005) Des hypervidéos pour créer et échanger des analyses de documents audiovisuels in H2PTM 2005, Créer, jouer, échanger : expériences de réseaux, nov. 2005, Paris, 10 pp. Show abstract Les possibilités offertes par l'hypertexte ne sont pas encore très exploitées dès lors que l'on travaille avec des documents audiovisuels. Le besoin de métadonnées y est plus immédia- tement perceptible, la temporalité des documents pose des problèmes nouveaux en termes de manipulation et d'interaction, et la question des droits étant loin d'être réglée. Afin de pouvoir analyser les expérimentations novatrices actuelles dans ce domaine et éclairer la construction de nouveaux systèmes, nous présentons la notion d'hypervidéo comme cadre général mettant en évidence différentes dimensions des hypermédias à base de documents audiovisuels. Le projet open-source Advene nous permet de mener des expérimentations de mise en application des hypervidéos et fournit un support à l'innovation par le bas propice à l'émergence de nouveaux modes d'interaction audiovisuelle.
  • Olivier Aubert and Yannick Prié. (2005) Advene: Active Reading through Hypervideos in ACM Conference on Hypertext and Hypermedia 05, 2005, Salzburg, Austria, pp. 235-244. [AR:36%] doi Show abstract Active reading and hypermedia usage are an integral part of scholar daily practices, but the full exploitation of their potentialities still lies far ahead. In the search for new methods and tools, we focus in this article on the use of audiovisual material in a scholar context. One of the results of active reading applied to audiovisual material can be hypervideos, that we define as views on audiovisual documents associated with an annotation structure. The notion of hypervideo is useful to analyse existing video-based hypermedia systems as well as building new systems. The Advene project proposes an implementation of hypervideos through a framework that allows experimentations of new visualisation and interaction modalities for enriched videos.
  • Olivier Aubert, Yannick Prié. (2004) Documents audiovisuels instrumentés. Temporalités et détemporalisations dans les hypervidéos in Document numérique, numéro spécial : Temps et document, vol. 8, num. 4, pp.143-168, 2004. doi Show abstract Nous nous intéressons tout d’abord dans cet article aux documents audiovisuels et aux diverses temporalités qui leur sont liées. Après un état des lieux des possibilités actuelles des systèmes d’information audiovisuelle, nous définissons plus précisément ce que sont les hyper- vidéos, ou documents audiovisuels numériques instrumentés. Nous décrivons et illustrons alors un certain nombre des caractéristiques des objets hypervidéos en lien avec les temporalités dé- finies, qui en font des objets complexes. Nous présentons alors un début de cadre d’analyse des synchronisations entre temporalités des hypervidéos générées et vécues.
  • Olivier Aubert, Yannick Prié. (2004) From video information retrieval to hypervideo management in International Workshop on Multidisciplinary Image, Video, and Audio Retrieval and Mining. Sherbrooke-Canada, October 25-26 2004 Show abstract The digital revolution that has somehow taken place with the World-Wide Web took advantage of the availability and interoperability of tools for visualisation and manipulation of text-based data, as well as the satisfying pertinence of search engines results that makes them usable by non-expert users. If we are to undergo such a revolution in the audiovisual domain, a number of issues mainly related to the temporal nature of audiovisual documents have to be resolved. In this article, we expose our view of the current state of Audiovisual Information Systems, and suggest that they should be considered as video information management systems rather than video retrieval systems, in order to foster new uses of video information. We define the notion of hypervideo that can be used as an analysis framework for this issue. We then describe our implementation of hypervideos in the Advene framework, which is aimed at DVD material annotation and analysis. We eventually discuss how the notion of hypervideo puts video information retrieval in a somewhat different light.
  • Olivier Aubert, Pierre-Antoine Champin, Yannick Prié. (2003) Instrumentation de documents audiovisuels : temporalisations et détemporalisation dans le projet Advene in Workshop "Temps et documents numériques", Nov 2003, Grenoble, France. 2003 Show abstract Nous présentons dans cet article les différents éléments du modèle Advene pour représenter des annotations audiovisuelles, et présenter celles-ci à l’utilisateur sous la forme l’hypervidéos, documents hypermédias générés à partir du document audiovisuel original annoté et des annotations. Les hypervidéos sont des documents multimédias complexes, du fait de la part importante prise par l’élément vidéo, qui possède des temporalités particulières. Afin d’être à même de construire des hypervidéos conduisant à des utilisations novatrices du matériel audiovisuel, nous proposons un modèle d’analyse de celles-ci suivant différentes temporalités.

  Related software

  • Advene

  Additional comments

  Please use ACM Hypertext 2005 reference as main citation.

  Additional illustrations

  
• 

  Cinecast: Film enrichments and video metadata exchange

  Document Engineering Interpretation Systems Video Annotations

  Film enrichment in public film libraries and on the web. Exchange of film annotations on social networks. We worked on hypervideos models, and video annotations sharing.

  Details

  Partners: Researchers (LIRIS, IRI / Centre Pompidou, LIST / CEA, Telecom Paris Tech); major film libraries (Forum des Images, Cinémathèque française, BNF, BPI, INA); and industrial partners (NETIA, Globe Cast, Univers Ciné, lesite.tv, Jamespot, Exalead, Allociné, VodKaster).
  Funding: French Industry Ministry funding, FUI9 call
  Beginning: 2010-10-01
  End: 2012-12-31
  

  Related publications

  • Madjid Sadallah, Olivier Aubert, Yannick Prié. (2014) CHM: an Annotation- and Component-based Hypervideo Model for the Web in Multimedia Tools and Applications, 70(2), 869-903, may 2014 doi Show abstract Hypervideos are hypermedia documents that focus on video content. While they have long been deployed using specialized software or even hardware, the Web now offers a ground for them to fit into standardized languages and implementations. However, hypervideo design also currently uses very specific models limited to a single class of documents, or very generic hypermedia models that may not appropriately express their specific features. In this article we describe such features, and we introduce CHM, an annotation-driven and component-based model to conceptualize hypervideos through a high level operational specification. An extensible set of high level components is defined to emphasize the presentation and interaction features modeling, while lower level components offer more flexibility and customization opportunities. Being annotation-based, the model promotes a clear separation between video content/metadata and their various potential presentations. We also describe WebCHM, an implementation of CHM with standard Web technologies that provides a general framework to experiment with hypervideos on the Web. Two examples are provided as well as a preliminary usage study of the model and its implementation to validate our claims and proposals.
  • Yannick Prié, Vincent Puig. (2013) Nouveaux modes de perception active de films annotés in Cinéma, interactivité et société, direction, ouvrage, Université de Poitiers & CNRS, pp. 373-387. 2013 Show abstract Notre approche est à la fois théorique et technologique. Théoriquement, il s’agit de (re)penser nos rapports d’appropriation au sens large des ½uvres cinématographiques en s’appuyant sur les possibilités de perception active offertes par le numérique. Technologiquement, il s’agit alors de se donner les moyens de décrire et d’implémenter différents outils de perception active de corpus de films. La notion d’annotation audiovisuelle comme matérialisation première de toute rencontre avec celui-ci est à cet égard centrale et sert de base à la construction d’un cadre de pensée adéquat de la perception active. Nous décrivons quelques exemples d’outils de lecture active de films pour la critique et l'enseignement, ainsi que des dispositifs artistiques de restitution. Différentes questions de recherche en cours d’exploration sont enfin abordées et permettent d’illustrer la richesse des travaux en cours.
  • Madjid Sadallah, Olivier Aubert, Yannick Prié. (2011) Hypervideo and Annotations on the Web in Workshop on Multimedia on the Web 2011. Graz, Austria. Sept 2011. doi Show abstract Effective video-based Web information system deployment is still challenging, while the recent widespread of multimedia further raises the demand for new online audiovisual document edition and presentation alternatives. Hyper video, a specialization of hypermedia focusing on video, can be used on the Web to provide a basis for video-centric documents and to allow more elaborated practices of online video. In this paper, we propose an annotation-driven model to conceptualize hyper videos, promoting a clear separation between video content/metadata and their various potential presentations. Using the proposed model, features of hyper video are grafted to wider video-based Web documents in a Web standards-compliant manner. The annotation-driven hyper video model and its implementation offer a general framework to experiment with new interaction modalities for video-based knowledge communication on the Web.
  • Madjid Sadallah, Olivier Aubert, Yannick Prié. (2011) Component-based Hypervideo Model: high-level operational specification of hypervideos in ACM DocEng. Mountain View, California. pp. 53-56. Sept 2011. doi Show abstract Hypervideo offers enhanced video-centric experiences. Usually defined from a hypermedia perspective, the lack of a dedicated specification hampers hypervideo domain and concepts from being broadly investigated. This article proposes a specialized hypervideo model that addresses hypervideo specificities. Following the principles of component-based modeling and annotation-driven content abstracting, the Component-based Hypervideo Model (CHM) that we propose is a high level representation of hypervideos that intends to provide a general and dedicated hypervideo data model. Considered as a video-centric interactive document, the CHM hypervideo presentation and interaction features are expressed through a high level operational specification. Our annotation-driven approach promotes a clear separation of data from video content and document visualizations. The model serves as a basis for a Web-oriented implementation that provides a declarative syntax and accompanying tools for hypervideo document design in a Web standards-compliant manner.
  • Benoît Encelle, Pierre-Antoine Champin, Yannick Prié, Olivier Aubert. (2011) Models for Video Enrichment in Proceedings of the 11th ACM symposium on Document engineering. Mountain View, California. pp. 85-88. Sept 2011. doi Show abstract Videos are commonly being augmented with additional content such as captions, images, audio, hyperlinks, etc., which are rendered while the video is being played. We call the result of this rendering "enriched videos". This article details an annotation-based approach for producing enriched videos: enrichment is mainly composed of textual annotations associated to temporal parts of the video that are rendered while playing it. The key notion of enriched video and associated concepts is first introduced and we second expose the models we have developed for annotating videos and for presenting annotations during the playing of the videos. Finally, an overview of a general workflow for producing/viewing enriched videos is presented. This workflow particularly illustrates the usage of the proposed models in order to improve the accessibility of videos for sensory disabled people.

  Related software

  • Advene

  Additional comments

  This ambitious project was featuring a really great line up.

  Additional illustrations

  
• 

  COCo: Comin Open Courseware

  Technology-Enhanced Learning Hypervideos Video Annotations

  Research and development aimed at leveraging annotations in video-centered pedagogical resources, creating open courseware multimodal content for knowledge diffusion, and exploring new techniques for e-learning. More than 30 courses and seminar videos have been produced.

  Details

  Partners: LS2N
  Funding: Labex Cominlabs
  Cofunding: Région Pays de la Loire
  Beginning: 2013-09-01
  End: 2016-12-31
  Website: http://www.comin-ocw.org/
  

  Related publications

  • Camila Canellas, Olivier Aubert, Yannick Prié. (2015) Prise de note collaborative en vue d’une tâche : une étude exploratoire avec COCoNotes Live in 17ème Conférence sur les Environnements Informatiques pour l'Apprentissage Humain (EIAH 2015), Jun 2015, Agadir, Maroc. pp.204-209, 2015 Show abstract Les enregistrements de présentations orales telles que celles effectuées lors de conférences peuvent souvent constituer une base pour 1) une nouvelle activité basée sur leur contenu, 2) une autre forme pour le même contenu, ou 3) une combinaison avec d'autres documents. Cependant, les enregistrements vidéo peuvent se révéler difficiles à naviguer et explorer. Nous présentons ici les résultats préliminaires d’une situation où un outil de microblogging est utilisé durant l'enregistrement pour produire des annotations catégorisées et synchronisées avec l'enregistrement. Les annotations sont ensuite utilisées pour naviguer dans l'enregistrement et produire de nouveaux documents, dans le contexte d'une tâche spécifique.
  • Olivier Aubert, Yannick Prié, Camila Canellas. (2014) Leveraging video annotations in video-based e-learning in 6th International Conference on Computer Supported Education, Barcelona, Spain, April 2014 Show abstract The e-learning community has been producing and using video content for a long time, and in the last years, the advent of MOOCs greatly relied on video recordings of teacher courses. Video annotations are information pieces that can be anchored in the temporality of the video so as to sustain various processes ranging from active reading to rich media editing. In this position paper we study how video annotations can be used in an e-learning context - especially MOOCs - from the triple point of view of pedagogical processes, current technical platforms functionalities, and current challenges. Our analysis is that there is still plenty of room for leveraging video annotations in MOOCs beyond simple active reading, namely live annotation, performance annotation and annotation for assignment; and that new developments are needed to accompany this evolution.

  Related software

  • COCoNotes
  • COCoNotes Live

  Additional comments

  The project was focused on open education and open course ware (OCW), all material is available on the project website.
• 

  DIADEME: Digital Instrument and Activity DEvelopMEnt

  Digital Instruments Experience analysis and modelling Interpretation Systems Trace-based Activity Analysis

  A small research program around the notion of Digital Instrument, and their co-evolution with humans, mainly related to Amaury Belin PhD thesis, as well as HdR writing. May be reactivated.

  Details

  Partners: LS2N
  Funding: Amaury Belin PhD funding, Atlanstic 2020
  Beginning: 2012-10-01
  End:
  

  Related publications

  • Amaury Belin, Yannick Prié, Aurélien Tabard. (2014) Supporting the Development of Digital Skills in Digital Intelligence, Nantes, France, sept 2014. Show abstract In a world where activities, goals and available software are rapidly changing, users must constantly adapt. In this position paper, we discuss how digital skills are different from traditional skills due to their highly dynamic na-ture, both in the tools used and the tasks to be carried out. We advocate the needs both for interdisciplinary theory to conceptualize digital skill develop-ment, and for longitudinal, large-scale and trace-based methods to observe such phenomenon. We illustrate how digital tools could better support users in the development of skills, highlighting how traces of interaction could be leveraged within reflective and skill-sharing tools.
  • Amaury Belin, Antipov Grigory, Julien Blanchard, Fabrice Guillet, Yannick Prié. (2014) Mining Users Skills Development From Interaction Traces: an exploratory study in Atelier Fouille visuelle de données temporelles, IHM 2013, Nov 2014, Bordeaux, France. 2p, 2014 Show abstract In this paper, we report an exploratory work related to the detection of user skills development from large interactions traces. We present a statistical approach to tackle the development of user's low level skills by identifying patterns of action and studying their progressive automations (i.e. as a user develops skills, they execute common subsequent actions faster) as well as their evolutions (i.e. user elaborating innovative ways to achieve a common task).
  • Amaury Belin, Yannick Prié. (2012) Towards a model for describing appropriation processes through the evolution of digital artifacts in Designing Interactive Systems DIS2012, pp. 645-654, Newcastle, June 2012. [AR:20%] doi Show abstract Appropriation of technology is a process by which users complete the work of designers by making interactive systems functional within the frame of their situated activities. While existing theories and studies about appropriation are oriented toward the psychological or organizational dimension of this process, we propose a model to describe it through evolutions of digital artifacts and information structures. We also present a case study demonstrating how this model helps to identify particular user operations, and related digital transformations, as a part of the appropriation process. These findings open perspectives to bridge scattered theoretical approaches of appropriation around a low-level, artifact-oriented, and objective way of describing appropriation. Our model could also improve the way appropriation is taken into account in design, by bringing more focus on technical aspects of interactive systems.
  • Yannick Prié. (2011) Vers une phénoménologie des inscriptions numériques. Dynamique de l’activité et des structures informationnelles dans les systèmes d’interprétation in Habilitation à Diriger des Recherches en Informatique, Université Claude Bernard Lyon 1, Nov 2011, 247 p. (Soutenue le 18 novembre 2011 à l'Université Claude Bernard Lyon 1 - Laboratoire LIRIS - UMR 5205 CNRS - École Doctorale Informatique et Information pour la Société) This study is mandatory to get a Professor position in France. The document is entitled Towards a Phenomenology of Digital Inscriptions. Dynamics of Activity and Informational Structures in Interpretation Systems. In the first chapter I first study the notion of inscription of knowledge as proposed by Bruno Bachimont. As I consider it is not sufficiently tied with human action and activity as considered by post-cognitivist theories, I propose to consider the notion of informational structures (structures informationnelles). Informational structures are digital inscriptions of knowledge that are actually manipulated by subjects during their computer-mediated activity, they can be canonical (explicitely manipulated by the system) or non canonical (but as such, they can become canonical if reified). The notion of informational structure is useful to study how digital inscriptions and activity are interwoven at different analysis levels, it is associated to the notion of information space that users enact during their activities, and enables to study at the individual level what could be called a phenomenology of digital inscriptions. In the second chapter of the document, I focus on explicit information spaces that users enact during knowledge work, and detail the notions of interpretation systems within which inscriptions of knowledge circulate. I present my work on video active reading and the Advene project. In the third chapter, I propose to define traces as inscriptions that are used during an activity as signs of the past, and I discuss the notion of interpretation systems that are dedicated to traces. I present my work on such systems, along first the Musette (Modelling Users and Tasks to Traces Experience) approach and then the mTBS (Modelled Trace-Based systems) approach.
    Examination committee: Nathalie Aussenac (Directeur de Recherche, CNRS, Toulouse; referee), Bruno Bachimont (Enseignant-Chercheur HDR, UT Compiè̀gne; referee), Sylvie Leleu-Merviel (Professeur, Université de Valenciennes; referee), Michel Beaudouin-Lafon (Professeur, Université Paris-Sud; examiner), Catherine Garbay (Directeur de Recherche, CNRS, Grenoble; examiner), Alain Mille (Professeur, Université Lyon 1; examiner), Pascal Salembier (Professeur, UT Troyes; examiner). Show abstract Ce mémoire est l'occasion de présenter nos travaux à l'Université Claude Bernard Lyon 1. Il est composé de trois parties, la première visant à proposer une thématisation originale des liens entre activité et inscriptions numériques de l'activité sous la forme de structures informationnelles, les deux suivantes étant consacrées à nos thématiques principales de recherche qui sont, d'une part la lecture active audiovisuelle et les systèmes d'interprétation, et d'autre part les systèmes à base de traces modélisées.
    Le premier chapitre est consacré à l'exploration des liens entre l'activité informatiquement médiée et les représentations qui y sont impliquées. Une première étude critique décrit la notion d'inscription de connaissances proposée par B. Bachimont au sein d'une théorie du support pour permettre de penser la rencontre ou l’expérience qu’une conscience fait des dispositifs technologiques et des inscriptions. Cette proposition vise à thématiser la numéricité des inscriptions et nous intéresse donc à ce titre, mais elle se révèle à notre sens insuffisante pour penser un individu actif engagé dans un processus de manipulation d'inscriptions. Une seconde étude est alors consacrée aux liens entre action, activité et inscriptions dans les théories dites « post-cognitivistes » de la cognition. Les inscriptions soutiennent l'activité et la sous-tendent tout à la fois, et nous nous intéresserons particulièrement aux inscriptions dans leur perception et leur manipulation par un être humain suivant la théorie instrumentale. Cependant, les inscriptions en tant qu'elles peuvent être numériques ne sont que peu thématisées. Notre dernière étude est orientée autour de la proposition de penser la notion de structure informationnelle et les instruments associés comme permettant d'articuler le monde numérique à l'activité humaine et l'activité humaine au monde numérique. Une structure informationnelle est une inscription numérique en acte, objectivable, mais non obligatoirement canonique, c'est-à-dire manipulée explicitement par le système. Cette proposition permet de penser le côté humain de l'activité instrumentée tout en conservant le calcul et les représentations associées comme préoccupation informatique. Nous présentons également la notion d'espace informationnel qu'un utilisateur énacte, et les directions de recherche ouvertes par nos propositions.
    Le deuxième chapitre est principalement consacré à la présentation de nos travaux sur les systèmes de lecture active audiovisuelle. Nous proposons d'abord une étude rapide sur le cadre général des technologies intellectuelles comme soutenant le travail intellectuel, l'activité ouverte d'interprétation et de manipulation d'inscriptions de connaissances personnelles. De telles inscriptions et réinscriptions se font au sein de ce que nous proposons d'appeler des systèmes d'interprétation qui offrent aux utilisateurs la possibilité de manipuler consciemment des structures informationnelles de toutes sortes, par exemple sous la forme de données, de schémas ou de feuilles de style et de formulaires, et de partager celles-ci comme réifications de pratiques. La lecture active audiovisuelle est une activité intellectuelle qui s'effectue dans un système d'interprétation permettant de construire des hypervidéos à partir d'annotations. Nous présentons alors nos travaux autour du projet Advene (Annotate Digital Video, Exchange on the NEt), notamment les modèles associés, l'outil générique Advene pour l'annotation et la construction d'hypervidéos, ainsi que quelques applications de lecture active associées à l'analyse des interactions et la critique filmique. Nous pouvons alors tirer un bilan des presque dix ans de ce projet et proposer quelques directions pour la suite.
    Le troisième et dernier chapitre présente essentiellement nos travaux sur les traces numériques. Nous définissons d'abord la notion de trace en général comme inscription permettant de viser le passé au cours d'une interprétation et présentons comment les traces d'activité médiées sont largement utilisées au sein de systèmes variés allant de l'analyse à la réflexivité. Nous considérons que l'enjeu est de manipuler des traces numérique explicites définies comme « inscriptions canoniques temporellement orientées » dans des systèmes d'interprétation orientés trace. Nous pouvons alors présenter la notion de trace modélisée comme un certain type de trace explicite, ainsi que nos travaux dans ce cadre depuis plus de dix ans : l'approche Musette, le cadre général des systèmes de base de traces modélisées (SBTm) ainsi que la formalisation des traces et des transformations pour construire des systèmes de gestion de bases de traces (SGBT). Différents travaux applicatifs sont ensuite présentés qui permettent d'illustrer les différentes utilisations des traces modélisées dans des contextes applicatifs variés, visant notamment le support à l'awareness, à la remémoration, à la réflexivité, à la redocumentation et au partage, ou encore à la reprise d'activité.

  Related software

  • GVMT
• 

  ECBridge: Education-Curation Bridge

  Data Visualization Learning Analytics Technology-Enhanced Learning

  EC Bridge aims to create software linking platforms dedicated to the world of education and cultural mediation, to offer cultural institutions support for new educational experiences linking schools, museums, and families. In this project we are implicated in the scenarios in the design of trace-based dashboard for learners, curators, and students.

  Details

  Partners: Mazedia, Maskott, LS2N, Ifé ENS-Lyon, Cité des Télécoms, Musée gallo-romain de Fourvière
  Funding: FUI 23
  Beginning: 2017-09-01
  End: 2019-12-31
  
• 

  eFIL: e-Feedback for interactive Lecture

  Data Visualization Learning Analytics Reflective Systems Technology-Enhanced Learning

  The project aims at experimenting in lab / classroom the usage of a handwritten note-taking tablet application, that also allows the creation of pedagogical activities (quiz, whiteboards). We want to study the impact of technology instrumentation on active learning methods, and the usage of activity traces for analysis and production of synchronous and post-hoc activity visualisation dashboards, mainly targeted at teachers.

  Details

  Partners: LS2N, IntuiDoc, LP3C
  Funding: Cominlabs Labex
  Beginning: 2017-10-01
  End: 2019-12-31
  Website: https://efil.cominlabs.u-bretagneloire.fr/fr
  
• 

  EPISTEME: Transdisciplinary epistemology of digital technologies

  Data Visualization Digital Instruments Interpretation Systems Reflective Systems Trace-based Activity Analysis

  Work on data analysis of astrophysical research activity, producing datavisualizations that highligh different processes related to observations, the organization of working groups, as well as the software "black boxes".

  Details

  Partners: IRI Centre Pompidou, LINA, LIRIS, CHS Paris 1, CEA Irfu, France Televisions, Mediapart
  Funding: French National Research Agency (ANR)
  Beginning: 2014-10-01
  End: 2018-02-28
  Website: https://projet-episteme.org/
  

  Additional comments

  The aim of this transdisciplinary program EPISTEME is first to identify the most characteristic aspects of the transformation of two chosen disciplines (astrophysics, history) in their relations to their objects under the effect of digital formalization. The other goal is to design instruments for the scientific communities of peers that allow contributive categorization, to support debates and controversies, based on the concept of transindividuation, and to articulate bottom-up -stemming from individual research activities- and top down processes -more or less local, laboratory-, school- or discipline-wide certification processes from peer community debates.

  We developed the Herschel Mission Explorer. We are currently working on a second version that will feature a general design overhaul.

  Additional illustrations

  
• 

  EREdWeB: Enriched Reality for Education and Digital Well-Being

  Virtual Reality

  One of the first projects of the "European University of Well-being" (EuniWell). This project explores the potential of state-of-the-art virtual reality technology for teaching, aiming at digital well-being. It will bring about groundbreaking research on the relation between virtual reality and education, namely regarding its cognitive, physiological, educational and social aspects.

  Details

  Partners: University of Florence (IT, leader), University of Birmingham (UK), Linnaeus University (SE), University of Leiden (NL), University of Nantes (FR)
  Funding: EuniWell
  Beginning: 202O-07-01
  End: 2021-06-30
  
• 

  EXTENT: Executive Functions Testing in VR with Embodied Cognition

  VR and Psychotherapy Virtual Reality

  Joint program between Hanyang University, University of Nantes, Nantes University Hospital, and Hanyang University Hospital to design, built, and test embodied cognition-based franco-korean VR environments for testing executive functions. 7 Korean students spent 6 months in Nantes, and 7 VR environments were built to assess several executive disfunctions. The project continued with a postdoctoral contract to further develop and test 3 environments.

  Details

  Partners: LS2N, CHU Nantes, Hanyang University, Hanyang University Hospital
  Funding: IITP (Korea), West Creative Industry (France)
  Cofunding: PHC STAR (French/Korean),
  Beginning: 2019-09-01
  End: 2021-06-30
  

  Related publications

  • . () in
  • Jieun Han, Gyu Hyun Kwon, Roman Malo, Yannick Prié, and Zied Kefi (Ed). (2023) Future Immersive Healthcare Experience in Medical and Home Setting in Proceedings of the 2023 ACM International Conference on Interactive Media Experiences Workshops (IMXw '23), Nantes, France, June doi Show abstract The rising accessibility of immersive media technologies opens the opportunity to use them easily in in-hospital and home environments for healthcare. The possibility to completely immerse users or patients in virtual environments, or to augment their surroundings, while tracking their movements and physiological reactions, provide new ways to induce or analyze user experiences. In the healthcare context, immersive media give new opportunities for diagnosis through the study of patient behaviors in controlled environments, as well as rehabilitation, by enabling patients to train in specific and personalized controlled tasks. In an at-home context, immersive media give opportunities to monitor and train mental and physical skills in order to improve users’ health and/or well-being. Care IMX is a half-day workshop of the IMX 2023 Conference, bringing together interdisciplinary medical, technological and social science researchers.
  • Nicolas Ribeiro, Toinon Vigier, Jieun Han, Gyu Hyun Kwon, Hojin Choi, Samuel Bulteau, Yannick Prié. (2024) Three Virtual Reality Environments for the Assessment of Executive Functioning Using Performance Scores and Kinematics: an Embodied and Ecological Approach to Cognition in Cyberpsychology, Behavior, and Social Networking, 27(2), 127-134. doi Show abstract Virtual reality is an attractive technology for cognitive assessment because it provides a more embodied experience than typical test situations using, for example, paper and pencil. In addition, virtual reality can immerse individuals in complex situations similar to real-life situations, thereby improving the ecological validity (i.e., face validity) of the assessment. Virtual reality can also provide improved scoring of tests as it facilitates the tracking of kinematic information and the temporal tracking of activities. This study assesses the correlation between scores on executive function assessments using standard neuropsychological tasks in paper-and-pencil format, on a tablet, and in three immersive virtual reality environments, each designed to involve specific aspects of executive function. This study also aims to assess the correlation between these performance scores and a set of kinematic measures (speed, duration, distance of the hand) collected in virtual reality. The outcomes (performance score and kinematic measure) correlate with previous assessment methods (paper and pencil and computerised 2D) and with each other, suggesting their potential usefulness in clinical and research contexts. The discussion focuses on the advantages of embodied, situated and spatialised tests for cognitive assessment and the benefits of kinematic tracking in virtual reality tests for the quality of this assessment.

  Additional comments

  Korean IITP funding: “Development of VR and IoT-based Systems for Evaluation of Embodied Cognition”: 7 Korean interns come in France for 6 months

  PHC (Programme Hubert Curien) STAR funding “EXTENT” Korea <> France travels

  Additional illustrations

  
• 

  HOMERIC: Hybridization of methOds for Modeling ExpeRIenCe in interactive systems

  Affordances Experience analysis and modelling VR and Psychotherapy Virtual Reality

  Our goal is to propose concepts, methods, and models for the study of user experience in virtual reality. We explore an hybrid approach that associates qualitative and quantitative methods, particularly first and third person methods. Our applicative aim is to build an affordance-based VR environment for elicitating several Rdoc dimensions associated to depression.

  Details

  Partners: LS2N, Sphere
  Funding: RFI Ouest Industries Créatives
  Cofunding: Université de Nantes
  Beginning: 2018-10-01
  End: 2019-12-31
  

  Related publications

  • Roman Malo, Yannick Prié. (2023) L’entretien d’explicitation comme outil d’investigation des processus psychiques en contexte expérimental : l’exemple de la flexibilité psychologique in Expliciter - Hors-série 1 - Actes du colloque Pierre Vermersch, pp 197-212 Show abstract Le travail que nous présentons ici est au croisement de la psychologie clinique et des sciences cognitives, dont l’enjeu est de proposer une investigation centrée subjectivité d’un processus psychologique. Il s’agit de souligner l’importance de l’expérience vécue explicitée pour affiner la compréhension des processus psychologiques, en prenant ici comme exemple la notion de flexibilité psychologique. Nous décrivons d’abord le “retour” du phénoménologique dans les modèles en psychologie actuelle dans un bref aperçu théorique, puis nous présentons des résultats du projet HOMERIC qui enrichissent notre compréhension du processus complexe qu’est la flexibilité en nous appuyant sur l’entretien d’explicitation comme outil mobilisé dans l’étude d’une situation expérimentale.
• 

  HUBBLE: HUman oBservatory Based on anaLysis of E-learning traces

  Learning Analytics

  The objective of this Learning Analytics project is to create a national observatory for the construction and sharing of data analysis processes of traces left in e-learning environments. In this project the LS2N has essentially worked with interns on the traces of the Open Classrooms MOOCs platform to predict the success or failure of students.

  Details

  Partners: LIG, LS2N, LIUM, LIRIS, IFé, STEF, LabSTICC, Open Classrooms
  Funding: French National Research Agency (ANR)
  Beginning: 2014-10-01
  End: 2018-12-31
  
• 

  IDEA: Immersive Data Exploration and Analysis

  Data Visualization Immersive Analytics Virtual Reality

  IDEA explores immersive analytics by proposing original representations, interaction modalities, and evaluations means. It focuses on multi-dimensional data and HMD-based virtual representations. This project has led to several prototypes, it now includes the PhD thesis of Adrien Fonnet.

  Details

  Partners: LS2N, Readi Design Lab
  Funding: West Creative Industry cluster
  Cofunding: University of Nantes / Ministry of Higher Education and Research
  Beginning: 2016-10-01
  End: 2020-08-31
  

  Related publications

  • Adrien Fonnet, Toinon Vigier, Grégoire Cliquet, Fabien Picarougne, Yannick Prié. (2018) Axes and Coordinate Systems Representations for Immersive Analytics of Multi-Dimensional Data in 4th International Symposium on Big Data Visual and Immersive Analytics BDVA 2018, Konstanz, Germany, October 2018 doi Show abstract Axes are the main components of coordinate systems representations. They play a critical role for the visual analysis of multi-dimensional data. However their representation seems to have always be considered self evident, with oriented lines crossing at an origin, completed with labels such as ticks and names. Such classical representation show limits when it comes 3D visualization and immersive analytic (IA), mainly because orthogonal projection of points on linear axes is hard in a 3d environment, and because the user can move therefore the axes can get out of his field of view. In this paper we propose a task-based definition of axes and coordinate systems representation, as well as a tentative design space for coordinates systems representation in immersion. We also present an exploratory user study we carried out to compare three grid-based representations of coordinate systems for multidimensional data analysis with 3D scatterplots.
  • Adrien Fonnet, Florian Melki, Yannick Prié, Fabien Picarougne, Grégoire Cliquet. (2018) Immersive Data Exploration and Analysis in Student Interaction Design Research conference, May 2018, Helsinki, Finland. Show abstract This paper belongs to the emerging immersive analytics research field. It presents a system for exploratory analysis of multi-dimensional data focusing on providing freedom of locomotion while remaining comfortable for long user session. The solution allows to investigate temporal traces through a 3D scatterplot using a cylindrical coordinate system. Users are free to move inside the tracked area either standing up or seated in a mobile chair. A limited user study (n=6) has been conducted to investigate if the designed solution was deemed appropriate for mobility and user comfort.
  • Grégoire Cliquet, Matthieu Perreira, Fabien Picarougne, Yannick Prié, Toinon Vigier. (2017) Towards HMD-based Immersive Analytics in Immersive analytics Workshop, IEEE VIS 2017, Phoenix, Arizona, United States. Show abstract Advances in 3D hardware and software have led to increasingly cheaper and simple-to-use immersive virtual reality systems that can provide real-time interactive 3D data representation. The immersive analytics field is developing as the newest avatar of 3D visual analyt- ics, which may relaunch the long enduring 2D vs 3D visualization debate. However, the terms of the debate have changed: leveraging 3D human perception within virtual environments is now easier, and the immersive quality of today’s rendering is sufficient enough for researchers to concentrate on testing and designing immersive data representation and interaction rather than on technological problems. In this position paper we propose a short historical perspective on the use of immersive technologies for visual analytics and on the 2D vs 3D debate. We stress out five principles that we think should be followed to explore the HMD-based visual analytics design space, before introducing ongoing work within the IDEA project.

  Additional illustrations

  
• 

  ITHACA: Interative Trace for Human Awareness in Collaborative Annotation

  Reflective Systems Technology-Enhanced Learning Trace-based Systems Video Annotations

  In this project carried with education scientists, we explored the use of traces in synchronous videoconference-based communication, mainly in the context of language teaching. We built two versions of the Visu tool, that was used in multiple settings..

  Details

  Partners: LIRIS, Lyon 2 University, ICAR Laoratory, TECFA Geneva
  Funding: French National Research Agency (ANR)
  Beginning: 2008-10-01
  End: 2011-12-31
  Website: https://projet.liris.cnrs.fr/ithaca/
  

  Related publications

  • Elise Lavoué, Gaëlle Molinari, Yannick Prié, Safè Khezami. (2015) Reflection-in-Action Markers for Reflection-on-Action in Computer-Supported Collaborative Learning Settings in Computers & Education, Volume 88, October 2015, Pages 129–142. doi Show abstract We describe an exploratory study on the use of markers set during a synchronous collaborative interaction (reflection-in-action) for later construction of reflection reports upon the collaboration that occurred (reflection-on-action). During two sessions, pairs of students used the Visu videoconferencing tool for synchronous interaction and marker setting (positive, negative or free) and then individual report building on the interaction (using markers or not). A quantitative descriptive analysis was conducted on the markers put in action, on their use to reflect on action and on the reflection categories of the sentences in these reports. Results show that the students (1) used the markers equally as a note-taking and reflection means during the interaction, (2) used mainly positive markers both to reflect in and on action; (3) paid more attention in identifying what worked in their interaction (conservative direction) rather than in planning on how to improve their group work (progressive direction); (4) used mainly their own markers to reflect on action, with an increase in the use of their partners' markers in the second reflection reports; (5) reflected mainly on their partner in the first reflection reports and more on themselves in the second reports to justify themselves and to express their satisfaction.
  • Elise Lavoué, Gaëlle Molinari, Safè Khezamy, Yannick Prié. (2013) How do Students Use Socio-Emotional Markers for Self-Reflection on their Group Work in CSCL Settings? A Study with Visu: a Synchronous and Delayed Reflection Tool in 10th International Conference on Computer Supported Collaborative Learning, Madison, USA, 2013 [AR:39%] Show abstract This paper describes an exploratory study on the use of reflective markers set during synchronous collaborative learning sessions (reflection in action) for later construction of self-reflection reports upon the collaboration that occurred (reflection on action). During 2 sessions, students used the Visu tool for interaction and marker setting (positive, negative, free) and then report building on the interaction (using markers or not). A quantitative descriptive analysis has been conducted on the markers used and on the reflective categories of the sentences in the reports. Results show that students (1) paid more attention in repairing their relationship than reflecting on learning and task goals; (2) used mainly positive markers to both reflect in and on action; (3) used more their partner's markers in the second reports; (4) reflected more on themselves in the second reports to justify successes and failures, and to express satisfaction.
  • Elise Lavoué, Safè Khezami, Gaëlle Molinari, Yannick Prié. (2013) The Visu Reflection Tool for Socio-Emotional Awareness in CSCL situations in Workshop on Tools and Technologies for Emotion Awareness in Computer-Mediated Collaboration and Learning. Alpine Rendez-Vous (ARV) 2013, Villars-de-Lans, Jan 2013. Show abstract An exploratory study has been conducted in which 12 students in Bachelor of Science in Psychology were asked to use the Visu reflection tool during Computer-Supported Collaborative Learning (CSCL) situations. Visu is a web videoconferencing platform that allows participants to put reflective markers during their collaborative learning activity, and to review the traces of their synchronous collaboration later. In this study, co-learners used two types of markers: (1) free markers and (2) socio-emotional markers to express either negative or positive feelings about the way they collaborate together. Our contribution is related to the first focus of the workshop: emotion awareness in CSCL. In line with this focus, our main questions are as follows: (1) How can the Visu tool help learners express and share their feelings about collaboration? How does this affect the way they interact and learn together? (2) How do they use the Visu markers after the collaboration to self-reflect on their group processes? In this paper, we first describe the Visu platform. We then present a study we have carried out to answer the questions presented above. We finally conclude with the contributions of our research to the workshop topic.
  • Nicolas Guichon, Mireille Bétrancourt, Yannick Prié. (2012) Managing written and oral negative feedback in a synchronous online teaching situation in Computer-Assisted Language Learning, 25(2):181-197, 2012. doi Show abstract This case study focuses on the feedback that is provided by tutors to learners in the course of synchronous online teaching. More specifically, we study how trainee tutors used the affordances of Visu, an experimental web videoconferencing system, to provide negative feedback. Visu features classical functionalities such as video and chat, and it also offers a unique marking tool that allows tutors to take time-coded notes during the online interactions for later pedagogical remediation. Our study shows that tutors mainly use verbal and chat feedback, with significant inter-individual variability, and that tutors who provide verbal feedback are more likely to use markers. Marking takes time because of the dual task that it entails for the tutor. Idiosyncratic strategies in the use of markers are evidenced. These results clearly show the value of markers for negative feedback, signal the need for their explicitness, and also call for an evolution of the Visu interface so that tutors can better negotiate the task of online tutoring and the pedagogical stance they have to take on in their interactions with the learners.
  • Mireille Bétrancourt, Nicolas Guichon, Yannick Prié. (2011) Assessing the use of a Trace-Based Synchronous Tool for distant language tutoring in 9th International Conference on Computer Supported Collaborative Learning. Hong Kong. pp 486-493. Jul 2011. [AR:36%] Show abstract This article presents a pilot study carried out to investigate the potential of a functionality marker setting, included in a synchronous collaborative videoconferencing platform (VISU). Markers, supported by a trace-based system, are designed to facilitate tutors' activity. They provide tutors with (1) the possibility of annotating their distant learners' learning activity, and (2) information pertaining to their own behavior during pedagogical interaction, which can potentially enhance their professional performance as online language tutors. This study concentrates on the marker-based traces of eight language tutors collected in the course of pedagogical interactions with their distant learners during a seven-week transnational collaborative project. It presents both quantitative and qualitative analyses of the use of markers during synchronous language teaching sessions and assesses the utility and usability of such a functionality for language tutoring in order to inform future design and training.

  Related software

  • GVMT
  • VISU
• 

  Meta-éducation

  Learning Analytics Trace-based Activity Analysis

  A project hosted in Lyon at LIRIS on traces and learning environments. We worked on conceptual graph building activity, designing dashboard for teachers to appreciate students behavioral and cognitive engagement.

  Details

  Partners: LIRIS, ERDENET, IRI Centre Pompidou, ITOP
  Funding: Investissement d'avenir - E-éducation
  Beginning: 2014-02-01
  End: 2017-12-31
  

  Related publications

  • Rubiela Carrillo, Yannick Prié, Leslie Guillaume, Elise Lavoué. (2018) Suivre l’engagement des apprenants dans l’activité de construction de cartes mentales in Sciences et Technologies de l’Information et de la Communication pour l’Éducation et la Formation (STICEF). 25(1). Show abstract Notre travail a pour objectif d’aider les enseignants à suivre l’engagement de leurs apprenants pendant des activités de construction de cartes mentales. Les cartes mentales sont utilisées pour aider les apprenants à organiser les contenus étudiés en classe. Étant donnée la complexité des cartes, les enseignants ont besoin d’outils qui assistent la compréhension des cartes produites et de leur processus de construction. Nous avons défini avec des enseignants un ensemble d’indicateurs d’engagement, reposant sur les traces d’interaction des apprenants. Ces indicateurs ont été implémentés dans un tableau de bord à destination des enseignants. Cet article présente la première version du tableau de bord implémenté, sa conception, son implémentation, et les résultats d’une évaluation sur les aspects d’utilisabilité et utilité.
  • Rubiela Carrillo, Clément Renaud, Yannick Prié, Élise Lavoué. (2017) Dashboard for monitoring student engagement in mind mapping activities in IEEE 17th International Conference on Advanced Learning Technologies (ICALT), Jul 2017, pp. 433-437, Timisoara, Romania. doi Show abstract Our work aims at helping teachers to monitor learners' engagement during mind mapping activities. The use of mind maps helps students to elaborate cognitive learning strategies like creating and organizing contents. To assess the quality of these mind maps, teachers need tools to understand students' choices and strategies when constructing their mind maps. We have defined in collaboration with teachers a set of indicators based on learners' interaction traces. Those indicators have been implemented and integrated in a dashboard dedicated to teachers. In this paper, we introduce a first version of the dashboard: its design, implementation, and the results of its evaluation conducted during interviews with real teachers.
  • Rubiela Carrillo Rozo, Clément Renaud, Yannick Prié and Elise Lavoué. (2017) Tableau de bord pour le suivi de l'engagement des apprenants lors de la construction de cartes mentales in 8ème Conférence sur les Environnements Informatiques pour l'Apprentissage Humain (EIAH 2017), Jun 2017, Strasbourg, France. pp.113-124, 2017. Show abstract Les cartes mentales permettent aux apprenants de mettre en place des stratégies cognitives d'apprentissage telles que l'élaboration et l'organisation des contenus étudiés en classe. Étant donné leur complexité, les enseignants ont besoin d'outils pour les aider à comprendre les cartes mentales produites et leurs processus de construction. Notre objectif est de permettre aux enseignants de suivre l'engagement des apprenants lors de ce type d'activité. Nous avons défini avec des enseignants des indicateurs basés sur les traces d'interaction des élèves permettant de suivre et d'expliquer les modalités et les choix de construction des cartes mentales. Ces indicateurs ont été implémentés dans un tableau de bord à l'usage des enseignants, que nous avons commencé à évaluer au cours d'une série d'entretiens. Nous présentons dans ce papier le prototype du tableau de bord ainsi que les premiers résultats de cette évaluation.
  • Rubiela Carrillo, Élise Lavoué, Yannick Prié. (2016) Towards Qualitative Insights for Visualizing Student Engagement in Web-based Learning Environments in Proceedings of the 25th International Conference Companion on World Wide Web, pp. 893-898 Show abstract Learning Sciences argue that student engagement is composed of behavioral, motivational and cognitive dimensions. Many proposals in Learning Analytics have provided teachers with quantitative indicators focusing only on students' behaviors, such as the number and the duration of their actions with the learning environment. In this paper, we propose visual representations of cognitive indicators to add explanatory elements to behavioral indicators. We describe our general architecture for collecting and aggregating data used to build the proposed visualizations. We illustrate the use of these indicators in various pedagogical scenarios oriented towards supporting teachers in students' actions and performances understanding.

  Additional comments

  PhD thesis of Rubiela Carillo
• 

  Musette: Modelling USEs and Tasks for Tracing Experience

  Knowledge Engineering Trace-based Activity Analysis Trace-based Systems

  A first tentative towards modelling information systems use experience as traces, going from Case-Based reasoning to Trace-based reasoning, with a modelled approach to traces, and episodes querying to foster user assistance and activity analysis.

  Details

  Partners: false
  Funding: various
  Beginning: 2000-10-01
  End: 2005-12-31
  

  Related publications

  • Julien Laflaquière, Yannick Prié. (2009) Musette : Modéliser les Usages et les Tâches pour Tracer l'Expérience in Réutilisation de l'expérience : modèles et applications, Hermès, 2009. Show abstract L’objectif général de ce chapitre est de présenter l’approche Musette – Modéliser les USages et les Tâches pour Tracer l’Expérience – développée au LIRIS depuis plusieurs années.
  • Pierre-Antoine Champin, Yannick Prié and Alain Mille. (2004) MUSETTE: a Framework for Knowledge Capture from Experience in short paper in EGC'04, Clermont Ferrand, jan. 2004 Show abstract Nous présentons dans cet article une nouvelle approche de modélisation de l'expérience d'utilisation d'un système informatique, avec pour objectif de réutiliser cette expérience en contexte pour assister l'utilisateur à effectuer sa tâche. Le modèle se base sur la capture d'une trace d'utilisation conforme à un modèle d'utilisation général, lequel décrit les objets et les relations manipulés par l'utilisateur du système informatique visé. Cette trace primitive peut être considérée comme une base de connaissances neutre par rapport à la tâche, qui peut être analysée a posteriori à l'aide de signatures de tâche expliquées permettant d'y localiser des épisodes signifiants qui pourront être réutilisés par des assistants logiciels comme connaissances contextualisées. Quatre scénarios illustrent cette approche. (paper in english)
  • Julien Laflaquière, Yannick Prié. (2003) Modélisation d'utilisation de système pour une assistance à base de trace : une application de MUSETTE à la tâche de veille documentaire in Workshop Traces, interactions, co-constructions collectives et relations à la cognition. AS CoMETE, Paris, 19 déc. 2003. Show abstract La complexification des activités de haut niveau s'appuyant sur des outils informatiques et la généricité grandissante de ces mêmes outils, posent la question de la (re)définition de la place et du rôle de l'assistance relativement à ces activités et à ces outils. Cet article présente une approche de modélisation d'utilisation (MUSETTE), destinée à la création d'une assistance basée sur la réutilisation d'expérience, ainsi que sa mise en ½uvre dans le domaine particulier de la veille. Ce travail a débuté lors d'un stage de DEA au LIRIS. La réutilisation d'expérience dans une assistance visant spécifiquement une activité de haut niveau est une réponse possible, au problème de (re)définition de l'assistance, se démarquant d'approches plus « classiques ». L'approche globale MUSETTE et trois étapes clefs de modélisation sont développées, les perspectives offertes par ce travail sont discutées.
  • Pierre-Antoine Champin and Yannick Prié. (2003) Musette: uses-based annotation for the Semantic Web in Annotation for the Semantic Web, S. Handschuh (ed.), IOS Press, Amsterdam, pp. 180-190, 2003. Show abstract In Tim Berners-Lee’s vision of the Semantic Web, software agents must be able to retrieve knowledge relevant to the end-user’s task, despite the heterogeneity and scale issues which are inherent to the web. Annotating web resources is considered a promising way to achieve this vision. We argue in this chapter that every actual use of a resource may provide useful knowledge about that resource and therefore should annotate it. We propose a framework for taking uses into account and annotating resources with their uses. This framework is the result of a generalization effort in our team after a number of projects focusing on modeling uses and experience. We believe that this framework is relevant to the Semantic Web where uses-based annotation may prove valuable.
  • Elöd Egyed-Zsigmond, Alain Mille, Yannick Prié́. (2003) Club (Trèfle) : a use trace model in 5th International Conference on Case-Based Reasoning, LNCS, volume 2689, Trondheim, NO, pp. 146–160. [AR:21%] doi Show abstract In this paper we present a use trace model which allows the collection and reuse of user experience, based on a homogeneous and interconnected representation of users, procedures and objects. All these notions form a connected labeled directed graph containing highly connected and explained use traces. This model enables assistance in non trivial, creativity requiring situations. Our model uses the Case Based Reasoning (CBR) paradigm in order to reuse experience. After a formal description of the model we discuss how it can serve to capitalize and re-use experience.
  • Pierre-Antoine Champin, Yannick Prié. (2002) Modéliser l'utilisateur ou l'utilisation ? in Document Virtuel Personnalisable 2002, Brest, july 2002, pp. 97-102. Show abstract Les modèles d’utilisation nous semblent au moins aussi légitimes que les modèles d’utilisateurs pour piloter la personnalisation. En effet, personnaliser un document virtuel revient à influer sur ses modes d’utilisation pour faciliter la tâche de l’utilisateur, et donc nécessite de modéliser son utilisation. Or d’une part, il n’existe pas de relation bi-univoque entre utilisation et utilisateur : le même utilisateur a souvent plusieurs tâches à effectuer, pour lesquelles ses préférences peuvent varier grandement ; inversement, des utilisateurs effectuant les mêmes tâches devraient pouvoir partager des traits de personnalisation — dans le contexte de cette tâche. D’autre part, modéliser l’utilisation à partir d’un modèle de l’utilisateur implique un changement d’objet de modélisation (utilisation/utilisateur), et donc l’utilisation de méta-connaissances, souvent introduites a priori, sur les correspondances entre ces modèles, et dont l’acuité conditionne la pertinence de la personnalisation proposée. À l’inverse, personnaliser à partir d’un modèle d’utilisation évite d’avoir à changer d’objet de modélisation. Nous développons l’idée que les traces d’utilisation devraient être prises en compte pour la personnalisation de documents virtuels, et nous en proposons un modèle issu de nos travaux récents. Nous donnons également quelques exemples de réutilisation des usages pour la personnalisation de documents virtuels.
  • Pierre-Antoine Champin, Yannick Prié, Alain Mille. (2002) Conteneurs de connaissances : une approche fondée sur les usages pour le Web sémantique in Reconnaissances de Formes et Intelligence Artificielle, 2002, Angers, 10 pp. Show abstract Dans la vision du Web Sémantique proposée par Tim Berners-Lee, les agents logiciels doivent être capables de retrouver des connaissances pertinentes pour la tâche d'un utilisateur. Nous proposons dans cet article un cadre général pour prendre en compte les usages, centré sur la notion de conteneur de connaissances, et sur des modèles d'utilisation de ces conteneurs. Nous présentons ensuite deux projets dans lesquels les auteurs sont impliqués, où ce cadre est implanté : ARDECO, qui vise à assister la réutilisation en CAO, et RECIS, dont le sujet est l'indexation du contenu de documents audiovisuels. Nous concluons que toute utilisation d'une ressource devient annotation de cette ressource, et constitue ainsi un nouveau conteneur de connaissances.
  • Pierre-Antoine Champin, Yannick Prié, Alain Mille. (2001) Annotating with uses: a promising way to the semantic web in Workshop on Knowledge Markup and Semantic Annotation, K-CAP 2001, S. Handschuh, R. Dieng, S. Staab (Eds.), Victoria, oct. 2001, pp. 79-86. Show abstract In Tim Berners-Lee's vision of the Semantic Web, software agents must be able to retrieve knowledge relevant to the end-user's task, despite the heterogeneity and scale issues which are inherent to the web. We argue in this paper that users' tasks are themselves quite numerous and various, so that they can not be all predicted, and hence implemented. Therefore, considering actual uses may be an efficient way of fulfilling users' unpredicted needs. We propose a general approach for taking uses into account. Then we present how this approach was applied in two projects involving the authors: ARDECO, aiming at assisting the reuse of CAD documents, and RECIS, aiming at content-based retrieval of audiovisual documents. After a comparison of this approach with other works related to the Semantic Web, we conclude that any use of a resource should annotate this resource, and that such annotations, as they provide additional knowledge about the resource, should become part of the Semantic Web as resources of their own.
  • Yannick Prié, Alain Mille. (2000) Reuse of knowledge containers: a "local semantics" approach in Workshop on Flexible Strategies for Maintaining Knowledge Containers, 14th European Conference on Artificial Intelligence ECAI 2000, Mirjam Minor (Ed.), Berlin, Aug. 2000, pp.38-45. Show abstract In this paper we present two ways of considering reuse of systems of terms as knowledge containers. The rigorous approach is concerned with strict organization of terms and inferences, while the lazy one is based on terms used in various different contexts, without strictly predefined semantics. We argue that it is necessary to handle both advantages of such approaches in a median one, and present the AI-Strata model for description and use of audiovisual documents. This model has interesting properties that could help use and reuse term systems in different tasks and contexts.

  Additional illustrations

  
• 

  OPEVA: OutPatient Experience in Virtual Reality for Architectural Design

  Patient Experience Experience analysis and modelling Virtual Reality

  A project where we compared the experiences of patients in VR and in reality. We designed and implemented a virtual reality environment replicating an actual ambulatory pathway. We used elicitation interviews to collect the experiences of actual patients in both our virtual prototype and the ambulatory pathway, that we modeled and compared. Results show that patients live comparable experiences along various dimensions of experience such as thought, emotion, sensation, social and sensory perception.

  Details

  Partners: LS2N, AIA Life Designer, Clinique Jules Verne
  Funding: West Creative Industry (France)
  Beginning: 2021-11-01
  End: 2022-08-31
  

  Related publications

  • . () in
• 

  PAPM: Progressive Analytics and Pattern Mining

  Data Visualization Interpretation Systems Progressive analytics

  The project concentrates on the main difficulty of interactive mining, which is not post-mining interaction, nor pre-interaction mining, rather interaction and mining occurring at the same time. We focus on interaction for progressive analytics, our application is in progressive pattern mining of frequent event sequences of user activity traces.

  Details

  Partners: LS2N
  Funding: PhD funding
  Beginning: 2015-10-01
  End: 2018-12-31
  

  Related publications

  • Vincent Raveneau, Julien Blanchard, Yannick Prié. (2018) Progressive sequential pattern mining: steerable visual exploration of patterns with PPMT in Visualization in Data Science (VDS at IEEE VIS 2018), Berlin, Germany, 2018. [AR:30%] Show abstract The progressive visual analytics (PVA) paradigm has been proposed to describe visual analytics systems whose main goal is to reach a thorough coupling between the analyst and her system by getting rid of waiting periods classically encountered during data processing. PVA systems use algorithms that both provide intermediate results throughout their execution, and are steerable by the analyst to change the strategy used to perform the remaining computation. Our focus is on progressive sequential pattern mining, as in the seminal work of Stolper et al. (2014). Here we go further mainly by considering the temporal nature of patterns related to their occurrences. We propose a pattern-oriented data model, a pattern analysis task model, and guidelines for designing progressive pattern mining algorithms. We introduce PPMT, a tool to support an analyst progressively explore activity traces, based on a modification of the GSP algorithm. We evaluate our proposal on the technical performances of our progressive algorithm, and on the effect of steering on analysts' performances.
  • Vincent Raveneau, Julien Blanchard, Yannick Prié. (2016) Towards an open-source tool for pattern-based progressive analytics on interaction traces in Workshop "Event Event" at IEEE VIS 2016, Baltimore. 3p. Show abstract Due to the ever increasing use of data analysis tools, their performance has become an important scientific question, notably regarding the tools’ ability to assist the analyst using them. We present our first proposals toward building a data analysis tool allowing an analyst to interact with his pattern mining algorithm, while working with interaction traces. First, we present an architecture for such a system, based on the ideas of progressive analytics. We then present our work toward a first implementation of this architecture
  • Katy Tcha-Tokey, Toinon Vigier, Yannick Prié. (2016) Towards an Affordance-Based Approach for Scenario Modeling and Experience Assessment in VR in IEEE VIS 2016, Baltimore. oct. 2016. 2p (Poster) Show abstract Due to the ever increasing use of data analysis tools, their performance has become an important scientific question, notably regarding the tools’ ability to assist the analyst using them. We present our first proposals toward building a data analysis tool allowing an analyst to interact with his data mining algorithm, while working with interaction traces. First, we present an architecture for such a system, based on the ideas of progressive analytics. We then present the challenges that will need to be faced in future work.

  Related software

  • PPMT

  Additional illustrations

  
• 

  PEEL: Patient Experience Evaluation Lab

  Experience analysis and modelling Patient Experience

  The interdisciplinary association of researchers, design companies, health institutions to work on methodologies to analyze patient experience, design experience-based interventions, and assess these interventions.

  Details

  Partners: LS2N, ENDA (Care Design Lab), Sensipode, AIA, MOSWO, CHU Nantes, CPAM...
  Funding: RFI Ouest Industries Créatives.
  Cofunding: Various compagnies
  Beginning: 2021-06-01
  End: 2023-06-30
  Website: https://www.youtube.com/watch?v=CEHsENnJYv4&list=PLbNT3anDykcrGAU7LbI3M8Jv-ukECNDmo
  
• 

  PROMETEA: OutPatient Experience in Virtual Reality for Architectural Design

  VR and Psychotherapy Virtual Reality

  A collaboration within Nantes University with clinical psychologists to design an build an ecological VR environment dedicated to prospective memory testing. The system will be evaluated in 2024.

  Details

  Partners: LS2N, LPPL, MIP
  Funding: ELIT Cluster
  Beginning: 2023-04-01
  End:
  
• 

  Reading Analytics

  Learning Analytics Trace-based Activity Analysis

  Reading analytics are Learning Analytics focusing on learners' content consumption traces, for authors to gain insights on the way they can improve their courses. We mainly worked on the traces of OpenClassrooms Mooc courses, had some results on reading sessions detection, and built a dashboard for teachers.

  Details

  Partners: LS2N, LIRIS, CERIST
  Funding: CERIST, Alger
  Beginning: 2012-10-01
  End:
  

  Related publications

  • . () in
  • Madjid Sadallah, Benoît Encelle, Azze-Eddine Maredj, Yannick Prié. (2015) Towards Reading Session-based indicators in Educational Reading Analytics in 10th European Conference on Technology Enhanced Learning, Toledo, Spain, sept. 2015, Lecture Notes in Computer Science, vol 9307. pp. 297-310 [AR:15%] doi Show abstract It is a challenging task to identify eLearning courses parts that have to be revised to best suit learners’ requirements. Reading being one of the most salient learning activities, one way of doing so is to study how learners consume courses. We intend to support course authors (e.g. teachers) during courses revision by providing them with reading indicators. We use the concept of reading session to denote a learner’s active reading period, and we provide several associated reading indicators. In our server-side approach, reading sessions and indicators are calculated using web server logs. We evaluate the relevance of our proposals using logs from a major French eLearning platform. Results are promising: calculated reading sessions are theoretically more precise than other best applicable approaches, and course authors consider suggested indicators to be appropriate to courses revision. Using reading sessions and associated indicators could facilitate authors’ work of course reengineering.
  • Madjid Sadallah, Benoît Encelle, Azze-Eddine Maredj, Yannick Prié. (2013) A Framework for Usage-based Document Reengineering in Proceedings of the 2013 ACM symposium on Document engineering DocEng '13, pp. 99-102, 2013. doi Show abstract This ongoing work investigates usage-based document reengineering as a means to support authors in modifying their documents. Document usages (i.e. usage feedbacks) cover readers' explicit annotations and their reading traces. We first describe a conceptual framework with various levels of assistance for document reengineering: indications on reading, problem detection, reconception suggestions and automatic reconception propositions, taking our example in e-learning document management. We then present a technical framework for usage-based document reengineering and its associated models for documents, annotations and traces representation.

  Related software

  Additional comments

  This project mainly consists in Madjid Sadallah’s PhD thesis.

  Additional illustrations

  
• 

  Redoc: Activity traces redocumentation

  Document Engineering Knowledge Engineering Reflective Systems Trace-based Systems

  The redocumentation process of computer-mediated activity is as a narrative construction that ties together contents of activity traces and users’ knowledge of their own activities within exchangeable documents. We built a system for redocumenting the traces of exploring a 19th century newspaper digital library.

  Details

  Partners: LIRIS
  Funding: University of Constantine
  Beginning: 2008-10-01
  End: 2012-10-à1
  

  Related publications

  • Leila Yahiaoui, Yannick Prié, Zizette Boufaïda. (2013) Du traçage de l’activité informatique à sa redocumentation en texte in Technique et science informatique, n°1/2013 pp. 7-40, 2013. doi Show abstract Dans ce travail, nous présentons une approche générique pour le processus de redocumentation de l’activité médiée informatiquement. Cette approche combine le contenu de traces de l’activité et les connaissances des utilisateurs dans des formes narratives textuelles, facilement échangeables, décrivant l’activité de façon personnalisée pour les utilisateurs. Elle utilise des modèles formels pour les traces et les documents produits, ainsi que les principes de la théorie de la structure rhétorique(RST). Elle soutient la redocumentation à travers deux phases : une phase automatique pour générer un document texte initial et une phase interactive pour personnaliser ce document. Nous présentons notre outil de redocumentation ActRedoc, une implémentation de notre approche, ainsi que les résultats de sa première évaluation.
  • Leila Yahiaoui, Yannick Prié, Zizette Boufaïda, Pierre-Antoine Champin. (2012) Redocumenting computer mediated activity from its traces: a model-based approach for narrative construction in Journal of Digital Information, vol. 12. n°3. 2011. Show abstract Our activities are becoming more and more computer-mediated. For documenting these activities, it is no longer sufficient to automatically record their traces. In this paper we introduce the redocumentation process of computer-mediated activity as a narrative construction that ties together the content of activity traces and the users’ knowledge in describing their activities in new easily exchangeable documents. We present a generic semi-automatic approach for this process, which is based on rhetorical structure theory. This approach uses formal models for process input and output, and handles the process through two main phases: an automatic phase to generate a fragmented document from traces as a first description of the activity and an interactive phase to allow the user to tailor this first description according to his particular needs and choices. We also present ActRedoc, a tool developed for text-based redocumentation, for which a first evaluation was conducted.
  • Leila Yahiaoui, Zizette Boufaïda, Yannick Prié. (2010) Un outil générique pour la redocumentation en texte d'une activité médiée informatiquement in 1ére conférence Maghrébine sur l'Extraction et la Gestion des Connaissances (EGC-M 2010), Alger, Algérie. pp. 73-84. 2010. Show abstract Dans ce papier nous présentons un outil générique de redocumentation (semi-automatique) en texte d’une activité médiée informatiquement. Il s’agit d’un outil pouvant assister son utilisateur à exploiter les traces générées automatiquement, après leur modélisation sémantique, pour générer un nouveau document personnalisé décrivant l’activité tracée de façon plus appropriée. Cela consiste à réarranger le contenu des traces, à filtrer ce contenu, à reformuler certaines de ses parties et à ajouter des informations sur le contexte de production des traces. De plus, une forme d’appropriation peut être appliquée au document texte final. Il s’agit d’un document facilement échangeable, que ce soit à des fins d’analyse ou à des fins d’assistance. L’outil de redocumentation développé exploite différentes technologies du Web Sémantique. Il est basé sur les principes de la théorie de la structure rhétorique et ceux de la composition de documents personnalisés.
  • Leila Yahiaoui, Yannick Prié, Zizette Boufaïda. (2009) The redocumentation process of computer mediated activity traces: A general framework in 20th ACM Conference on Hypertext and Hypermedia (HT 2009), Torini, Italy, pp. 363-364, jul 2009 (Poster) Show abstract The digital world enables the creation of personalized documents. In this paper we are interested in describing a computer mediated activity by a person throughout a semi-automatic redocumentation process. This process uses traces generated automatically, during a user-system interaction, to assist a person in producing a personalized document describing the traced activity. To support that, a general framework for an authoring tool is proposed through two main phases. During the first phase, an automatic and parameterized transformation is applied on the input activity trace to generate a fragmented document. Each fragment describes one or many observed elements of the modeled trace and relations between fragments are deduced from relations between these elements. The second phase consists in interactive transformations on the intermediate produced document until getting the final hypermedia document. Our authoring tool uses composition of personalized document issues and RST principals to interpret user's choices and to maintain the coherence of the produced document.
  • Leila Yahiaoui, Yannick Prié, Zizette Boufaïda. (2008) Redocumentation des traces d'activité médiée informatiquement dans le cadre des transactions communicationnelles in 19es Journées Francophones d'Ingénierie des Connaissances (IC 2008), Nancy, France. pp.197-209, Jun 2008 Show abstract La documentation de l'activité médiée informatiquement est souvent assurée par l'ensemble des fichiers traces générés suite à l'interaction de l'utilisateur avec le système informatique utilisé. Or ces documents traces manquent de lisibilité, d'intelligibilité et de portée sociale. Dans cet article nous présentons une nouvelle approche de redocumentation de l'activité médiée informatiquement, dans le cadre de la théorie des transactions communicationnelles. Cette redocumentation se base sur l'exploitation des traces d'interaction dans le but de créer un nouveau document numérique décrivant cette activité. Ce document, personnalisé par l'utilisateur ou d'autres personnes, offre aux traces un statut documentaire plus renforcé, dans le but de mémoriser l'activité de l'utilisateur tracé, d'interpréter son comportement ou d'assister d'autres utilisateurs ayant une activité similaire. Nous présentons également un scénario de notre approche ainsi que certains problèmes posés.

  Additional comments

  PhD thesis of Leila Yahiaoui, between University of Lyon and University of Constantine, Algeria.

  Additional illustrations

  
• 

  Revistim-X: Virtual Reality and Brain Stimulation: an experiential approach

  Experience analysis and modelling VR and Psychotherapy Virtual Reality

  This was the first feasability and acceptability study of the association of cerebral stimulation and virtual reality for treatment in psychotherapy. We designed and built an environment related to acrophobia, and analyzed patients' experiences using mixed methods. The project continues with the development of the EVEREST VR environment.

  Details

  Partners: LS2N, Sphere, IAC 18
  Funding: Université de Nantes
  Cofunding: CHU Nantes, Fondations Université de Nantes, Fondation CHU Nantes
  Beginning: 2017-10-01
  End: 2018-09-31
  

  Related publications

  • Toinon Vigier, Yannick Prié, Florian Melki, Matthieu Perreira Da Silva, Fabien Picarougne, Samuel Bulteau, Olivier Charlet. (2018) New VR Experiences for the Personalization of Care in Psychiatry: the Example of ReViSTIM Project in Care TVX Workshop at ACM TVX 2018, June 2018, Seoul, South Korea. doi Show abstract This paper presents a new VR platform in hospital environment for the study of mental diseases. The Revistim platform consists in several hardware devices and software tools for the immersion of subjects, the collection of various subjective, physiological and behavioral data as well as the use of new psychotherapeutic systems. The platform is used both for research in neuro-psychiatry (diagnosis, rehabilitation and treatment) and user experience modeling. Considering care, the main objective is to propose personalized care in psychiatry through the assessment of the patient state, the priorisation of targeted interventions, and an efficient evaluation of the VR therapy on the different dimensions of the disease.
  • Samuel Bulteau, Andrew Laurin, Kalyane Bach-Ngohou, Morgane Péré, Marie-Anne Vibet, Jean-Benoit Hardouin, Véronique Sebille, Lydie Lagalice, Élodie Faurel-Paul, Didier Acier, Thomas Rabeyron, Valéry-Pierre Riche, Anne Sauvaget, Florian Melki, Toinon Vigier, Matthieu Perreira Da Silva, Olivier Charlet, Yannick Prié. (2022) Feasibility of combining transcranial direct current stimulation and full embodied virtual reality for visual height intolerance: a double-blind randomized controlled study in Journal of Clinical Medecine 2022, 11(2), 345; doi Show abstract Background: Transcranial direct current stimulation (tDCS) and virtual reality exposure therapy (VRET) may be used to reduce anxiety in visual height intolerance (vHI). Objective/Hypothesis: Our study aims to in-vestigate the feasibility of the association between wireless tDCS and full embodied 360° VRET to reduce anxiety in vHI. Methods: We carried out a randomized, double-blind, controlled study to associate VRET with tDCS by targeting the ventromedial prefrontal cortex in 28 participants. Primary outcomes were the sense of presence level and the tolerability. Secondary outcomes were the anxiety severity and the salivary cortisol concentration. Results: We confirmed the feasibility of the association tDCS+VRET with a good sense of presence without severe adverse effect. tDCS was not superior to sham to reduce anxiety. We observed in two groups a significant reduction of anxiety after stimulations but a significant increase in anxiety at each level of difficulty. Cortisol concentration significantly decreased in the sham group but remained stable in the tDCS group. Conclusion: Our study confirms the feasibility of the association between wireless tDCS and full embodied VRET with a validated environment to induce anxiety. Effects of VRET seem to be stronger than tDCS effects.

  Additional comments

  Revistim-X clinical trial recording.

  Additional illustrations

  
• 

  Revistim-xX: Virtual Reality and Brain Stimulation: an experiential approach

  Experience analysis and modelling VR and Psychotherapy Virtual Reality

  This second study on the association of cerebral stimulation and virtual reality for treatment in psychotherapy deals with rumination, a dimension of depression. In this study we collect data with questionnaires, EEG, eye-tracking, and micro-phenomenology.

  Details

  Partners: LS2N, Sphere, IAC 18
  Funding: partly funded by RFI Atlanstic 2020
  Cofunding:
  Beginning: 2018-03-01
  End: 2019-09-31
  

  Additional comments

  The “zen” virtual reality environment we use is designed by Healhty Mind

  Revistim-xX clinical trial recording.
• 

  TBS: Trace-Based Systems

  Knowledge Engineering Trace-based Activity Analysis Trace-based Systems

  The main research axis of the SILEX team at LIRIS around Modelled-trace-based systems for activity analysis and user reflexivity, before it was called learning analytics. General knowledge-based modelling of Trace-bases systems, with a semantic web flavour, and lors of applications in various educational context

  Details

  Partners: A lot
  Funding: Various projects
  Beginning: 2007-01-01
  End: 2013-01-01
  

  Related publications

  • Pierre-Antoine Champin, Alain Mille, Yannick Prié. (2013) Les traces numériques comme objets de premier niveau : une approche par les traces modélisées in Intellectica, 2013-1 n°59, pp. 171-204, 2013. Show abstract Toute activité médiée par l’outil informatique laisse des traces numériques qui constituent en puissance des connaissances utiles, pour cette activité ou des activités futures. L’approche proposée dans cet article vise à exploiter effectivement ces connaissances dans des systèmes informatiques dits à base de m-traces. Cette approche consiste à proposer une modélisation a priori des traces, et à capturer leurs diverses interprétations possibles dans des transformations reproductibles. Nous proposons un méta-modèle général des systèmes à base de traces modélisées, et illustrons sa mise en œuvre dans deux applications. Nous discutons des défis que pose la conception de tels systèmes, notamment pour la prise en compte pluridisciplinaire de l’activité humaine tracée.
  • Olivier Curé, Yannick Prié, Pierre-Antoine Champin. (2012) Extending Knowledge-Based Systems to Reason Over Interaction Traces in The 18th International Conference on Knowledge Engineering and Knowledge Management EKAW 2012, Galway, oct 2012, 10p [AR:29%] Show abstract This paper presents a trace-based framework for assisting personal- ization and enrichment of end-user experience in an application. We propose a modular ontology-based architecture, to provide semantics for interaction traces, observed elements and their associated objects, and we extend existing inference services, with a declarative and generic approach, in order to reason with those interaction traces. We present the architecture of our framework and its reasoning levels, provide a proof of concept on a medical Web application, and emphasize that different kinds of actors can benefit from the supported inferences.
  • Damien Clauzel, Karim Sehaba, Yannick Prié. (2011) Enhancing synchronous collaboration by using interactive visualisation of modelled traces in Simulation Modelling Practice and Theory Journal. 19(1). pp. 84-97, 2010. doi Show abstract This article addresses issues related to traces modelling for formally describing human interactions of people engaged in a synchronous collaborative learning activity. The objective is to propose models and tools for representing, transforming, sharing and visualising traces of users’ experiences. The traces here represent the users’ activities in their interactions with the learning platform. Our proposition is based on reflexive learning defined as the ability to interact with the situation, in order to meet one’s own limitations. This work takes place in the ITHACA project which aims at developing an on-line learning platform that uses interaction traces as knowledge sources on, and for, the learners’ learning as individuals or groups. In this paper, we propose a general framework for trace management and sharing, a generic model of synchronous collaborative activity based on the notion of interaction modes that we specialized for whiteboard sharing and text chatting, and a conceptual framework for modelling the exploitation of modelled traces, in particular for interactive visualisation on the user side. This article extends our previous work [1] on the instrumented prototypes, by presenting our theorisation of the interactive visualisation of modelled traces.
  • Olivier Champalle, Karim Sehaba, Alain Mille, Yannick Prié. (2011) A framework for observation and analysis of learners’ behavior in a full-scope simulator of a nuclear power plant - Approach based on modelled traces in ICALT 2011 - The 11th IEEE International Conference on Advanced Learning Technologies</span>, Athens, Georgia, USA. July 2011. (Poster) doi Show abstract Our work deals with the subject of activity observation in full scope simulators. In this paper we present a Modelled Trace approach for designing a methodological framework and associated tools to manage the observation process and the corresponding analysis activity.
  • Olivier Champalle, Karim Sehaba, Alain Mille, Yannick Prié. (2011) Des transformations à bases de règles pour capitaliser et partager l’expertise d’observation de l’activité sur simulateur pleine échelle in IC 2011 - 22èmes Journées francophones d’Ingénierie des Connaissances</span>, Chambéry , France. 2011. (Poster) Show abstract Notre travail porte sur l’aide à l’observation et à l’analyse de l’activité dans les simulateurs pleine échelle de centrale nucléaire à l’Unité de Formation Production Ingénierie d’EDF. Les concepts clefs de notre approche sont basés sur la théorie de la trace modélisée développée par l’équipe SILEX afin de proposer un cadre méthodologique et les outils associés pour capitaliser, gérer et partager l’expertise d’observation.
  • Olivier Champalle, Karim Sehaba, Alain Mille, Yannick Prié. (2011) Observation et analyse de comportements collectifs et/ou individuels par la découverte de connaissances issues de l’exploitation d’un corpus de M-Traces d’activité sur simulateur pleine échelle in EIAH 2011 - Environnements Informatiques pour l’Apprentissage Humain, Mons - Belgique. 2011. 2p (Poster) Show abstract Nos travaux s’inscrivent dans le domaine de l’aide à l’observation de l’activité dans les EIAH de type simulateur pleine échelle de conduite de centrale nucléaire. Dans cette communication, nous présentons une démarche fondée sur la théorie de la trace modélisée dans le but de proposer un cadre méthodologique et les outils associés pour assister et gérer l'activité d'observation.
  • Tarek Djouad, Lotfi Sofiane Settouti, Yannick Prié, Christophe Reffay, Alain Mille. (2010) Un Système à Base de Traces pour la modélisation et l'élaboration d'indicateurs d'activités éducatives individuelles et collectives. Mise à l'épreuve sur Moodle in Technique et science informatique. 29(6). pp. 721-741, 2010. Show abstract Nous présentons dans ce travail un Système à Base de Trace implémenté pour le calcul des Indicateurs dans la plateforme d’apprentissage Moodle (SBT-IM). Nous présentons d’abord un cadre conceptuel permettant de penser tout système exploitant des traces modélisées en définissant les notions de trace, de modèle de trace et de transformation. Nous décrivons ensuite la mise en oeuvre de ce cadre conceptuel dans le cas d’un système à base de trace dédié au calcul des indicateurs dans les situations d’apprentissage collaboratif.
  • Julien Laflaquière, Yannick Prié. (2009) Interactive trace visualization in Trace-based systems in Workshop Interacting with temporal data at CHI 2009. Boston, MA, April 2009. 4pp. Show abstract We first introduce the notions of modeled-traces (M-traces) and trace-based systems. We then present activity analysis and activity support as the two main uses of M-traces, and we discuss interactive visualization challenges related to active reading of traces.
  • Lotfi-Sofiane Settouti, Yannick Prié, Jean-Charles Marty, Alain Mille. (2009) A Trace-Based System for Technology-Enhanced Learning Systems Personalisation in 9th IEEE International Conference on Advanced Learning Technologies, Riga, Latvia. 2009. [AR:27%] doi Show abstract This paper deals with the subject of personalization in technology-enhanced learning (TEL) Systems. As we want user centred solutions, we claim that considerable advantage can be gained from allowing observation and interpretation of learnerspsila interactions traces in TEL systems. The key concepts of our approach are modelled traces, and trace transformations. These transformations such as abstraction or filtering enable a better understanding of the interactions taking place in the TEL systems and allow to provide services for personalization.
  • Damien Cram, Béatrice Fuchs, Yannick Prié, Alain Mille. (2009) Découverte complète et interactive de motifs temporels avec contraintes numériques à partir de séquences d'événements in Atelier "Fouille de données temporelles et analyse de flux de données" à EGC2009, Strasbourg. jan 2009. 6p. Show abstract Nous proposons une méthode complète d'extraction de motifs temporels fréquents avec contraintes numériques à partir d'une séquence d'événements. Ces motifs, appelés chroniques, sont très expressifs par rapport aux motifs séquentiels plus « classiques » tels que les épisodes parallèles ou en série, ce qui implique une grande complexité dans la résolution complète. Un algorithme de résolution permettant l'introduction d'heuristiques et de contraintes utilisateur est proposé. Il est ensuite discuté comment mettre à profit les interactions avec l'utilisateur pour compenser sa grande complexité et le rendre praticable.
  • Damien Clauzel, Karim Sehaba, Yannick Prié. (2009) Modelling and visualising traces for reflexivity in synchronous collaborative systems in International Conference on Intelligent Networking and Collaborative Systems (INCoS 2009), Barcelona, Spain. pp. 16-23. 2009 doi Show abstract This article addresses issues related to traces modelling and visualization in synchronous collaborative learning. The objective is to propose models and tools for representing, transforming, sharing and visualizing traces of users' experiences. The traces here represent the users' activities in their interactions with the learning platform. Our proposition is based on reflexive learning defined as the ability to interact with the situation, in order to meet one's own limitations. This work takes place in the ITHACA project which aims at developing an online learning platform that uses interaction traces as knowledge sources on, and for, the learners' learning as individuals or groups. In this paper, we propose a general framework for trace management and sharing, a generic model of synchronous collaborative activity based on the notion of interaction modes, which we specialized for whiteboard sharing and text chatting. We modelled an IRC client and developed a first implementation.
  • Lotfi Sofiane Settouti, Yannick Prié, Damien Cram, Pierre-Antoine Champin, Alain Mille. (2009) A Trace-Based Framework for supporting Digital Object Memories in 1st International Workshop on Digital Object Memories (DOMe'09) in the 5th International Conference on Intelligent Environments (IE 09), Barcelona, Spain. IOS Press, pp.39-44. 2009. doi Show abstract In this paper, we present a Trace Based framework for managing and transforming traces of observation and use of real life objects. Considering trace based systems as Digital Object Memories (DOMe), we describe how our frame-work can be used to manage DOMe, using trace models and transformations to raise the abstraction level of traces and infer useful knowledge. To demonstrate our approach, we present a simple example of smart home where the use and the state of some of everyday objects are observed, and the resulting traces of such observations are exploited as DOMe providing useful services.
  • Julien Laflaquière, Yannick Prié, Alain Mille. (2008) Ingénierie des traces numériques d'interaction comme inscriptions de connaissances in Ingénierie des Connaissances 2008, june 2008, Grenoble, 12 pp. Show abstract Cet article s'intéresse aux traces numériques d'interaction, et en particulier à ce que nous définissons comme les traces modélisées. Contrairement à une majorité d'approches se basant sur les traces, nous considérons trace et modèle de trace explicite comme indissociables. Cet article aborde deux questions en particulier : la première, théorique, concernant le statut d'inscription de connaissances des traces modélisées ; la seconde, méthodologique, concernant une démarche de modélisation de trace que nous considérons comme une « ingénierie de la trace ». Nous y considérons l'exploitation des traces dans la dynamique d'une activité toujours susceptible d'évoluer et à laquelle le modèle de trace doit rester adapté.
  • Damien Cram, Béatrice Fuchs, Yannick Prié, Alain Mille. (2008) An approach to User-Centric Context-Aware Assistance based on Interaction Traces in Modeling and Reasoning in Context, June 2008, Delft, Netherland, 12 pp. Show abstract This paper presents an approach to context-aware assisting systems that reuse user's personal experience as an alternative to traditional systems having an explicit context model associated to reasoning capabilities on this model. This approach proposes to model the use of the environment through interaction traces representing user's experience and different processes to manage user's traces intended to be exploited for later reuse. A user's assistance is based on the reuse of past traces that are semantically similar to the current sequence of interactions representing the current context of the user. The assistance is based on the identification of recurrent task signatures, which are sequential structures representing typical tasks. These signatures are user-centric since they have been interactively elaborated in accord with the user himself. This paper mentions some possible applications in terms of experience-based user assistance and makes a comparison of this approach to the case-based reasoning paradigm (CBR).
  • Julien Laflaquière, Lotfi-Sofiane Settouti, Yannick Prié, Alain Mille. (2007) Traces et inscriptions de connaissances in Ingénierie des Connaissances 2007, Grenoble, July 2007, 2 pp. (Poster) Show abstract Trace, traces d’utilisation, environnements numériques, inscription de connaissances, modèles de trace, Systèmes à Base de Traces.
  • Julien Laflaquière, Yannick Prié, Alain Mille. (2007) Des traces modélisées, un nouvel objet pédagogique ? in Learning Object Repositories Research Network (LORNET 2007), Montréal, 2007. Show abstract La notion de trace numérique prend de plus en plus d'importance dans le domaine des EIAH. C'est notamment le cas en ce qui concerne les systèmes visant à établir une réflexivité de l'activité. Cet article propose une nouvelle approche des traces d'utilisation pour la réflexivité dans le cadre d'une plateforme d'apprentissage collaboratif en ligne, et propose de les considérer comme un nouvel objet pédagogique.
  • Julien Laflaquière, Yannick Prié. (2007) Traces d'utilisation et réflexivité : la question du collectif in Colloque ARCo'07. 8 pp. Show abstract Cet article propose de retracer le travail qui nous a amené à nous poser la question de l’exploitation des traces d’utilisation dans le cadre d’une situation de travail collectif instrumentée. Contrairement aux travaux classiques dans ce domaine, la question du collectif ne s’est pas posée directement mais indirectement. Cet article montre donc comment, à partir d’un travail sur la création de traces d’utilisation destinées à soutenir une réflexivité de l’activité quand celle-ci est médiée, nous avons été amenés à nous interroger sur ce que pourrait être la réflexivité dans un groupe de travail instrumenté, et comment un système à base de trace destiné à la réflexivité pourrait y jouer un rôle.
  • Lotfi-Sofiane Settouti, Yannick Prié, Alain Mille, Jean-Charles Marty. (2006) Systèmes à base de traces pour l'apprentissage humain in TICE 2006, oct 2006, Toulouse, 8 pp. Show abstract L’observation à base de traces dans les EIAH est au coeur des expérimentations et des études. Dans cet article nous proposons une définition de la trace d’apprentissage humain médié. Nous présentons aussi le système à base de traces, un système informatique permettant l’exploitation des traces. Pour cela, nous présentons un modèle formel et une architecture supportant ce système.
  • Julien Laflaquière, Lotfi-Sofiane Settouti, Yannick Prié, Alain Mille. (2006) A trace-based System Framework for Experience Management and Engineering in Second International Workshop on Experience Management and Engineering (EME'2006) in conjunction with KES2006. sept 2006, 8 pp. Show abstract The paper deals with experience management in computer-mediated environments. It particularly focuses on complex tasks whose support relies more on experience than on knowledge. The presented approach is based on the ?use traces? concept to investigate the ?activity reflexivity? as a first step in experience management, and experience sharing and reusing as possible applications. The paper also outlines the framework supporting Trace-Based Systems creation. Traces, trace models and trace life cycle are formally defined. The main parts of the framework architecture: collecting, transformation, visualization, and query systems are also detailed.
  • Julien Laflaquière, Pierre-Antoine Champin, Yannick Prié, Alain Mille. (2005) Approche de modélisation de l'expérience d'utilisation de systèmes complexes pour l'assistance aux tâches de veille informatiquement médiées in ISKO'France 2005 Organisation des connaissances dans les systèmes d'informations orientés utilisation : Contexte de veille et d'intelligence économique, Apr. 2005, 17 pp. </em> Show abstract L’une des évolutions les plus marquantes de ces dernières années dans les domaines de recherche en système d’information porte sur l’orientation de l’utilisation de ces systèmes. Il ne s’agit plus simplement de modéliser les informations en fonction de leurs contenus mais en fonction des utilisations qui en seront faites. Il faut donc prendre en compte à la fois les caractéristiques liées aux problèmes à résoudre par les informations ainsi que les particularités des utilisateurs de ces systèmes. Les contextes d’utilisation s’élargissent, des différents types de veille (technologique, environnemental, concurrentiel, juridique, etc.) à la veille stratégique et à l’intelligence économique.

  Related software

  • kTBS
• 

  TECNIS: Immersive Teleclinic

  Patient Experience Trace-based Activity Analysis Trace-based Systems VR and Psychotherapy Virtual Reality

  Designing and evaluating patients' journeys in a virtual teleclinic to carry out series of neuropsychological tests with practitioners.

  Details

  Partners: LS2N, CHU Nantes, BioSerenity
  Funding: Plan de relance.
  Cofunding:
  Beginning: 2022-09-01
  End: 2024-08-31
  

Other funded projects

Spectacle en lignes (French National Research Agency (ANR) Corpus, 2013-2014) In this projet dedicated to live performing art creation genetics, we design models for publishing annotation as open data and for annotation-based hypervideo generation publishing. partners: IRI-Centre Pompidou, INRIA, LIRIS, Université Paris-Diderot, Ubicast, Festival d’Aix en Provence, Théâtre les Célestins Lyon. [Hypervideos, Video annotations]

• Thomas Steiner, Pierre-Antoine Champin, Benoît Encelle, Yannick Prié. (2014) Self-Contained Semantic Hypervideos Using Web Components in ISWC Developers Workshop 2014, pp. 96-101. CEUR-WS 1268. 2014. Show abstract The creation of hypervideos—displayed video streams that contain embedded user-clickable anchors and annotations— is a~manual and tedious job, requiring the preparation of assets like still frames, the segmentation of videos in scenes or chapters, and sometimes even the isolation of objects like faces within the video. In this paper, we propose a~semi-automated Web-Components-based approach to self-contained hypervideo creation. By self-contained we mean that all necessary intrinsic components of the hypervideo, e.g., still frames, should come from the video itself rather than be included as external assets. Web Components is a~set of specifications, which let Web developers apply their HTML, CSS, and JavaScript knowledge to build widgets that can be reused easily and reliably. By leveraging this evolving standard, we obtain a~high degree of abstraction, which reduces the burden of creating hypervideos to the familiar task of textually marking them up with HTML elements.
• Thomas Steiner, Hannes Mühleisen, Ruben Verborgh, Pierre-Antoine Champin, Benoît Encelle, Yannick Prié. (2014) Weaving the Web(VTT) of Data in Linked Data on the Web (LDOW2014), Christian Bizer, Tom Heath, Sören Auer, Tim Berners-Lee ed. Seoul. pp. 1-10. 2014. Show abstract Video has become a first class citizen on the Web with broad support in all common Web browsers. Where with structured mark-up on webpages we have made the vision of the Web of Data a reality, in this paper, we propose a new vi- sion that we name the Web(VTT) of Data, alongside with concrete steps to realize this vision. It is based on the evolving standards WebVTT for adding timed text tracks to videos and JSON-LD, a JSON-based format to serial- ize Linked Data. Just like the Web of Data that is based on the relationships among structured data, the Web(VTT) of Data is based on relationships among videos based on WebVTT files, which we use as Web-native spatiotemporal Linked Data containers with JSON-LD payloads. In a first step, we provide necessary background information on the technologies we use. In a second step, we perform a large-scale analysis of the 148 terabyte size Common Crawl corpus in order to get a better understanding of the status quo of Web video deployment and address the challenge of integrat- ing the detected videos in the Common Crawl corpus into the Web(VTT) of Data. In a third step, we open-source an online video annotation creation and consumption tool, targeted at videos not contained in the Common Crawl cor- pus and for integrating future video creations, allowing for weaving the Web(VTT) of Data tighter, video by video.

MUSIMORPHOSE - From discomorphosis to the digitamorphosis French National Research Agency (ANR) Émergences des cultures et des phénomènes culturels (2013-2014) This project was about the effects of the digital turn on the tastes morphology and the uses of music in the everyday life. Questions here are about traces, collection, data analysis and visualisation. Partners: CAPHI, CRAL, LINA. [Data Visualization, Trace-based Activity Analysis]

CLAIRE - Community Learning through Adaptive and Interactive multichannel Resources for Education (French Industry Ministry, “Education” call: 2012-2014) In this project I worked on models, languages and tools for (video) online courses in the web in the context of LMS. The partners were: Open Classrooms, LIRIS Lyon 1, WAM INRIA. [Document Engineering, Hypervideos]

eGonomy - Image base exploration (French Industry Ministry, “GE Digitization” call: 2012-2014) In this project we designed and studied prototypes on the reflexive use of navigation traces.The partners are: Moben Fact, Réunion des Musées Nationaux, Grand Palais, art & fact, Pertimm, CEA List, LIRIS Lyon 1, IRI-Centre Pompidou, Lutin Userlab, Xedix. [Trace-based Systems]

PROCOGEC - PROgiciel COllaboratif de GEstion des Connaissances (ANR RNTL, 2007-2009) A project on the use of traces in a Collaborative Document Management System. Partners: Knowings, G-Scop, MUSI, LIRIS, SYSCOM, ANTECIM, Gaz de France. [Knowledge Engineering, Trace-based Systems]

Langage et Technologie Numérique (ACI - Education formation, 2005-2006) This interdisciplinary project had several directions, one of which allowed us to carry a small study on Word constellations perception and interpretation, using eye tracking on various word graphs. Partners : LIRIS (Y. Prié), LIL (B. Pincemin), LPEQ (P. Therouanne, T. Colombi, T. Baccino), LACO (J.F. Rouet, E. Meme, M. Puustinen, J. Bernicot, O. Volkaert, J.P. Thibaut), LSE (B. Lemaire, P. Dessus) [Interpretation Systems, Trace-based Activity Analysis]

Personnalisation dans les EIAH (Projet cluster Région, 2006-2010) Regional project on trace-based personalization in tutoring systems. Partners: TEL and Education Sciences Laboratories in Rhône-Alpes Region. This project did fund Lotfi Settouti’s PhD thesis. [Reflective Systems, Technology-Enhanced Learning, Trace-based systems]

IncorporActions (Action Concertée Incitative Tarrains Techniques Théories, 2004-2007) A project lead by Lorenza Mondada on how to model descriptions of actions as well as oral transcriptions in interaction studies. Lots of video, transcriptions, and annotations. Partners: LIRIS, ICAR, France Telecom R&D. [Experience analysis and modelling, Interpretation Systems, Video Annotations]

ACTEURS: Activités Collectives et Tutorat dans l’Enseignement Universitaire : Réalités et Scénarios (ACI TTT) 20 Education and Computer science researchers from Rhône-Alpes Region worked on usages of collaborative distant teaching platform for graduate students, with various methodologies (a description in french) [Technology-Enhanced Learning, Trace-based Activity Analysis]

CPER “Développement d’un pôle de compétences méthodologiques pour l’étude des usages d’Internet” An interdisciplinary workgroup around web and internet usages Partners: Georges Antoniadis, Jean Emmanuel Le Bray (Grenoble 3), Alain Mille, Yannick Prié (Lyon 1), Lorenza Mondada, (Lyon 2), Laurent Jeannin (CNRS ICAR), Karine Robineau (INRP), Igor Babou, Barbara Doucet, Joëlle Le Marec, Christine Develotte (ENS LSH) [Trace-based Activity Analysis]

ISDN (Institut des Sciences du documents numérique):. ISDN was a french interdisciplinary research group around the study of digital documents. I participated in several subprojects, with several publications. [Document Engineering] 

• Emergence collective du document: I worked with sociologists and information scientists on the way documents emerged as stable forms within organizations.
• Multistructuralité du document: with computer scientists on multi-structured documents (vs. classically mono-structured documents).
  • Rocio Abascal, Michel Beigbeder, Aurélien Bénel, Sylvie Calabretto, Bertrand Chabbat, Pierre-Antoine Champin, Noureddine Chatti, David Jouve, Yannick Prié, Béatrice Rumpler, Eric Thivant. (2004) Documents à structures multiples in SETIT 2004, Susa Tunisia, mars 2004 Show abstract Dans cet article, nous présentons les résultats de recherches collectives sur la multistructuralité des documents, menées au sein de l’Institut des Sciences du Document Numérique (ISDN2) et financées par la région Rhône-Alpes (Programmes Thématiques 2000-2003). Nos recherches ont abouti à la définition formelle et générique d’un document à structures multiples et d’un certain nombre d’opérations sur ce type de documents (édition, fusion/éclatement, ...). Nous illustrons nos propositions sur deux cas d’application : la diffusion des thèses de l’INSA de Lyon et la mise en ligne de la Chronique des Fouilles Archéologiques de l’Ecole française d’Athènes.
  • Rocio Abascal, Michel Beigbeder, Aurélien Bénel, Sylvie Calabretto, Bertrand Chabbat, Pierre-Antoine Champin, Noureddine Chatti, David Jouve, Yannick Prié, Béatrice Rumpler, Eric Thivant. (2003) Modéliser la structuration multiple des documents in H2PTM'03: Hypertextes, HyperMédias, Paris, sept. 2003, 4 pp. Show abstract Dans cet article, nous présentons les résultats de recherches collectives sur la multistructuralité des documents, menées au sein de l’Institut des Sciences du Document Numérique (ISDN ) et financées par la région Rhône-Alpes. Ces recherches ont abouti à la définition formelle et générique d’un document à structures multiples.
• Temps et documents numériques with information and computer scientists.
  • Sylvie Calabretto, Geneviève Lallich-Boidin, Jocelyne Nanard, Yannick Prié, Florence Sedes. (2007) Les temps du document numérique in Documents et contenu : création, indexation, navigation. Paris : Cepadues. 2007.

MAZETTE (Multi-agent Musette): a non defended PhD thesis (Jesus Arana) with Salima Hasses around the use of multi-agents systems to share traces. [Knowledge Engineering, Trace-based Systems]

• Jesus Arana, Salima Hassas, Yannick Prié. (2004) MAZETTE: Multi Agent MUSETTE for Sharing and Reusing Ontologies in WOSE Workshop on Ontologies, Semantics, and E-learning, LNCS 3292, oct 2004. Show abstract During the realization of a document-mediated collective task the participants act and interact by creating documents and ontologies, by modifying them, annotating them and exchanging them. This article presents the general principles of a model based on a multi-agent architecture, aimed at facilitating the co-construction of common ontologies. The model is built on the MUSETTE (Modelling USEs and Tasks for Tracing Experience) approach, which was designed to provide users with assistance based on the recording and reusing of their system use traces. Our model, called MAZETTE (Multi-Agent MUSETTE), defines a framework for considering sharing and reusing of collective traces and experience, amongst which we focus on ontology co-construction and reuse.

RECIS - Recherche et exploration de contenus par l’image et le son (RNRT, 1999-2002) A project on image and sound retrieval and exploration. We did audiovisual documents indexing and retrieval, with a touch of Case Based Reasoning and Semantic Web. Partners: LIRIS, INRIA-IMEDIA, France Telecom R&D. This project funded Elöd Egyed-Zsigmond PhD thesis. [Document Engineering, Video Annotations, Trace-based Systems]

• Elöd Egyed-Zsigmond, Alain Mille, Yannick Prié́. (2003) Club (Trèfle) : a use trace model in 5th International Conference on Case-Based Reasoning, LNCS, volume 2689, Trondheim, NO, pp. 146–160. [AR:21%] doi Show abstract In this paper we present a use trace model which allows the collection and reuse of user experience, based on a homogeneous and interconnected representation of users, procedures and objects. All these notions form a connected labeled directed graph containing highly connected and explained use traces. This model enables assistance in non trivial, creativity requiring situations. Our model uses the Case Based Reasoning (CBR) paradigm in order to reuse experience. After a formal description of the model we discuss how it can serve to capitalize and re-use experience.
• Weihong Huang, Yannick Prié, Pierre-Antoine Champin, Alain Mille. (2002) Semantic Context Representation of Resources Using Annotation Graph in MIS 2002 International Workshop on Multimedia Information Systems, nov 2002, Tempe, Arizona, pp. 48-55
• Weihong Huang, Yannick Prié, Pierre-Antoine Champin, Alain Mille. (2002) A Framework for Semantic Context Representation of Multimedia Resources in Proceedings of the 18th international conference CODATA 2002 - Frontiers of Scientific and Technical Data, sept. 2002, Montreal Canada. Show abstract With the explosion of online multimedia resources, requirement of intelligent content-based multimedia service increases rapidly. One of the key challenges in this area is semantic contextual knowledge representation of multimedia resources. Although current image and video indexing techniques enable efficient feature-based operation on multimedia resources, there still exists a "semantic gap"between users and the computer systems, which refers to the lack of coincidence between the information that one can extract from the visual data and the interpretation that the same data has for a user in a given situation. In this paper, we present a novel model: annotation graph (AG) for modeling and representing contextual knowledge of various types of resources such as text, image, and audio-visual resources. Based on the AG model, we attempt to build an annotation graph framework towards bridging the "semantic gap" by offering universal flexible knowledge creation, organization and retrieval services to users. In this framework, users will not only benefit from semantic query and navigation services, but also be able to contribute in knowledge creation via semantic annotation. In the AG model, four types of concrete description elements are designed for concrete descriptions in specific situations, while two types of abstract description elements are designed for knowledge reusing in different situations. With these elements and directed arcs between them, contextual knowledge at different semantic levels could be represented through semantic annotation. Within the global annotation graph constructed by all AGs, we provide flexible semantic navigation using derivative graphs (DG) and AG. DGs enable complement contextual knowledge representation to AGs by focusing on different types of description elements. Towards semantic query, we present a potential graph (PG) tool to help users visualize query requests as PGs, and execute queries by performing sub-graph matching with PGs. Prototype system design and implementation aim at an integrated user-centered semantic contextual knowledge creation, organization and retrieval system.
• Elöd Egyed-Zsigmond, Yannick Prié, Alain Mille, Jean-Marie Pinon. (2002) Trèfle, modèle de traces d'utilisation in Ingénierie des Connaissances 2002, Rouen, France, May 2002, pp. 39-50. Show abstract Dans cet article nous présentons un modèle de représentation de traces d’utilisation. Nous par- tons de l’idée que lors de l’utilisation d’une application informatique des utilisateurs manipu- lent des objets à l’aide de procédés. Nous propo- sons donc de représenter l’évolution d’un sys- tème à l’aide d’un graphe qui contient des repré- sentations des utilisateurs, procédés et objets manipulés, en les interconnectant. Ce graphe évolue au fur et à mesure de l’utilisation de l’application fournissant ainsi une base toujours plus riche de capital d’expérience que l’on peut réutiliser.
• Aurélien Benel, Elöd Egyed-Zsigmond, Yannick Prié, Sylvie Calabretto, Alain Mille, Jean-Marie Pinon. (2001) Truth in the Digital Library: From Ontological to Hermeneutical Systems in ECDL - European Conference on Digital Libraries, Darmstadt, sept. 2001, pp. 366-377. [AR:28%] doi Show abstract This paper deals with the conceptual structures which describe document contents in a digital library. Indeed, the underlying question is about the truth of a description: obvious (ontological), by convention (normative) or based on interpretation (hermeneutical). In the first part, we examine the differences between these three points of view and choose the hermeneutical one. Then in the second and third part, we present two “assisted interpretation systems”. (AIS) for digital libraries (audiovisual documents and scholarly publications). Both provide a dynamic annotation framework for readers’ augmentations and social interactions. In the fourth part, a few synthetic guidelines are given to design such “assisted interpretation systems” in other digital libraries.
• Elöd Egyed-Zsigmond, Yannick Prié, Alain Mille, Jean-Marie Pinon. (2000) A graph-based audiovisual document annotation and browing system in Short article RIAO'2000, Paris, apr. 2000, vol. 2, pp. 1381-1389. Show abstract In this paper we present an audiovisual (AV) data annotation and exploitation system, which was developed as a feasibility demonstration of several theoretical models and algorithms. These models provide a graph based AV document annotation model, a rapid search method to exploit these annotations and the representation of connected, distributed graphs in XML. First we present the theoretical background underlining the new approach of our model to the audiovisual document annotation and exploitation. In the second part we describe our demonstration application illustrating the use cases related to the theoretical model. At the end we will present our plans to enhance as well the theoretical models as the application.

SESAME - Système d’Exploration de Séquences Audiovisuelles et Multimédia enrichi par l’Expérience (FT CCETT, 1999-2002) The project that funded my PhD thesis. Partners were France Telecom R&D, INA and France3 Rhône-Alpes-Auvergne. [Document Engineering, Video Annotations]

• Yannick Prié, Tahar Limane, Alain Mille. (2000) Isomorphisme de sous-graphe pour la recherche d'information audiovisuelle contextuelle in 12ème congrès Reconnaissance de Formes et Intelligence Artificielle, RFIA2000, Paris, feb. 2000, vol. I, pp. 277-286. Show abstract Cet article présente un algorithme de recherche d'information dans un système d'information représenté par un graphe. L'approche des Strates Interconnectées par les Annotations pour l'annotation de documents audiovisuels est d’abord expliquée, ainsi que la notion de graphes potentiels caractérisés pour décrire des contextes audiovisuels et des requêtes. Le problème de la résolution des requêtes se ramène alors à un problème de recherche d'isomorphismes de sous-graphes correspondant à un graphe potentiel, dans un graphe global. Après avoir présenté ce problème dans sa généralité, nous détaillons un algorithme de multi-propagation reposant sur l’étude d’une heuristique raisonnable en recherche d'informations.
• Yannick Prié. (2000) Sur la piste de l'indexation conceptuelle de documents. Une approche par l'annotation in Document Numérique, numéro spécial L'indexation, Vol. 4, Num. 162, Dec. 2000, pp. 11-35. Show abstract Dans une première partie, nous justifions l’engouement actuel pour l’indexation conceptuelle de documents que nous définissons comme ensemble de connaissances symboliques en permettant l’exploitation, ce que nous illustrons en présentant des modèles récents de descriptions de documents fondées sur les connaissances. Nous proposons également de considérer toute structure documentaire comme structure sémantique ayant forme de graphe. Dans une deuxième partie, nous proposons de voir toute mise en place d’indexation conceptuelle comme se faisant lors d’une annotation, laquelle correspond à une documentation de sa propre pratique d’exploitation du document. Nous présentons quelques exemples récents liés à la notion d’annotation, et discutons les perspectives ouvertes tant par le concept d’indexation conceptuelle que celui d’annotation.
• Yannick Prié. (1999) Modélisation de documents audiovisuels en Strates Interconnectées par les annotations pour l'exploitation contextuelle in PhD Dissertation in Computer Science, INSA de Lyon, Dec 1999, 270 p. My PhD thesis was dedicated to audiovisual document representation for their use in audiovisual information systems. The document is entitled Representing video documents with annotation-interconnected strata for contextual exploitation. I proposed a model called Annotation Interconnected Strata (AI-Strata) and several models and concepts to exploit it with context in mind. I did also study current and futur use of audiovisual documents in the humanities.
  Examination committee: Marie-France Bruandet (Université Joseph-Fourier, Grenoble, referee), Marc Nanard (CNAM, Paris, referee), Bruno Bachimont (UTC Compiègne, Direction de la Recherche, examiner), Françoise Chassaing (CCETT-CNET, Rennes, examiner), Alain Mille (LISA, CPE, Lyon, co-supervisor), Jean-Marie Pinon (LISI, INSA-Lyon, co-supervisor) Show abstract La numérisation et la création numérique de flux audiovisuels (AV) permettent leur exploitation dans des systèmes d'information audiovisuels. Cela nécessite une modélisation et une instrumentation des contenus des documents AV qui en autorise l'accès direct et soit adaptée aux diverses utilisations possibles : recherche, indexation, navigation, etc. Après une revue des propositions actuelles de modélisation de documents AV, nous justifions quelques nécessités en vue de leur représentation. Nous proposons de représenter les documents AV en Strates Interconnectées par les Annotations (Strates-IA), c'est à dire d'écrire sur le flux à l'aide de termes (éléments d'annotation). Ceux-ci annotent des morceaux de flux (unités audiovisuelles), sont en relation entre eux, et sont instances d'éléments d'annotations abstraits décrits dans un graphe de relations conceptuelles. L'ensemble du système représente alors un graphe de connaissances global dans lequel nous définissons la notion de contexte comme extrémité de chemin contextuel et pour l'exploitation duquel nous décrivons divers outils contextuels basés sur la notion de graphe potentiel. Les graphes potentiels représentent des visées de description de l'utilisateur liées à ses tâches particulières. Ils s'instancient dans le graphe global (recherche d'isomorphismes de sous-graphes partiels) et nous en proposons un algorithme efficace de multi-propagation. Nous proposons également une modélisation dessystèmes documentaires permettant le stockage intelligent de l'expérience sous la forme de cas d'utilisation en vue d'une réutilisation pour l'assistance. Enfin, nous discutons plus généralement les rapports entre documents et connaissances.
  Mots-clés: Modélisation de documents et contenus audiovisuels, représentation de connaissances, exploitation contextuelle, isomorphisme de sous-graphes, Strates Interconnectées par les Annotations, expérience d'utilisation
• Yannick Prié, Alain Mille et Jean-Marie Pinon. (1999) Modèle d'utilisation et modèles de tâches pour l'assistance à l'utilisateur basée sur l'expérience : le cas d'un système d'information audiovisuelle in Ingénierie des Connaissances 1999, Palaiseau, jun. 1999, pp. 21-30. Show abstract Nous présentons dans cet article un modèle original de description de systèmes informatiques en modèle d'utilisation et modèles de tâches, qui permettent de décrire l'activité concrète d'un utilisateur en cas d'utilisation, base de l'apprentissage pour l'aide à l'utilisateur basée sur l'expérience. Comme illustration des ces principes dans un système d'information audiovisuel, nous présentons tout d'abord le modèle d'utilisation de SESAME, les Strates-Interconnectées par les Annotation, avec notamment la notion originale de graphes potentiels contextuels. Un modèle de la tâche centrale de description de SESAME, la tâche d'annotation, est ensuite décrit. Une discussion sur les possibilités d'apprentissage dans les cadres mis en place vient conclure cet article.
• Gwendal Auffret, Yannick Prié, Bruno Bachimont. (1999) Digitizing TV and Radio Archives: supporting scholarship by providing new means of access to audiovisual documents in Digital Ressources in the Humanities DRH'99, London, UK, sept. 199, pp. 95-99.
• Yannick Prié, Alain Mille, Jean-Marie Pinon. (1999) Connaissances et documents audiovisuels : un modèle pour l'exploitation contextuelle des annotations in Document numérique, numéro spécial Gestion des documents et gestion des connaissances, dec. 1999, Vol. 3, Num. 3-4, pp. 241-262. Show abstract Cet article est consacré à l’étude de l’utilisation de connaissances contenues dans les documents audiovisuels. Nous montrons d’abord qu’il est nécessaire d’expliciter celles-ci sous la forme d’annotations rendant possible leur exploitation contextuelle. Nous présentons ensuite le modèle des Strates Interconnectées par les Annotations (Strates-IA) et les outils contextuels permettant une telle exploitation. Nous étudions enfin la nature des différentes connaissances utilisées dans un système Strates-IA.
• Yannick Prié, Alain Mille, Jean-Marie Pinon. (1999) A Context-Based Audiovisual Representation Model for Audiovisual Information Systems in Context'99, Second International and Interdisciplinary Conference on Modeling and using Context, Trento, sept. 1999. Lecture Notes in Artificial Intelligence, vol. 1688, pp. 296-309. doi Show abstract In this paper we present a contextual representation model of audiovisual (AV) documents for AV information systems. In the first part, we study AV medium, and show that AV intra-document context is always related to a user task seen as a general description task. We then present the AI-Strata model for AV description: audiovisual units (pieces of AV documents) are annotated with annotation elements described in a knowledge base. The annotation elements are connected at the document level. The whole system being considered as a single graph, we define a context of one element as end points of graph-paths starting with this element. In order to control contextual paths, we define the notion of potential graphs as graph-patterns instantiated in the general graph. Finally, we show how these graphs are used in the main task of AV information system: navigation, indexing and retrieval.
• Gwendal Auffret, Yannick Prié. (1999) Managing Full-indexed Audiovisual Documents: a New Perspective for the Humanities in Computer and the Humanities, special issue on Digital Images, dec. 1999, Vol. 33, Num. 4, pp. 319-344. Show abstract The digitization of library documents and archives increasingly extends to audiovisual (AV) document repositories. As a consequence, new computer-aided techniques are being devised, providing opportunities for new uses of AV documents. As scholars work mainly by reading, annotating, reusing, and producing documents they are directly concerned by these changes. The first part of this article describes AV document use in the humanities, as well as the current and future influence computers might have on evolving practices. After establishing that “full-indexing” (indexing of the content for random access to any segment of an AV document) is a necessary condition if scholars are to develop new practices in using AV material, we will focus on the specific problems raised by AV indexing as opposed to text indexing, followed by a discussion of related AV indexing projects as well as standardization issues. The third part will propose a representation model for the description of AV material (AI-Strata) and an exchange format of AV annotations (AEDI), based on a free segmentation approach. An example of annotation is also provided. The last part is devoted to a discussion regarding potential long-term influences of digital AV indexing techniques on scholarly uses of AV documents.
• Yannick Prié, Alain Mille et Jean-Marie Pinon. (1998) Une approche de modélisation de documents audiovisuels en strates interconnectées par les annotations in Ingénierie des Connaissances 1998, Pont-à-Mousson, may 1998, pp. 143-152. Show abstract Nous donnons tout d'abord quelques spécificités fondamentales de l'audiovisuel. Nous présentons ensuite une méthode d'analyse des modèles pour la recherche d'information audiovisuelle et les deux principales approches structurelles utilisées. Nous proposons une approche originale de modélisation audiovisuelle en strates interconnectées par les annotations. Un document s'analyse suivant des dimensions d'analyse permettant de repérer des objets d'intérêt qui peuvent être de tous types (structurels, conceptuels, etc.). L'annotation est alors structurée à partir d'éléments d'annotation (EA), qui sont des termes représentant aussi bien des objets d'intérêt que des relations, une base de connaissances permettant de contrôler leur utilisation. Nous montrons ensuite comment il est possible de construire à partir de relations entre EA (et donc entre strates) des vues contextuelles de celles-ci, c'est-à-dire de les annoter à l'aide d'EA annotant au départ d'autres strates. Nous présentons pour finir quelques tâches de description de documents audiovisuels dans leurs liens avec l'approche de modélisation proposée.
• Yannick Prié, Alain Mille, Jean-Marie Pinon. (1998) AI-STRATA: A User-centered Model for Content-based description and Retrieval of Audiovisual Sequences in 1st International Conference on Advanced Multimedia Content Processing (AMCP'98), Osaka, Japan, nov. 1998, Lecture Notes in Computer Science, vol. 1554, pp. 328-343. doi Show abstract We first insist on the need for conceptual and knowledge-based audiovisual (AV) models in AV and multimedia information retrieval systems.We then propose several criteria for characterizing audiovisual representation approaches, and present a new approach for modeling and structuring AV documents with Annotations Interconnected Strata (AI-STRATA). This consists in analyzing AV documents through analysis dimensions allowing the detection of objects of interest of any type (structural, conceptual,...). Annotations are structured by annotation elements (AE) representing both objects of interest and relationships. A knowledge base is used in order to monitor the annotation process. We show how to use annotations to link different strata on the base of explicit or implicit contexts and how AI-Strata can be used to build contextual views of a stratum, using both annotation and knowledge levels. We finally show how the model can efficiently support different description tasks such as indexing, searching and browsing audiovisual material.
• Yannick Prié, Jean-Michel Jolion et Alain Mille. (1998) SESAME : modèle conceptuel de description de documents audiovisuels et assistants d'annotation de contenu in 4èmes Journées d'Etudes et d'Echanges COmpression et REprésentation des Signaux Audiovisuels (Coresa'98), CNET, Lannion, jun 1998, pp. 227-234. Show abstract Sesame (Système d’Exploration de Séquences Audiovisuelles et Multimédias enrichi par l’Expérience) est un projet rassemblant plusieurs équipes de différents laboratoires (LISI et RFV de l’Insa-Lyon, LIP de l’Ens-Lyon et LISA de Cpe-Lyon) autour de la probléma- tique générale de l’accès à des séquences audiovisuelles (SAV) par le contenu pour dif- férentes tâches comme la recherche, l’analyse, la manipulation, le montage, etc. Le groupe de recherche constitué autour de SESAME s’est fixé comme objectif de prendre en compte l’ensemble du problème comme un tout indissociable: modèle de description de morceaux d’audiovisuel, outils d’assistance, modèle de base de données audiovisuelles, archi- tecture parallèle pour les serveurs, etc. Cette approche globale facilite la prise en compte mutuelle des contraintes des différents aspects du problème. L’ambition de maquetter par morceaux les différents éléments de SESAME puis d’intégrer les résultats sur un démonstrateur (sous la responsabilité du CISM Lyon1-INSA) est raisonnable compte tenu des partenariats installés avec l’INA d’une part et FRANCE 3 d’autre part. La communication se focalisera sur quelques aspects particulièrement critiques pour la construction globale de SESAME : un modèle de description de morceaux audiovisuels, des mécanismes d’annotation automatiques ou semi-automatiques permettant de rendre cette description accessible, avant d’aborder la notion plus générale d’assistant de description de documents audiovisuels.

Other non funded projects

Archives Michaux: we worked with Olivier Aubert on consolidating the Henri Michaux painting and drawing annotated catalog (catalogue raisonné). There was also some work on automated corpus analysis and classification. [Document Engineering, Interpretation Systems]

Datadata: a program at Ecole des Beaux-Arts de Lyon, resulting in a collaboration with artists and anthropologists (3 years of seminar and workshops, notably around the Advene tool). Datadata website is down, but a post-hoc description is available (french). [Hypervideos, Interpretation Systems]

CONSTELLATIONS (Ecrire et interpréter des constellations de mots): an interdisciplinary research group around the use of graphs (semantic network, UML diagrams, etc.) considered as word constellations that have to be written and read taking into account the spatial positionning of nodes and links. Partners: ICAR (B. Pincemin, linguistics), GREYC (B. Morand, semiotics), Culture et Communication (Daniel Jacobi, communication studies) [Experience analysis and modelling]

COHERENCE (Corpus hétérogènes pour la recherche en sciences humaines et sociales) A small project with communication science and semiotics researchers on heterogeneous corpora for the humanities. Partners: LIRIS (A. Mille), CS2O-ENS-Lyon (I. Babou, J. Le Marec)