1. |
Landragin, F., Gaiffe, B.,
Bellalem, N. & Romary, L., Fusion de contraintes pour la
synchronisation des modalités et pour la résolution des
références dans un énoncé multimodal, In:
Colloque sur les Interfaces Multimodales, 10 ans de
multimodalité, Grenoble, France, 2000 (4 pages) |
Presentation of an algorithm for multimodal
references resolution, based on the semantic treatment of logical
forms. As opposed to my other publications, this algorithm
(the first of my Ph.D. work) is highly rooted in formal semantics. |
2. |
Landragin, F., Bellalem,
N. & Romary, L., Compréhension automatique du geste et de
la parole spontanés en communication homme-machine :
apport de la théorie de la pertinence, In: Oralité
et gestualité. Interactions et comportements multimodaux dans
la communication (actes du colloque ORAGE'01), L'Harmattan,
Paris, France, 2001 (4 pages) |
Synthesis on the problems linked to the Language
and Dialogue team's approach for automatic understanding of multimodal
actions. Presentation of the main themes of a methodology based on
Relevance Theory. |
3. |
Landragin, F., Bellalem,
N. & Romary, L., Visual Salience and
Perceptual Grouping in Multimodal Interactivity, In: Workshop on
Information Presentation and Natural Multimodal Dialogue, Verona,
Italy, 2001 (5 pages) |
Synthesis on the visual salience criteria and on
the perceptual grouping of the objects from a visual scene.
Exploitation of these two implicit phenomena for the understanding
of linguistic and multimodal referring actions. |
4. |
Landragin, F., Bellalem,
N. & Romary, L., Referring to Objects
with Spoken and Haptic Modalities, In: IEEE International
Conference on Multimodal Interfaces (ICMI'02, Pittsburgh, PA),
IEEE CS Press, Los Alamitos, CA, 2002 (6 pages) |
Demonstration of the possibility to adapt the
reference domain model to a particular interaction type, the one
of the force feedback gesture (haptic gesture) of the MIAMM
project. |
5. |
Landragin, F.,
The Role of Gesture in Multimodal Referring Actions,
In: IEEE International Conference on Multimodal Interfaces
(ICMI'02, Pittsburgh, PA), IEEE CS Press, Los Alamitos, CA, 2002
(6 pages) |
Synthesis on the possible roles of pointing gestures,
in terms of reference domains delimitation or referents extraction
from implicit domains. Proposition of an algorithm for multimodal
reference resolution on this basis.. |
6. |
Landragin, F., De Angeli, A.,
Wolff, F., Lopez, P. & Romary, L., Relevance and Perceptual
Constraints in Multimodal Referring Actions, In: van Deemter, K. &
Kibble, R. (Eds.), Information Sharing: Reference and
Presupposition in Language Generation and Interpretation, CSLI
Publications, Stanford, CA, 2002 (19 pages) |
Presentation of a computational adaptation of
Relevance Theory to help multimodal reference resolution. A particular
attention is provided to the visual context. |
7. |
Landragin, F., Salmon-Alt,
S. & Romary, L., Ancrage référentiel en situation de
dialogue, Traitement Automatique des Langues 43(2),
Hermès-Lavoisier, Paris, France, 2002 (31 pages) |
Presentation of the integration of visual perception,
language, gesture, and task in an unified framework. This journal
paper is the core of my Ph.D. work. The outcome is the specification
of an algorithm based on contextual subsets modeling into feature
structures, and on the interpretation process modeling into the
unification of these feature structures. |
8. |
Landragin, F., La saillance
comme point de départ pour l'interprétation et la
génération, Journée d'étude de
l'Association pour le Traitement Automatique des LAngues (ATALA) sur
le thème : structure communicative/structure
informationnelle, Paris, France, 2003 (4 pages) |
Characterization of salience, with the identification
of visual salience criteria and linguistic salience criteria. This
presentation focus on linguistic salience, with the transposition
of visual criteria to linguistics and the proposition of a method
for salience computation. The link with communicative structure is
established. |
9. |
Landragin, F., Une
caractérisation de la pertinence pour les actions de
référence, In: Dixième conférence sur
le traitement automatique des langues (TALN 2003), Batz-sur-Mer,
France, 2003 (10 pages) |
Proposition of a characterization of the relevance
of multimodal referential expressions, with a particular interest on
the processing effort. This characterization exploits the notion of
reference domain, and is more homogeneous and sharp than the one of
publication 6. |
10. |
Landragin, F., Clues for the
Identification of Implicit in Multimodal Referring Actions, In:
Tenth International Conference on Human-Computer Interaction (HCI
International 2003, Heraklion, Crete, Greece), Lawrence Erlbaum
Associates, Mahwah, NJ, 2003 (5 pages) |
Synthesis on the problem of the interpretation in
multimodal human-machine dialogue, from the implicit identification
point of view. Exploitation of the clues given by the utterance of the
user and by the visual scene that constitutes the interaction support. |
11. |
Landragin, F. & Romary,
L., Referring to Objects Through Sub-Contexts in Multimodal
Human-Computer Interaction, In: Seventh Workshop on the Semantics
and Pragmatics of Dialogue (DiaBruck'03), Saarbrücken,
Germany, 2003 (8 pages) |
Systematic investigation of referring phenomena in
multimodal dialogue. Proposition of a classification of referring
modes linked to the existence of reference domains. Specification
of an algorithm for the identification of the possible modes for
each type of nominal phrases. |
12. |
Gaiffe, B., Landragin, F. &
Quignard, M., Le dialogue naturel comme un service dans un contexte
multi-applicatif, In: Journée d'étude de
l'Association pour le Traitement Automatique des LAngues (ATALA) sur
les relations entre systèmes multi-agents et traitement
automatique des langues (AGENTAL), Paris, France, 2004 (10
pages) |
Investigation of a multi-agent architecture to
clearly distinguish the dialogue aspects (one agent: the dialogue
manager) from the task-linked aspects (one agent per application).
Description of the dialogue manager using three modules: linguistic,
collaborative, and applicative. Exploitation of the reference
domain model in the linguistic module, with an illustration taken
from the OZONE project. |
13. |
Landragin, F., L'utilisation
de scores numériques en sémantique computationnelle,
Journées scientifiques de Sémantique et
Modélisation (JSM'04), Lyon, France, 2004 |
Synthesis of the various methods to quantify
semantic aspects, with the example of salience computation.
This presentation completes the publication 8. |
14. |
Landragin, F., Interface
sémantique-pragmatique et domaines de référence,
In: Quatrièmes Journées d'Études Linguistiques
de Nantes (JEL 2004), Nantes, France, 2004 |
First step towards the integration of verbal
semantics to the reference domain model, with the construction of
linguistic domains on the basis of predication. |
15. |
Landragin, F., Denis, A.,
Ricci, A. & Romary, L., Multimodal Meaning Representation for
Generic Dialogue Systems Architectures, In: Fourth International
Conference on Language Resources and Evaluation (LREC 2004, Lisbon,
Portugal), 2004 (4 pages) |
Description of the update of MMIL (MultiModal
Interface Language) for semantic representation, during the
re-using of MIAMM architecture to the OZONE demonstrator. |
16. |
Landragin, F. & Romary,
L., Dialogue History Modelling for Multimodal Human-Computer
Interaction, In: Eighth Workshop on the Semantics and Pragmatics of
Dialogue (Catalog'04), Barcelona, Spain, 2004 (8
pages) |
Synthesis on the design and the management of a
dialogue history inside a multimodal human-machine dialogue system.
Description of the nature of the modality-linked local histories,
and, following publication 15, of the unified representation of
information in the histories. |
17. |
Landragin, F., Saillance
physique et saillance cognitive, Cognition, Représentation,
Langage (CORELA) 2(2), 2004 (24 pages) |
List of the factors that contribute to visual
salience and to linguistic salience, with a distinction between
physical factors that come from the physical environment, and
cognitive factors that depend on mental states of the subject. The
notions of P-salience (physical salience) and C-salience (cognitive
salience) are proposed in this way. |
18. |
Landragin, F., Dialogue
homme-machine multimodal, Hermès-Lavoisier, Paris,
France, 2004, ISBN 2-7462-0992-6 (270 pages) |
Updated version of my Ph.D. thesis, including three
parts. The first part deals with the problems and methods relevant
with the resolution of references to objects in human-machine
dialogue. The second part presents the concepts and the proposed model
based on reference domains. And the third part describes some
applications for this model. |
19. |
Landragin, F., Traitement des
actes de langage dans un système de dialogue homme-machine,
Journées scientifiques de Sémantique et
Modélisation (JSM'05), Paris, France, 2005 |
Synthesis on the phenomena that may be taken into
account for speech acts processing in dialogue systems. Focus on
complex speech acts, i.e., indirect speech acts and composite speech
acts. A comparison is made between complex speech acts processing and
collaborativeness in task-oriented dialogue. |
20. |
Landragin, F.,
Modélisation de la saillance visuelle et linguistique, In:
Sixième Colloque des Jeunes Chercheurs en Sciences
Cognitives (CJCSC'05), Bordeaux, France, 2005 (6 pages) |
Proposition of five dimensions or study axes that can
be useful when apprehending the notion of salience. The first axis
deals with the distinction between visual and linguistic salience, the
second one with salience linked to the message form versus the message
content, the third one with immediate effect salience versus
continuous effect salience, the fourth one with prior salience versus
new salience, and the fifth one with informative salience versus
rhetorical salience. |
21. |
Landragin, F.,
Traitement automatique de la saillance, In:
Douzième conférence sur le traitement automatique
des langues (TALN 2005), Dourdan, France, 2005 (10 pages) |
Proposition of two principles that characterize
the notion of salience and that allow its contextual quantification.
The first principle is called primordiality principle and favors
the most important entities. The second is called singularity
principle and favors the most singular entities, i.e., the entities
characterized by the most distinctive features (that the other
entities in the context do not have). |
22. |
Landragin, F.,
Modeling Context for Referring in Multimodal Dialogue Systems, In:
Fifth International and Interdisciplinary Conference on Modeling
and Using Context (CONTEXT'05), Paris, France, 2005 (14 pages) |
Synthesis corresponding to the core of my
Ph.D. thesis, with the presentation of the multimodal reference
domains model. In particular concerning salience and gesture aspects,
the contents of publications 3 and 5 are updated and integrated. |
23. |
Landragin, F., Indirect Speech
Acts and Collaborativeness in Human-Machine Dialogue Systems, In:
First International Symposium on the Exploration and Modelling of
Meaning (SEM-05), Biarritz, France, 2005 (8 pages) |
Following publication 19 and with the aim to make
the dialogue systems more collaborative, synthesis on the
parameters that a system may take into account when
processing indirect and composite speech acts. |
24. |
Landragin, F., Une
caractérisation de la référence ostensive indirecte,
Revue de Sémantique et de Pragmatique (RSP) 18,
Presses Universitaires d'Orléans, France, 2005 (19 pages) |
List of the phenomena that correspond to deferred
ostension, i.e., when the gesture points out an object that is not the
referent of the simultaneous linguistic utterance. Classification of
the underlying semantic transfers, and presentation of a bottom-up
method for the automatic interpretation of these phenomena in
multimodal human-machine dialogue. |
25. |
Landragin, F., Visual Perception,
Language and Gesture: A Model for their Understanding in Multimodal
Dialogue Systems, Signal Processing 86(12), Elsevier, Amsterdam,
The Netherlands, 2006 (18 pages) |
Updated version of the state of the art, approach and
concepts from publication 22, together with the algorithm from
publication 11. This paper corresponds to the definitive english version
of the core of my Ph.D. work. |
26. |
Landragin, F., Modélisation
du sens et du contexte sur la base de représentations des objets
référés, Journées scientifiques de
Sémantique et Modélisation (JSM'06), Bordeaux,
France, 2006 |
This communication questions the feasability of a
semantic representation theory that bases on the referring phenomena
(which are dominant in dialogue). Some research directions are
proposed in order to extend the multimodal reference domains model
into such a theory. |
27. |
Landragin, F., Influence
de la situation lors de la résolution des anaphores dans le
dialogue, Treizième conférence sur le traitement
automatique des langues (TALN 2006), Leuven, Belgique, 2006
(10 pages) |
Starting with the notion of salience and a study
of the events that intervene during dialogue, this study consists
of a characterization of exophora, i.e., anaphora with no
linguistic antecedent but situational antecedent. A method for
confronting linguistic and situational parameters is proposed. |
28. |
Sedogbo, C., Grisvard, O.,
Landragin, F., Lard, J. & Praud, S., HMI Engineering
Productivity: the Poor Child of MDE/MDA Trends. A Vision for
Model-Driven Human-Computer Interaction Engineering,
Dix-huitième Conférence Francophone sur l'Interaction
Homme-Machine (Model-Driven Engineering and Human-Computer
Interaction Workshop at IHM 2006), Montréal, Canada,
2006 (2 pages) |
With the statement that it is more and more
difficult to design multimodal dialogue systems (due to the
increasing complexity of the phenomena we want to address and the
increasing complexity of software production), a new method to
design systems is proposed. It exploits the principles of
model-driven engineering and model-driven architecture (MDE/MDA).
The method consists of specifying models for all runtime
modules, so that the modules are automatically derived from the
models. The main principles of the approach are presented in
this abstract and are applied within the EMODE project. |
29. |
Landragin, F., Physical,
Semantic and Pragmatic Levels for Multimodal Fusion and Fission,
Seventh International Workshop on Computational Semantics
(IWCS-7), Tilburg, The Netherlands, 2007 (5 pages) |
Synthesis on the possible decompositions for the
processes for multimodality management, including information fusion
and fission. Proposition of three levels (physical, semantic, and
pragmatic) with the same kinds of processes in input as well as in
output. Statement on the notion of symmetric multimodality. |
30. |
Landragin, F., Taking
Situational Factors into Account when Resolving Anaphora: an
Approach based on Events and Salience, Sixth Discourse Anaphora
and Anaphor Resolution Colloquium (DAARC 2007), Lagos, Portugal,
2007 (6 pages) |
English version of publication 27, with some
additional details dealing with the notion of salience, and some
corrections following the comments from the reviewers. |
31. |
Landragin, F., Un exemple de
polysémie du geste co-verbal en situation de communication
homme-machine et ses conséquences sur les analyses
sémantiques et pragmatiques, Deuxième Colloque
International de l'Association Française de Linguistique
Cognitive (AFLiCo 2), Lille, France, 2007 (2 pages) |
This presentation is a kind of statement dealing
with the famous put-that-there example that is at the origin of a
lot of work on multimodal systems. Co-verbal gesture is introduced
with various forms, and in particular with the form of a arc of a
circle in order to illustrate the moving characteristics of the
action. The multimodal fusion process is presented so that the
importance of the task is emphasized, for instance when it puts
forward some fusion hypotheses and therefore some interpretations. |
32. |
Landragin, F., L'anaphore à
antécédent flou : une caractérisation et ses
conséquences sur l'annotation des relations anaphoriques,
Journée d'étude de l'Association pour le Traitement
Automatique des LAngues (ATALA) sur la résolution des
anaphores, Paris, France, 2007 (2 pages) |
Following underspecified semantics, study of some
anaphora phenomena, where the precise identification of the antecedent
is useless (sometimes impossible). For instance, a pronoun may refer
to one antecedent or another with no consequences on the global
interpretation of the utterance or text. Introduction of the notion
of anaphora with fuzzy antecedent, and characterization of the types
of potential fuzzy antecedents. Technical consequences on the
annotation of anaphoric relations. |