Roman Danylak - Academia.edu (original) (raw)
Papers by Roman Danylak
University of Technology, Sydney. Faculty of Information Technology.NO FULL TEXT AVAILABLE. This ... more University of Technology, Sydney. Faculty of Information Technology.NO FULL TEXT AVAILABLE. This thesis contains 3rd party copyright material. The hardcopy may be available for consultation at the UTS Library.NO FULL TEXT AVAILABLE. This thesis contains 3rd party copyright material. ----- A Semiotic Analysis of Gesture and Emotion in Artistic Human-Computer Interaction In face to face human communication bodily gestures may play an expressive role, transmitting information on how the sender of the message feels, commonly known as body language. A key feature of such communication is the attachment of emotions to gestures. In Multimodal Human-Computer Interaction (MMHCI) interfaces are capable of detecting human gesture. The key focus of this research is to create an interactive artwork where the interactor experience generates gestures with associated emotions, focusing on a part of expressive gestural face to face communication. This interaction will be termed Artistic Human-Computer Interaction (AHCI). The practice of Interactive Film-Making serves as the foundation for evolving a design and research method. The design process is based on expressive gesture production in theatre where ‘natural’ face to face communication is synthesised by actors to produce the dramatic artform. Semiotic analysis is applied both to gestural theatre processes and to the multimodal interface elements creating a matrix, enabling the design. The interactive artwork takes the form of an Interactive Gestural Film Game where narrative film content is assembled through gestural action. The subjects are from a teenage, urban, English-speaking audience. The work is evaluated for gesture response and associated emotions as well as for coherency and engagement with the artwork
The discussion that follows is on the use ofStanislavskian text-based theatrical performance tomo... more The discussion that follows is on the use ofStanislavskian text-based theatrical performance tomodel an interactive design for whole body interaction.The focus is on the processes by which actors anddirectors create repeatable, gestural paradigms forexpressive performance in theatre. Three key phases -text, rehearsal and performance - as guided by directorialinterpretation, are refigured into a game format. Thedesign application and evaluation is demonstrated in theinteractive gestural film game To be or not to be (see http://www.Betaspace.Net.Au/Content/View/36/)Critical to the success of the model is a workabledefinition of gesture evolved from yogic exercise, atechnique used by Grotowski to prepare actors forperformance. Author Keywords Theatre, text, rehearsal, performance, gesture, game. ACM Classification Keywords H5.m. Information interfaces and presentation INTRODUCTION Whole body human-computer interaction brings thegestures of the human body to the forefront of interaction...
Intuition is the instantaneous knowledge that we possess of what is present in a perceived enviro... more Intuition is the instantaneous knowledge that we possess of what is present in a perceived environment, which includes the possible intentions of others. In human to human interaction, we intuitively respond to each other's desires and moods by reading 'body language', that is, we are responding to gestures. There has been considerable work done in recent years by technologists on the possibilities of computer recognition of this language, the principle being, that a machine could respond to human emotions rather than intellectual states. Semiotics, the science of signs and symbols, offers a precise grammar by which gestures can be interpreted; the core elements are metaphor, metonymy and utterance, which is the expression and interpretation of images in a sequential narrative structure. Significantly, the same analytical tools can be applied to components of computer interfaces, namely sound, image and text. What results is a set of distinctions and parallels between ge...
The act of naming, where sign and the signified are coupled as an act of touching, establishes th... more The act of naming, where sign and the signified are coupled as an act of touching, establishes the foundations for the meaningful use of language. The computer, a language machine, possesses the capacity to input sensory data from the physical environment where signification occurs. To design a computationally extended sensory environment with intuitive gestural interaction will necessarily then have touch as a foundational factor. The universal element in such systems is language; the specific is the context of place, a space where the significating action of touch occurs.
The design of gesture-based interactive systems, where an emotional response may be potentially e... more The design of gesture-based interactive systems, where an emotional response may be potentially evoked, is a substantial and enduring challenge for designers of interactive systems. The discussion to follow looks at the use of semiotics to analyse expressive gesture generation in theatre, yoga, and multimodal human-computer interaction to establish design foundations, these being interpretable, ephemeral, and programmable elements. A description of an interactive gestural film game To be or not to be is included as an appendix showing the application of the elements.
This paper describes practice-based research using an interactive gestural film game To be or not... more This paper describes practice-based research using an interactive gestural film game To be or not to be, to study the relationship between performed gestures and emotional response. The work is the result of a collaboration between an intermedia artist (first author), and a programmer / designer. The content of the interactive film game is based on lines from Act 3 scene i from Shakespeare's play Hamlet. Attention is given first to the original design method based on gestural expression in theatre using the key techniques of text, rehearsal and performance. A primary design goal was to construct an interface where coherent and continuous gestural and emotional interaction occurred. This is followed by observation and evaluation of audience / user interaction with the work (collaboration with second author) establishing research criteria to evaluate performed gestures and emotional responses of the audience.
78 New Trends in Software Methodologies, Tools and Techniques H. Fujita and P. Johannesson (Eds. ... more 78 New Trends in Software Methodologies, Tools and Techniques H. Fujita and P. Johannesson (Eds. l IOS Press, 2003 Short paper Gesture and Intuition ROMAN DANYLAK Creativity and Cognition Studios University of Technology, Sydney Abstract Intuition is the instantaneous ...
Theatrical Process as a Model for Full Body Gestural and Emotional Human-Computer Interaction Des... more Theatrical Process as a Model for Full Body Gestural and Emotional Human-Computer Interaction Design and Evaluation
Lecture Notes in Computer Science, 2014
The focus of this paper is to attempt to define some key qualities of what will be described as i... more The focus of this paper is to attempt to define some key qualities of what will be described as interactive film. Interactive film is the description of user interface experience, where the screen surface is seen as a continuous piece of film altered by data interactions, which are but edits to the film observed on screen. As such computer screen representations as a user experience are seen as the convergence of the medium of film with computer databases. The paper also examines the history of two other media, namely the book and painting over the last five hundred years as examples to understand how media-its form-and its content-the message behave, leading to interactive film.
Lecture Notes in Computer Science, 2013
The discussion that follows describes the design of an interactive artwork To be or not to be usi... more The discussion that follows describes the design of an interactive artwork To be or not to be using semiotics. The goal of To be or not to be was to create a user experience that was coherent and continuous, generating gestures and emotions-not uncommon Human Computer Interaction (HCI) objectives. The design problem was to isolate what could be elements of user experience as inputs and outputs in a multimedia interactive system. Essentially there were five parts to the process: the first was an understanding of gesture and its modeling within the framework of generating expressive gestures in theatre-defining media before simulation; the second, was to define the input and output process by which gestural interaction using HCI media might proceed; the third was to create a semiotic matrix of both the theatrical and HCI terms as equivalences, creating a system by which the design could follow; the fourth was evolving an experience, in this case an interactive film-game, that generated gestures and associated emotional content; the fifth was a user evaluation and statistical analysis (results summary only). The emphasis presented here is on the preparatory stage of correct process modeling, leading to the effective application of semiotic analysis. Readers are encouraged to access the URL for youtube description of the work.
International Journal of Arts and Technology, 2012
Human bodily gestures are now clearly present as a factor in human? computer interface design ext... more Human bodily gestures are now clearly present as a factor in human? computer interface design extending interactor experiences into designed space, in particular as interactive film, where the interactor may have gestural and emotional experiences with on screen material. This paper describes the organisation of time and space in three media forms where gestures are prominent, namely theatre, film and interactive film, showing their interrelation, where interactive film absorbs processes of the first two media and is ...
Somet, 2008
Abstract. Human bodily gestures are now clearly present as a factor in humancomputer interface de... more Abstract. Human bodily gestures are now clearly present as a factor in humancomputer interface design, used in commercially available game interfaces such as Sony's Eye Toy. However, many challenges still exist for this as yet new process, including the development of design strategies to further enable potential. This paper describes the organisation of time and space in three media forms where gestures are prominent, namely theatre, film and interactive film, showing their interrelation and aspects of technological innovation. The ...
Lecture Notes in Computer Science, 2014
With rise of the computer since the 1950's there has been a constant evolution for ways to furthe... more With rise of the computer since the 1950's there has been a constant evolution for ways to further enable calculating capacities. It is ironic, that a machine derived with such thoughtful and sophisticated process mathematically, has found a key and increasingly dominant expression in games, which has a reputation for a lack of seriousness. Games continue to rise as a very popular form of interaction. The paper examines the notion of language as a game to partly explain the phenomenon. The paper reflects upon the philosophical writings of Wittgenstein who proposed that language is a game. In addition semiotic analysis, a linguistic meta-tool, will be applied to a game with the aim of shedding light on the usefulness of semiotics in user interaction design.
Multidisciplinary Approaches, 2011
... Danylak, Roman. "Gestural Motivation, Learning and Evaluation using Interact... more ... Danylak, Roman. "Gestural Motivation, Learning and Evaluation using Interactive Game Design." In Handbook of Research on Improving Learning and Motivation through Educational Games: Multidisciplinary Approaches, ed. Patrick Felicia, 512-528 (2011), accessed October ...
The virtual world that computation now presents to us and involves us in, otherwise known as digi... more The virtual world that computation now presents to us and involves us in, otherwise known as digitisation - collapses the 'self; personal identity to which we have become accustomed to in the last four hundred years, is challenged by the illusion of many selves, transforming our experience of the usual. Multimedia presents us with dimensional multiplicity meaning that we are no longer forced to be just John Smith, our namesake. We have a possibility of playing in an infinite game where we can be many other beings, other types of people, rather than staying as 'who' we are. This has significant consequences for the interface design notion of the single user. The combination of separate, unique, historically evolved, representational technologies, best stated as counterfeit, production and simulation, distinctions established by Baudrillard[1] allows the diverse self to evolve in a digitised environment. This paper will focus on Spielberg's film Catch Me if You Can, th...
There is currently intense interest in the potential of gestural expression as input for human --... more There is currently intense interest in the potential of gestural expression as input for human --computer interaction, resembling the role that gesture plays in human to human communication. An interactive artwork, To be or not to be, designed to research gesture and emotional response, was created using a theatrical model of expressive gesture. Excerpts from the works of Augustine and Kant clarify the cognitive categories present in the theatrical model, whilst Wittgenstein's notion of language as a game, establishes the concept of the artwork. The interactor experience was evaluated for coherency and gestural engagement.
The discussion is focused on the experience of selfhood in current interactive formats. The self,... more The discussion is focused on the experience of selfhood in current interactive formats. The self, it is proposed, evolves from reflection within media forms such as textual narratives and image depictions and as such is a historical construction that changes as new media evolve. Terminology that is used to described communication with interactive systems, in particular Weiner's notion of the 'cyborg' and Arseth's notion of game-based 'ergodic literature' are discussed, as is the idea of the multiple game playing self. Artificial intelligence is also briefly discussed.
The discussion that follows is on the use of Stanislavskian text-based theatrical performance to ... more The discussion that follows is on the use of Stanislavskian text-based theatrical performance to model an interactive design for whole body interaction. The focus is on the processes by which actors and directors create repeatable, gestural paradigms for expressive performance in theatre. Three key phases -text, rehearsal and performance -as guided by directorial interpretation, are refigured into a game format. The design application and evaluation is demonstrated in the interactive gestural film game To be or not to be (s e e http://www.Betaspace.Net.Au/Content/View/36/) Critical to the success of the model is a workable definition of gesture evolved from yogic exercise, a technique used by Grotowski to prepare actors for performance.
University of Technology, Sydney. Faculty of Information Technology.NO FULL TEXT AVAILABLE. This ... more University of Technology, Sydney. Faculty of Information Technology.NO FULL TEXT AVAILABLE. This thesis contains 3rd party copyright material. The hardcopy may be available for consultation at the UTS Library.NO FULL TEXT AVAILABLE. This thesis contains 3rd party copyright material. ----- A Semiotic Analysis of Gesture and Emotion in Artistic Human-Computer Interaction In face to face human communication bodily gestures may play an expressive role, transmitting information on how the sender of the message feels, commonly known as body language. A key feature of such communication is the attachment of emotions to gestures. In Multimodal Human-Computer Interaction (MMHCI) interfaces are capable of detecting human gesture. The key focus of this research is to create an interactive artwork where the interactor experience generates gestures with associated emotions, focusing on a part of expressive gestural face to face communication. This interaction will be termed Artistic Human-Computer Interaction (AHCI). The practice of Interactive Film-Making serves as the foundation for evolving a design and research method. The design process is based on expressive gesture production in theatre where ‘natural’ face to face communication is synthesised by actors to produce the dramatic artform. Semiotic analysis is applied both to gestural theatre processes and to the multimodal interface elements creating a matrix, enabling the design. The interactive artwork takes the form of an Interactive Gestural Film Game where narrative film content is assembled through gestural action. The subjects are from a teenage, urban, English-speaking audience. The work is evaluated for gesture response and associated emotions as well as for coherency and engagement with the artwork
The discussion that follows is on the use ofStanislavskian text-based theatrical performance tomo... more The discussion that follows is on the use ofStanislavskian text-based theatrical performance tomodel an interactive design for whole body interaction.The focus is on the processes by which actors anddirectors create repeatable, gestural paradigms forexpressive performance in theatre. Three key phases -text, rehearsal and performance - as guided by directorialinterpretation, are refigured into a game format. Thedesign application and evaluation is demonstrated in theinteractive gestural film game To be or not to be (see http://www.Betaspace.Net.Au/Content/View/36/)Critical to the success of the model is a workabledefinition of gesture evolved from yogic exercise, atechnique used by Grotowski to prepare actors forperformance. Author Keywords Theatre, text, rehearsal, performance, gesture, game. ACM Classification Keywords H5.m. Information interfaces and presentation INTRODUCTION Whole body human-computer interaction brings thegestures of the human body to the forefront of interaction...
Intuition is the instantaneous knowledge that we possess of what is present in a perceived enviro... more Intuition is the instantaneous knowledge that we possess of what is present in a perceived environment, which includes the possible intentions of others. In human to human interaction, we intuitively respond to each other's desires and moods by reading 'body language', that is, we are responding to gestures. There has been considerable work done in recent years by technologists on the possibilities of computer recognition of this language, the principle being, that a machine could respond to human emotions rather than intellectual states. Semiotics, the science of signs and symbols, offers a precise grammar by which gestures can be interpreted; the core elements are metaphor, metonymy and utterance, which is the expression and interpretation of images in a sequential narrative structure. Significantly, the same analytical tools can be applied to components of computer interfaces, namely sound, image and text. What results is a set of distinctions and parallels between ge...
The act of naming, where sign and the signified are coupled as an act of touching, establishes th... more The act of naming, where sign and the signified are coupled as an act of touching, establishes the foundations for the meaningful use of language. The computer, a language machine, possesses the capacity to input sensory data from the physical environment where signification occurs. To design a computationally extended sensory environment with intuitive gestural interaction will necessarily then have touch as a foundational factor. The universal element in such systems is language; the specific is the context of place, a space where the significating action of touch occurs.
The design of gesture-based interactive systems, where an emotional response may be potentially e... more The design of gesture-based interactive systems, where an emotional response may be potentially evoked, is a substantial and enduring challenge for designers of interactive systems. The discussion to follow looks at the use of semiotics to analyse expressive gesture generation in theatre, yoga, and multimodal human-computer interaction to establish design foundations, these being interpretable, ephemeral, and programmable elements. A description of an interactive gestural film game To be or not to be is included as an appendix showing the application of the elements.
This paper describes practice-based research using an interactive gestural film game To be or not... more This paper describes practice-based research using an interactive gestural film game To be or not to be, to study the relationship between performed gestures and emotional response. The work is the result of a collaboration between an intermedia artist (first author), and a programmer / designer. The content of the interactive film game is based on lines from Act 3 scene i from Shakespeare's play Hamlet. Attention is given first to the original design method based on gestural expression in theatre using the key techniques of text, rehearsal and performance. A primary design goal was to construct an interface where coherent and continuous gestural and emotional interaction occurred. This is followed by observation and evaluation of audience / user interaction with the work (collaboration with second author) establishing research criteria to evaluate performed gestures and emotional responses of the audience.
78 New Trends in Software Methodologies, Tools and Techniques H. Fujita and P. Johannesson (Eds. ... more 78 New Trends in Software Methodologies, Tools and Techniques H. Fujita and P. Johannesson (Eds. l IOS Press, 2003 Short paper Gesture and Intuition ROMAN DANYLAK Creativity and Cognition Studios University of Technology, Sydney Abstract Intuition is the instantaneous ...
Theatrical Process as a Model for Full Body Gestural and Emotional Human-Computer Interaction Des... more Theatrical Process as a Model for Full Body Gestural and Emotional Human-Computer Interaction Design and Evaluation
Lecture Notes in Computer Science, 2014
The focus of this paper is to attempt to define some key qualities of what will be described as i... more The focus of this paper is to attempt to define some key qualities of what will be described as interactive film. Interactive film is the description of user interface experience, where the screen surface is seen as a continuous piece of film altered by data interactions, which are but edits to the film observed on screen. As such computer screen representations as a user experience are seen as the convergence of the medium of film with computer databases. The paper also examines the history of two other media, namely the book and painting over the last five hundred years as examples to understand how media-its form-and its content-the message behave, leading to interactive film.
Lecture Notes in Computer Science, 2013
The discussion that follows describes the design of an interactive artwork To be or not to be usi... more The discussion that follows describes the design of an interactive artwork To be or not to be using semiotics. The goal of To be or not to be was to create a user experience that was coherent and continuous, generating gestures and emotions-not uncommon Human Computer Interaction (HCI) objectives. The design problem was to isolate what could be elements of user experience as inputs and outputs in a multimedia interactive system. Essentially there were five parts to the process: the first was an understanding of gesture and its modeling within the framework of generating expressive gestures in theatre-defining media before simulation; the second, was to define the input and output process by which gestural interaction using HCI media might proceed; the third was to create a semiotic matrix of both the theatrical and HCI terms as equivalences, creating a system by which the design could follow; the fourth was evolving an experience, in this case an interactive film-game, that generated gestures and associated emotional content; the fifth was a user evaluation and statistical analysis (results summary only). The emphasis presented here is on the preparatory stage of correct process modeling, leading to the effective application of semiotic analysis. Readers are encouraged to access the URL for youtube description of the work.
International Journal of Arts and Technology, 2012
Human bodily gestures are now clearly present as a factor in human? computer interface design ext... more Human bodily gestures are now clearly present as a factor in human? computer interface design extending interactor experiences into designed space, in particular as interactive film, where the interactor may have gestural and emotional experiences with on screen material. This paper describes the organisation of time and space in three media forms where gestures are prominent, namely theatre, film and interactive film, showing their interrelation, where interactive film absorbs processes of the first two media and is ...
Somet, 2008
Abstract. Human bodily gestures are now clearly present as a factor in humancomputer interface de... more Abstract. Human bodily gestures are now clearly present as a factor in humancomputer interface design, used in commercially available game interfaces such as Sony's Eye Toy. However, many challenges still exist for this as yet new process, including the development of design strategies to further enable potential. This paper describes the organisation of time and space in three media forms where gestures are prominent, namely theatre, film and interactive film, showing their interrelation and aspects of technological innovation. The ...
Lecture Notes in Computer Science, 2014
With rise of the computer since the 1950's there has been a constant evolution for ways to furthe... more With rise of the computer since the 1950's there has been a constant evolution for ways to further enable calculating capacities. It is ironic, that a machine derived with such thoughtful and sophisticated process mathematically, has found a key and increasingly dominant expression in games, which has a reputation for a lack of seriousness. Games continue to rise as a very popular form of interaction. The paper examines the notion of language as a game to partly explain the phenomenon. The paper reflects upon the philosophical writings of Wittgenstein who proposed that language is a game. In addition semiotic analysis, a linguistic meta-tool, will be applied to a game with the aim of shedding light on the usefulness of semiotics in user interaction design.
Multidisciplinary Approaches, 2011
... Danylak, Roman. "Gestural Motivation, Learning and Evaluation using Interact... more ... Danylak, Roman. "Gestural Motivation, Learning and Evaluation using Interactive Game Design." In Handbook of Research on Improving Learning and Motivation through Educational Games: Multidisciplinary Approaches, ed. Patrick Felicia, 512-528 (2011), accessed October ...
The virtual world that computation now presents to us and involves us in, otherwise known as digi... more The virtual world that computation now presents to us and involves us in, otherwise known as digitisation - collapses the 'self; personal identity to which we have become accustomed to in the last four hundred years, is challenged by the illusion of many selves, transforming our experience of the usual. Multimedia presents us with dimensional multiplicity meaning that we are no longer forced to be just John Smith, our namesake. We have a possibility of playing in an infinite game where we can be many other beings, other types of people, rather than staying as 'who' we are. This has significant consequences for the interface design notion of the single user. The combination of separate, unique, historically evolved, representational technologies, best stated as counterfeit, production and simulation, distinctions established by Baudrillard[1] allows the diverse self to evolve in a digitised environment. This paper will focus on Spielberg's film Catch Me if You Can, th...
There is currently intense interest in the potential of gestural expression as input for human --... more There is currently intense interest in the potential of gestural expression as input for human --computer interaction, resembling the role that gesture plays in human to human communication. An interactive artwork, To be or not to be, designed to research gesture and emotional response, was created using a theatrical model of expressive gesture. Excerpts from the works of Augustine and Kant clarify the cognitive categories present in the theatrical model, whilst Wittgenstein's notion of language as a game, establishes the concept of the artwork. The interactor experience was evaluated for coherency and gestural engagement.
The discussion is focused on the experience of selfhood in current interactive formats. The self,... more The discussion is focused on the experience of selfhood in current interactive formats. The self, it is proposed, evolves from reflection within media forms such as textual narratives and image depictions and as such is a historical construction that changes as new media evolve. Terminology that is used to described communication with interactive systems, in particular Weiner's notion of the 'cyborg' and Arseth's notion of game-based 'ergodic literature' are discussed, as is the idea of the multiple game playing self. Artificial intelligence is also briefly discussed.
The discussion that follows is on the use of Stanislavskian text-based theatrical performance to ... more The discussion that follows is on the use of Stanislavskian text-based theatrical performance to model an interactive design for whole body interaction. The focus is on the processes by which actors and directors create repeatable, gestural paradigms for expressive performance in theatre. Three key phases -text, rehearsal and performance -as guided by directorial interpretation, are refigured into a game format. The design application and evaluation is demonstrated in the interactive gestural film game To be or not to be (s e e http://www.Betaspace.Net.Au/Content/View/36/) Critical to the success of the model is a workable definition of gesture evolved from yogic exercise, a technique used by Grotowski to prepare actors for performance.