Abstract: This paper details a method of collecting video and audio recordings of people inter- acting with a simple robot interlocutor. The interaction is recorded via a number of cameras and microphones mounted on and around the robot. The system utilised a number of technologies to engage with interlocutors including OpenCV, Python, and Max MSP. Interactions over a three month period were collected at The Science Gallery in Trinity College Dublin. Visitors to the gallery freely engaged with the robot, with interactions on their behalf being spontaneous and non-scripted. The robot dialogue was a set pattern of utterances to engage interlocutors in a simple conversation. A large number of audio and video recordings were collected over a three month period.

Keywords: human-robot interaction; multi-modal data collection; audio interface; platform-robot; WOZ, face detection

Abstract: Non-verbal communication is important in order to maintain fluency of communica- tion. Gestures, facial expressions, and eye-gazing function as efficient means to convey feedback and provide subtle cues to control and organise conversations. In this article, we discuss the relation of verbal and non-verbal feedback from the point of view of communicative activity, and focus especially on hand gestures and body movement in the coordination of the interaction. This is called “Visual Interaction Management”. Combining the top-down approach, i.e. man- ual annotation and analysis of the data, with the bottom-up analysis of the speech and visual signals, we can visualize the speakers’ speech and gesture activity, and align this with those ges- tures and body movements that the interlocutors interpret as communicatively important. As the method for the bottom-up analysis, we use Echo State Networks, an architecture for recurrent neural networks, and use it in the recognition analysis of communicative behaviour patterns.

Keywords: Embodied Communicative Activity; Naturalistic Interaction; Interaction Management

Abstract: In telemanipulation and 3D virtual interactions it is important to transmit force sen- sation from the remote or virtual environment to the operator. Due to the weak points (control issues, robustness, cost) of real force feedback devices, methods where force is rendered on non-native sensory channels have grounds. In this paper, a survey of the related literature is presented and the concept of sensor-bridging type cognitive infocommunications based force reflecting schemes is discussed. A complete experimental infrastructure with hardware and soft- ware components is built providing a background for the investigation of the proposed methods from practical usability aspects. This environment is utilized in a pilot experiment with human participants providing substantial observations on the usability of sensor-bridging type vibro- tactile force feedback methods. The test confirms that vibrotactile glove equipped with shaftless vibration motors can be successfully applied as tactile/haptic feedback device in immersive vir- tual reality applications.

Keywords: cognitive infocommunications, telemanipulation, force-feedback, vibrotactile feedback

Abstract: In this paper, we provide the finalized definition of Cognitive Infocommunications (CogInfoCom). Following the definition, we briefly describe the scope and goals of CogInfoCom, and discuss the common interests between CogInfoCom and the various research disciplines which contribute to this new field in a synergistic way.

Keywords: Cognitive Infocommunications, CogInfoCom channels

Abstract: Due to its multidisciplinary origins, the elementary concepts and terminology of Cognitive Infocommunications (CogInfoCom) would ideally be derived – among others – from the fields of informatics, infocommunications and cognitive sciences. The terminology used by these fields in relation to CogInfoCom are disparate not only because the same concepts are often referred to using different terms, but also because in many cases the same terms can refer to different concepts. For this reason, we propose a set of concepts to help unify the CogInfoCom-related aspects in these fields, from the unique viewpoint necessary for the human-oriented analysis and synthesis of CogInfoCom channels. Examples are given to illustrate how the terminology presents itself in engineering applications.

Keywords: Cognitive Infocommunications, CogInfoCom channels, icon layer, message layer, conceptual mapping

Abstract: The concept of ‘Future Internet’, ‘Internet of Things’ and ‘3D Internet’ opens a novel way for modeling ethological tests by rebuilding models of human-animal interaction in an augmented environment as an interactive mixture of virtual actors and real human observers. On the one hand these experiments can serve as a proof of concept, as a kind of experimental validation of formal ethological models, but on the other hand they can also serve as examples for the ways a human can communicate with things (i.e., with everyday objects) in a virtual environment (e.g. on the Internet). These kinds of experiments can also support Cognitive Infocommunication related research, the field that investigates how a human can co-evolve with artificially cognitive systems through infocommunications devices. The goal of the paper is to introduce an example for such an ethological test system, a possible way for embedding a prototype ethological model described as a fuzzy automaton in MATLAB to the 3D VirCA collaborative augmented reality environment. Some details of the applied ethological experiment paradigm developed for studying the dog-owner relationship in a standard laboratory procedure, as a demonstrative example for ethological model implementation, will also be discussed briefly in this paper.

Keywords: Cognitive Infocommunication; VirCA; modeling ethological experiments

Abstract: Spatial memory is a system to manage digital information in real space using three- dimensional position information and to operate networked devices to deliver the digital in- formation. The concept behind the system is Intelligence Space (iSpace), which is proposed as an intelligent environment. Using distributed intelligence networked devices (DINDs), iSpace can observe events in the real world, and provide information and physical services to users via intelligent agents such as mobile robots, computer devices, and digital equipment based on what has been observed. From this, spatial memory is regarded as a human interface for iSpace. Users of iSpace can store commands and information in the real environment according to their activities, and retrieve necessary information using their own bodies without using any devices such as a mouse and a keyboard. This paper presents an enhancement of the spatial memory to improve the usability of the system and to be applicable in wider areas.

Keywords: Spatial Memory, Intelligent Space, Observation of Human Activity, Human Interface

Abstract: This paper presents a new method for the characterization of the perspicuity of user interfaces based on users’ navigation structures and reaction times. The user interface of a web page or software is good if users easily find information that they need. Therefore, it seems to be logical to give search tasks to the users and characterize the perspicuity of a user interface on the basis of the search time and navigation structure of the users. Experiments started with tasks where users did not have to read; they searched among 2D and 3D geometrical objects and among pictures. The experiment investigated which areas of the screen were preferred. These experiments were followed by tasks where the participant had to read as well. Here information was organized in columns because on web pages and news portals, the user is often confronted with layouts in columns and must search textual information ordered in columns. These tasks investigated how multi-column layout guides visual attention and how settings can help or disturb users to find information they need, settings such as the number of columns, target positions, word length, meaningful/meaningless context, colour of the fore- and background, size of the area which had to be scanned.

Keywords: web usability; user interface; navigation; navigation strategy; web page layout

Abstract: The present work aims to examine emotion recognition within and across cultures. It reports on a set of perceptual experiments designed to explore the human ability to identify emotional expressions dynamically presented through visual and auditory channels. Two cross-modal databases of dynamic verbal and non-verbal emotional stimuli based on video-clips extracted from American and Italian movies, respectively, were defined and exploited for the experiments. In the first study, American, French, and Italian subjects were involved in a comparative analysis of subjective perceptions of six emotional states dynamically portrayed by visual and vocal cues, exploiting the database of American emotional stimuli. In the second study, American and Italian subjects were tested on their ability to recognize six emotional states through the visual and auditory channel, exploiting the database of Italian emotional stimuli. The aim is to investigate if there exists a difference in the efficacy of the visual and auditory channels to infer emotional information and if cultural context, in particular the language, may influence this difference. This hypothesis is investigated including as participants in each of the two studies, one group of native speakers of the language, belonging to the same cultural context of the video-clips used as stimuli (i.e., American subjects for the American stimuli in the former study, and Italian subjects for the Italian stimuli in the latter one). Results showed that emotional information is affected by the communication mode and that language plays a role.

Keywords: emotion recognition; vocal expressions; facial expressions; cultural differences

Abstract: Information and communication technologies (ICT) and cognitive sciences (CS) are both pursued at the Budapest University of Technology and Economics (BME). The Department of the Telecommunications and Media Informatics (TMIT) is focusing on ICT and its related applications and convergence issues. After deploying the convergence of the telecommunications, information and media technologies based on the evolution and common use of the digital technology, the widening convergence process embraced CS, and resulted in the establishment of cognitive infocommunications as a new scientific discipline at the end of the last decade. Contents involved in the convergence process are expanded, the information value chains are merged and also expanded, and the convergent ICT applications are completed by cognitive features.

Keywords: cognitive sciences; cognitive infocommunications; information and communication technologies; digital convergence; convergent applications

Abstract: Manufacturing equipment for small and medium scale production is often arranged in a setup of a Flexible Manufacturing Cell (FMC). Normally, such cell members are from various manufactures and they have all their specific capabilities when it comes to man-machine and inter-machine interaction. Existing man-machine interfacing is quite simple and is typically done via a screen/keyboard interface while inter-machine communication is carried out through designated I/O lines. Although such communication channels provide us a certain possibility to program and control the cell members, it is not enough to fully utilize the potential of the coordinated control of the complete unit. Especially in smaller production facilities, where often the FMC components are a mixture of older (poorly documented) and newer machines, there exists no ready-to-use solution for communication between the members of the cell. This article discusses and presents a new architecture for man-machine and inter-machine communication and control. This new architecture allows for the integration of both new and old equipment and is especially targeted for efficient man-machine interaction.

Keywords: shop- floor architecture; flexible manufacturing cells/systems; human-machine interaction

Abstract: This paper presents results concerning the observation process modelling in the framework of cognition processes which belong to a new pattern of human knowledge. The cultural origin of the patterns is analyzed in terms of philosophical, psychological and linguistic points of view. A scenario concerning a robot integrated in a cognitive system is given in order to test the theoretical approaches. The definitions of signatures and of signature classes are given as one of the first steps in an alternative modelling approach to the observation process. An example that deals with the observation process modelling is offered.

Keywords: cognition process; observation process modelling; signature classes; signatures

Abstract: We have designed and evaluated around 10 serious games under the EU Leonardo Transfer of Innovation Project: Game On Extra Time (GOET) project http://goet-project.eu/. The project supports people with learning disabilities and additional sensory impairments in getting and keeping a job by helping them to learn, via games-based learning, skills that will help them in their working day. These games help students to learn how to prepare themselves for work and for dealing with everyday situations at work, including money management, travelling independently, etc. This paper is concerned with the potential of serious games as effective and engaging learning resources for people with intellectual disabilities. In this paper we will address questions related to the design and evaluation of such games and our design solutions to suit the individual learning needs of our target audiences.

Keywords: intellectual disability; serious games; user interface testing