Search results for: gestures

Authors: Emre Akyürek, Anthony Huynh, Tatiana Kalganova

Abstract:

The Ambidextrous Robot Hand is a robotic device with the purpose to mimic either the gestures of a right or a left hand. The symmetrical behavior of its fingers allows them to bend in one way or another keeping a compliant and anthropomorphic shape. However, in addition to gestures they can reproduce on both sides, an asymmetrical mechanical design with a three tendons routing has been engineered to reduce the number of actuators. As a consequence, control algorithms must be adapted to drive efficiently the ambidextrous fingers from one position to another and to include grasping features. These movements are controlled by pneumatic muscles, which are nonlinear actuators. As their elasticity constantly varies when they are under actuation, the length of pneumatic muscles and the force they provide may differ for a same value of pressurized air. The control algorithms introduced in this paper take both the fingers asymmetrical design and the pneumatic muscles nonlinearity into account to permit an accurate control of the Ambidextrous Robot Hand. The finger motion is achieved by combining a classic PID controller with a phase plane switching control that turns the gain constants into dynamic values. The grasping ability is made possible because of a sliding mode control that makes the fingers adapt to the shape of an object before strengthening their positions.

Keywords: ambidextrous hand, intelligent algorithms, nonlinear actuators, pneumatic muscles, robotics, sliding control

Procedia PDF Downloads 297

Authors: Benjamin Bressolette, S´ebastien Denjean, Vincent Roussarie, Mitsuko Aramaki, Sølvi Ystad, Richard Kronland-Martinet

Abstract:

New sensors and technologies – such as microphones, touchscreens or infrared sensors – are currently making their appearance in the automotive sector, introducing new kinds of Human-Machine Interfaces (HMIs). The interactions with such tools might be cognitively expensive, thus unsuitable for driving tasks. It could for instance be dangerous to use touchscreens with a visual feedback while driving, as it distracts the driver’s visual attention away from the road. Furthermore, new technologies in car cockpits modify the interactions of the users with the central system. In particular, touchscreens are preferred to arrays of buttons for space improvement and design purposes. However, the buttons’ tactile feedback is no more available to the driver, which makes such interfaces more difficult to manipulate while driving. Gestures combined with an auditory feedback might therefore constitute an interesting alternative to interact with the HMI. Indeed, gestures can be performed without vision, which means that the driver’s visual attention can be totally dedicated to the driving task. In fact, the auditory feedback can both inform the driver with respect to the task performed on the interface and on the performed gesture, which might constitute a possible solution to the lack of tactile information. As audition is a relatively unused sense in automotive contexts, gesture sonification can contribute to reducing the cognitive load thanks to the proposed multisensory exploitation. Our approach consists in using a virtual object (VO) to sonify the consequences of the gesture rather than the gesture itself. This approach is motivated by an ecological point of view: Gestures do not make sound, but their consequences do. In this experiment, the aim was to identify efficient sound strategies, to transmit dynamic information of VOs to users through sound. The swipe gesture was chosen for this purpose, as it is commonly used in current and new interfaces. We chose two VO parameters to sonify, the hand-VO distance and the VO velocity. Two kinds of sound parameters can be chosen to sonify the VO behavior: Spectral or temporal parameters. Pitch and brightness were tested as spectral parameters, and amplitude modulation as a temporal parameter. Performances showed a positive effect of sound compared to a no-sound situation, revealing the usefulness of sounds to accomplish the task.

Keywords: auditory feedback, gesture sonification, sound perception, virtual object

Procedia PDF Downloads 302

Authors: Luisa Villani

Abstract:

The proposal aims to express concerns about the connection between mind, body, society, and environment in the Flyers’ dance, a very well-known rotatory dance in Mexico, to create meanings and to make the apprehension of the world possible. The interaction among the brain, mind, body, and environment, and the intersubjective relation among them, means the world creates and recreates a social interaction. The purpose of this methodology, based on the embodied cognition theory, which was named “A Performance-Embodied Theory” is to find the principles and patterns that organize the culture and the rules of the apprehension of the environment by Totonac people while the dance is being performed. The analysis started by questioning how anthropologists can interpret how Totonacs transform their unconscious knowledge into conscious knowledge and how the scheme formation of imagination and their collective imagery is understood in the context of public-facing rituals, such as Flyers’ dance. The problem is that most of the time, researchers interpret elements in a separate way and not as a complex ritual dancing whole, which is the original contribution of this study. This theory, which accepts the fact that people are body-mind agents, wants to interpret the dance as a whole, where the different elements are joined to an integral interpretation. To understand incorporation, data was recollected in prolonged periods of fieldwork, with participant observation and linguistic and extralinguistic data analysis. Laban’s notation for the description and analysis of gestures and movements in the space was first used, but it was later transformed and gone beyond this method, which is still a linear and compositional one. Performance in a ritual is the actualization of a potential complex of meanings or cognitive domains among many others in a culture: one potential dimension becomes probable and then real because of the activation of specific meanings in a context. It can only be thought what language permits thinking, and the lexicon that is used depends on the individual culture. Only some parts of this knowledge can be activated at once, and these parts of knowledge are connected. Only in this way, the world can be understood. It can be recognized that as languages geometrize the physical world thanks to the body, also ritual does. In conclusion, the ritual behaves as an embodied grammar or a text in movement, which, depending on the ritual phases and the words and sentences pronounced in the ritual, activates bits of encyclopedic knowledge that people have about the world. Gestures are not given by the performer but emerge from the intentional perception in which gestures are “understood” by the audio-spectator in an inter-corporeal way. The impact of this study regards the possibility not only to disseminate knowledge effectively but also to generate a balance between different parts of the world where knowledge is shared, rather than being received by academic institutions alone. This knowledge can be exchanged, so indigenous communities and academies could be together as part of the activation and the sharing of this knowledge with the world.

Keywords: dance, flyers, performance, embodied, cognition

Procedia PDF Downloads 59

Authors: Amira Benabdelkader

Abstract:

Communication presents the channel by which humankind create and maintain their relationship with others, express themselves, exchange information, learn and teach etc. The context of communication plays a distinctive role in deciding about the language to be used. The term context is mainly used to refer to the interlocutors, their cultures, languages, relationship, physical surrounding that is the communication setting, type of the information to be transmitted, the topic etc. Cultures, on one hand, impose on humans certain behaviours, attitudes, gestures and beliefs. On the other hand, the focus on language is inevitable as it is with its verbal and non-verbal components, a key tool in and for communication. Moreover, each language has its particularity in how people voice, address and express their thoughts, feelings and beliefs. Being in the same setting with people from different cultures and languages and having conversations with them would call upon the intercultural communicative competence. This latter would promote the success of their conversations. Additionally, this competence could manifest in several ways during their interactions, to the extent that no one can predict when and how the interlocutors would use it. The only thing probably that could be confirmed is that the setting and culture would in a way or another intervene and often shape the flow of their communication, if not the whole communication. Therefore, this paper will look at the intercultural communicative competence of language learners when introducing their cultures to each other in an online language tandem (henceforth OLT) using their second and/or foreign language with the L1 language speakers. The participants of this study are Algerian (use L2: French, FL: English), British (L1: English, L2/FL: French). In other words, this current paper will provide a qualitative analysis of the OLT experiment by emphasising how language learners can overcome the cultural differences in an intercultural setting while communicating online using Skype (video conversations) with people from different countries, cultures and L1. The non-verbal cues will have the lion share in the analysis by focusing on how they have been used to maintain this intercultural communication or hinder it through the misinterpretation of gestures, head movements, grimaces etc.

Keywords: intercultural communicative competence, non-verbal cues, online language tandem, Skype

Procedia PDF Downloads 283

Authors: Linghui Meng, James Atlas, Deborah Munro

Abstract:

There is an increasing demand for prosthetics capable of mimicking natural limb movements and hand gestures, but precise movement control of prosthetics using only electrode signals continues to be challenging. This study considers the implementation of machine learning as a means of improving accuracy and presents an initial investigation into hand gesture recognition using models based on electromyographic (EMG) signals. EMG signals, which capture muscle activity, are used as inputs to machine learning algorithms to improve prosthetic control accuracy, functionality and adaptivity. Using logistic regression, a machine learning classifier, this study evaluates the accuracy of classifying two hand gestures from the publicly available Ninapro dataset using two-time series feature extraction algorithms: Time Series Feature Extraction (TSFE) and Convolutional Neural Networks (CNNs). Trials were conducted using varying numbers of EMG channels from one to eight to determine the impact of channel quantity on classification accuracy. The results suggest that although both algorithms can successfully distinguish between hand gesture EMG signals, CNNs outperform TSFE in extracting useful information for both accuracy and computational efficiency. In addition, although more channels of EMG signals provide more useful information, they also require more complex and computationally intensive feature extractors and consequently do not perform as well as lower numbers of channels. The findings also underscore the potential of machine learning techniques in developing more effective and adaptive prosthetic control systems.

Keywords: EMG, machine learning, prosthetic control, electromyographic prosthetics, hand gesture classification, CNN, computational neural networks, TSFE, time series feature extraction, channel count, logistic regression, ninapro, classifiers

Procedia PDF Downloads 38

Authors: Teodora Vukovic, Christoph Hottiger, Noah Bubenhofer

Abstract:

The development of VIAN-DH aims at bridging two linguistic approaches: conversation analysis/interactional linguistics (IL), so far a dominantly qualitative field, and computational/corpus linguistics and its quantitative and automated methods. Contemporary IL investigates the systematic organization of conversations and interactions composed of speech, gaze, gestures, and body positioning, among others. These highly integrated multimodal behaviour is analysed based on video data aimed at uncovering so called “multimodal gestalts”, patterns of linguistic and embodied conduct that reoccur in specific sequential positions employed for specific purposes. Multimodal analyses (and other disciplines using videos) are so far dependent on time and resource intensive processes of manual transcription of each component from video materials. Automating these tasks requires advanced programming skills, which is often not in the scope of IL. Moreover, the use of different tools makes the integration and analysis of different formats challenging. Consequently, IL research often deals with relatively small samples of annotated data which are suitable for qualitative analysis but not enough for making generalized empirical claims derived quantitatively. VIAN-DH aims to create a workspace where many annotation layers required for the multimodal analysis of videos can be created, processed, and correlated in one platform. VIAN-DH will provide a graphical interface that operates state-of-the-art tools for automating parts of the data processing. The integration of tools that already exist in computational linguistics and computer vision, facilitates data processing for researchers lacking programming skills, speeds up the overall research process, and enables the processing of large amounts of data. The main features to be introduced are automatic speech recognition for the transcription of language, automatic image recognition for extraction of gestures and other visual cues, as well as grammatical annotation for adding morphological and syntactic information to the verbal content. In the ongoing instance of VIAN-DH, we focus on gesture extraction (pointing gestures, in particular), making use of existing models created for sign language and adapting them for this specific purpose. In order to view and search the data, VIAN-DH will provide a unified format and enable the import of the main existing formats of annotated video data and the export to other formats used in the field, while integrating different data source formats in a way that they can be combined in research. VIAN-DH will adapt querying methods from corpus linguistics to enable parallel search of many annotation levels, combining token-level and chronological search for various types of data. VIAN-DH strives to bring crucial and potentially revolutionary innovation to the field of IL, (that can also extend to other fields using video materials). It will allow the processing of large amounts of data automatically and, the implementation of quantitative analyses, combining it with the qualitative approach. It will facilitate the investigation of correlations between linguistic patterns (lexical or grammatical) with conversational aspects (turn-taking or gestures). Users will be able to automatically transcribe and annotate visual, spoken and grammatical information from videos, and to correlate those different levels and perform queries and analyses.

Keywords: multimodal analysis, corpus linguistics, computational linguistics, image recognition, speech recognition

Procedia PDF Downloads 110

Authors: Armita Ghobadi, Samantha Emerson, Seyda Ozcaliskan

Abstract:

Being a bilingual involves mastery of both speech and gesture patterns in a second language (L2). We know from earlier work in first language (L1) production contexts that speech and co-speech gesture form a tightly integrated system: co-speech gesture mirrors the patterns observed in speech, suggesting an online effect of language on nonverbal representation of events in gesture during the act of speaking (i.e., “thinking for speaking”). Relatively less is known about the online effect of language on gesture in bilinguals speaking structurally different languages. The few existing studies—mostly with small sample sizes—suggests inconclusive findings: some show greater achievement of L2 patterns in gesture with more advanced L2 speech production, while others show preferences for L1 gesture patterns even in advanced bilinguals. In this study, we focus on advanced bilingual speakers of two structurally different languages (Spanish L1 with English L2) in comparison to L1 English speakers. We ask whether bilingual speakers will follow target L2 patterns not only in speech but also in gesture, or alternatively, follow L2 patterns in speech but resort to L1 patterns in gesture. We examined this question by studying speech and gestures produced by 23 advanced adult Spanish (L1)-English (L2) bilinguals (Mage=22; SD=7) and 23 monolingual English speakers (Mage=20; SD=2). Participants were shown 16 animated motion event scenes that included distinct manner and path components (e.g., "run over the bridge"). We recorded and transcribed all participant responses for speech and segmented it into sentence units that included at least one motion verb and its associated arguments. We also coded all gestures that accompanied each sentence unit. We focused on motion event descriptions as it shows strong crosslinguistic differences in the packaging of motion elements in speech and co-speech gesture in first language production contexts. English speakers synthesize manner and path into a single clause or gesture (he runs over the bridge; running fingers forward), while Spanish speakers express each component separately (manner-only: el corre=he is running; circle arms next to body conveying running; path-only: el cruza el puente=he crosses the bridge; trace finger forward conveying trajectory). We tallied all responses by group and packaging type, separately for speech and co-speech gesture. Our preliminary results (n=4/group) showed that productions in English L1 and Spanish L1 differed, with greater preference for conflated packaging in L1 English and separated packaging in L1 Spanish—a pattern that was also largely evident in co-speech gesture. Bilinguals’ production in L2 English, however, followed the patterns of the target language in speech—with greater preference for conflated packaging—but not in gesture. Bilinguals used separated and conflated strategies in gesture in roughly similar rates in their L2 English, showing an effect of both L1 and L2 on co-speech gesture. Our results suggest that online production of L2 language has more limited effects on L2 gestures and that mastery of native-like patterns in L2 gesture might take longer than native-like L2 speech patterns.

Keywords: bilingualism, cross-linguistic variation, gesture, second language acquisition, thinking for speaking hypothesis

Procedia PDF Downloads 76

Authors: Rasha Ali Dheyab

Abstract:

In today's increasingly interconnected and technologically-driven world, communication has evolved beyond traditional verbal exchanges. This paper delves into the fascinating realm of multimodal communication, a dynamic field at the intersection of linguistics, gesture studies, and technology. The study of how humans convey meaning through a combination of spoken language, gestures, facial expressions, and digital platforms has gained prominence as our modes of interaction continue to diversify. This exploration begins by examining the foundational theories in linguistics and gesture studies, tracing their historical development and mutual influences. It further investigates the role of nonverbal cues, such as gestures and facial expressions, in augmenting and sometimes even altering the meanings conveyed by spoken language. Additionally, the paper delves into the modern technological landscape, where emojis, GIFs, and other digital symbols have emerged as new linguistic tools, reshaping the ways in which we communicate and express emotions. The interaction between traditional and digital modes of communication is a central focus of this study. The paper investigates how technology has not only introduced new modes of expression but has also influenced the adaptation of existing linguistic and gestural patterns in online discourse. The emergence of virtual reality and augmented reality environments introduces yet another layer of complexity to multimodal communication, offering new avenues for studying how humans navigate and negotiate meaning in immersive digital spaces. Through a combination of literature review, case studies, and theoretical analysis, this paper seeks to shed light on the intricate interplay between language, gesture, and technology in the realm of multimodal communication. By understanding how these diverse modes of expression intersect and interact, we gain valuable insights into the ever-evolving nature of human communication and its implications for fields ranging from linguistics and psychology to human-computer interaction and digital anthropology.

Keywords: multimodal communication, linguistics ., gesture studies., emojis., verbal communication., digital

Procedia PDF Downloads 82

Authors: Inkeri Jääskeläinen

Abstract:

The purpose of this study is to shed light on what is it like to study in apprenticeship training using Finnish as second language. This study examines the stories and experiences of apprenticeship students learning and studying Finnish as part of their vocational studies. Also, this pilot study examines the effects of learning to pronounce Finnish through body motions and gestures. Many foreign students choose apprenticeships and start vocational training too early, while their language skills in Finnish are still very weak. Both duties at work and school assignments require reasonably good general language skills (B1.1) and, especially at work, language skills are also a safety issue. At work students should be able to simultaneously learn Finnish and do vocational studies in a noisy, demanding, and stressing environment. Learning and understanding new things is very challenging under these circumstances and sometimes students get exhausted and experience a lot of stress - which makes learning even more difficult. Students are different from each other and so are their ways to learn. Thereafter, one of the most important features of apprenticeship training and second language learning is good understanding of adult learners and their needs. Kinaesthetic methods are an effective way to support adult students’ cognitive skills and make learning more relaxing and fun. Empirical findings show that language learning can indeed be supported physical ways, by body motions and gestures. The method used here, named TFFL (Touch and Feel Foreign Languages), was designed to support adult language learning, to correct or prevent language fossilization and to help the student to manage emotions. Finnish is considered as a difficult language to learn, mostly because it is so different from nearly all other languages. Many learners complain that they are lost or confused and there is a need to find a way to simultaneously learn the language and to handle negative emotion which come from Finnish language and the learning process itself. Due to the nature of Finnish language good pronunciation skills are needed just to understand the way the language work. Movements (body movements etc.) are a natural part of many cultures but not Finnish – In Finland students have traditionally been expected to stay still and that is not a natural way for many foreign students. However, kinaesthetic TFFL method proved out to be a useful way to help some L2 students to feel phonemes, rhythm and intonation, to improve their Finnish and, thereby, also to successfully complete their vocational studies.

Keywords: Finnish, fossilization, interference, kinaesthetic method

Procedia PDF Downloads 108

Authors: Inkeri Jaaskelainen

Abstract:

The purpose of this study is to shed light on what it is like to study in apprenticeship training using Finnish as a second language. This study examines the stories and experiences of apprenticeship students learning and studying Finnish as part of their vocational studies. Also, this pilot study examines the effects of learning to pronounce Finnish through body motions and gestures. Many foreign students choose apprenticeships and start vocational training too early, while their language skills in Finnish are still very weak. Both duties at work and school assignments require reasonably good general language skills (B1.1), and, especially at work, language skills are also a safety issue. At work, students should be able to simultaneously learn Finnish and do vocational studies in a noisy, demanding, and stressful environment. Learning and understanding new things is very challenging under these circumstances and sometimes students get exhausted and experience a lot of stress - which makes learning even more difficult. Students are different from each other and so are their ways to learn. Thereafter, one of the most important features of apprenticeship training and second language learning is a good understanding of adult learners and their needs. Kinaesthetic methods are an effective way to support adult students’ cognitive skills and make learning more relaxing and fun. Empirical findings show that language learning can indeed be supported in physical ways, by body motions and gestures. The method used here, named TFFL (Touch and Feel Foreign Languages), was designed to support adult language learning, to correct or prevent language fossilization, and to help the student to manage emotions. Finnish is considered as a difficult language to learn, mostly because it is so different from nearly all other languages. Many learners complain that they are lost or confused and there is a need to find a way to simultaneously learn the language and to handle negative emotion that comes from the Finnish language and the learning process itself. Due to the nature of the Finnish language, good pronunciation skills are needed just to understand the way the language work. Movements (body movements etc.) are a natural part of many cultures, but not Finnish. In Finland, students have traditionally been expected to stay still, and that is not a natural way for many foreign students. However, the kinaesthetic TFFL method proved out to be a useful way to help some L2 students to feel phonemes, rhythm, and intonation, to improve their Finnish, and, thereby, also to successfully complete their vocational studies.

Keywords: Finnish, fossilization, interference, kinaesthetic method

Procedia PDF Downloads 140

Authors: Eunyup Yeom

Abstract:

The roles of male and female in context of cuisine have developed into stereotypes throughout history. However¬— with Korea’s fast advancement in politics, technology, society and social standards¬— gender stereotypes have become blurred. This is not to say that such stereotypes no longer exist for they still remain present in media and advertisements embedding ‘idealistic’ ideas into the unconscious state of minds of viewers. Many media outlets, especially commercials, portray males expressing pleasure of food [that they are advertising] through audible qualities generally considered ‘rude’ and ‘unmannered’ in the Korean society. Females, on the other hand, express such pleasures only verbally. This happenstance of a stereotype is displayed bluntly in instant noodle, namely ramen, commercials. This research explores the cultural significance of a type of audible gesture that can be found in Korean speech in which is termed the Fricative Voice Gesture (FVG). There are two forms of FVGs: the reactive and the prosodic. The reactive FVG is a legitimate form of expression while the prosodic FVG works as a speech intensifier. So, in order to understand this stereotype of who is authorized to express sensorial pleasure as a reactive FVG as opposed to a prosodic FVG, information has been extracted from interviews and dissected numerous ramen/instant noodle commercials and its appearances in other mediums of media. The commercials were tediously analyzed in all aspects of dialogue, featured contents, background music, actors and/or actresses selling the product, body language, and voice gestures. To effectively understand the exact impact these commercials have on the audience, each commercial was viewed with an interviewee. In this research, there were main informants whom were all Korean students residing in South Korea. All three interviewees were able to attend interview and commercial viewing sessions via Skype. This research, overall, focuses and concludes on Harkness’s statement of how the reactive FVG is a recognizable index of the privileging of males for Korean culture norms and, in parallel, food commercials are still conforming to male ideals and fantasies.

Keywords: advertisement, food politics, fricative voice gestures, gender politics

Procedia PDF Downloads 227

Authors: Tamara Matevosyan

Abstract:

The present paper discusses the main problems that students face while developing oral skills through authentic films. It states that special attention should be paid not only to the study of verbal speech but also to non-verbal communication. Authentic films serve as an important tool to understand both native speaker’s gestures and their culture of pausing while speaking. Various phonetic difficulties causing phonetic interference in actual speech are covered in the paper emphasizing the role of authentic films in overcoming them.

Keywords: compressive speech, filled pauses, unfilled pauses, pausing culture

Procedia PDF Downloads 353

Authors: Thomas Ousterhout

Abstract:

In an effort to test the ability of a commercial grade EEG headset to effectively measure the N400 ERP, a replication study was conducted to see if similar results could be produced as that which used a medical grade EEG. Pictures of meaningful and meaningless hand postures were borrowed from the original author and subjects were required to perform a semantic discrimination task. The N400 was detected indicating semantic processing of the meaningfulness of the hand postures. The results corroborate those of the original author and support the use of some commercial grade EEG headsets for non-critical research applications.

Keywords: EEG, ERP, N400, semantics, congruency, gestures, emotiv

Procedia PDF Downloads 264

Authors: Anjalika Fernando, Banuka Athuraliya

Abstract:

This paper presents a comprehensive approach for converting Sinhala Sign Language (SSL) into grammatically correct sentences using Natural Language Processing (NLP) techniques in real-time. While previous studies have explored various aspects of SSL translation, the research gap lies in the absence of grammar checking for SSL. This work aims to bridge this gap by proposing a two-stage methodology that leverages deep learning models to detect signs and translate them into coherent sentences, ensuring grammatical accuracy. The first stage of the approach involves the utilization of a Long Short-Term Memory (LSTM) deep learning model to recognize and interpret SSL signs. By training the LSTM model on a dataset of SSL gestures, it learns to accurately classify and translate these signs into textual representations. The LSTM model achieves a commendable accuracy rate of 94%, demonstrating its effectiveness in accurately recognizing and translating SSL gestures. Building upon the successful recognition and translation of SSL signs, the second stage of the methodology focuses on improving the grammatical correctness of the translated sentences. The project employs a Neural Machine Translation (NMT) architecture, consisting of an encoder and decoder with LSTM components, to enhance the syntactical structure of the generated sentences. By training the NMT model on a parallel corpus of Sinhala wrong sentences and their corresponding grammatically correct translations, it learns to generate coherent and grammatically accurate sentences. The NMT model achieves an impressive accuracy rate of 98%, affirming its capability to produce linguistically sound translations. The proposed approach offers significant contributions to the field of SSL translation and grammar correction. Addressing the critical issue of grammar checking, it enhances the usability and reliability of SSL translation systems, facilitating effective communication between hearing-impaired and non-sign language users. Furthermore, the integration of deep learning techniques, such as LSTM and NMT, ensures the accuracy and robustness of the translation process. This research holds great potential for practical applications, including educational platforms, accessibility tools, and communication aids for the hearing-impaired. Furthermore, it lays the foundation for future advancements in SSL translation systems, fostering inclusive and equal opportunities for the deaf community. Future work includes expanding the existing datasets to further improve the accuracy and generalization of the SSL translation system. Additionally, the development of a dedicated mobile application would enhance the accessibility and convenience of SSL translation on handheld devices. Furthermore, efforts will be made to enhance the current application for educational purposes, enabling individuals to learn and practice SSL more effectively. Another area of future exploration involves enabling two-way communication, allowing seamless interaction between sign-language users and non-sign-language users.In conclusion, this paper presents a novel approach for converting Sinhala Sign Language gestures into grammatically correct sentences using NLP techniques in real time. The two-stage methodology, comprising an LSTM model for sign detection and translation and an NMT model for grammar correction, achieves high accuracy rates of 94% and 98%, respectively. By addressing the lack of grammar checking in existing SSL translation research, this work contributes significantly to the development of more accurate and reliable SSL translation systems, thereby fostering effective communication and inclusivity for the hearing-impaired community

Keywords: Sinhala sign language, sign Language, NLP, LSTM, NMT

Procedia PDF Downloads 107

Authors: Malika Yaici, Soraya Aloui, Sara Semchaoui

Abstract:

This paper deals with human activity recognition from sensor data. It is an active research area, and the main objective is to obtain a high recognition rate. In this work, a recognition system based on expert systems is proposed; the recognition is performed using the objects, object states, and gestures and taking into account the context (the location of the objects and of the person performing the activity, the duration of the elementary actions and the activity). The system recognizes complex activities after decomposing them into simple, easy-to-recognize activities. The proposed method can be applied to any type of activity. The simulation results show the robustness of our system and its speed of decision.

Keywords: human activity recognition, ubiquitous computing, context-awareness, expert system

Procedia PDF Downloads 119

Authors: Malika Yaici, Soraya Aloui, Sara Semchaoui

Abstract:

Recognition of human activities from sensor data is an active research area, and the main objective is to obtain a high recognition rate. In this work, we propose a recognition system based on expert systems. The proposed system makes the recognition based on the objects, object states, and gestures, taking into account the context (the location of the objects and of the person performing the activity, the duration of the elementary actions, and the activity). This work focuses on complex activities which are decomposed into simple easy to recognize activities. The proposed method can be applied to any type of activity. The simulation results show the robustness of our system and its speed of decision.

Keywords: human activity recognition, ubiquitous computing, context-awareness, expert system

Procedia PDF Downloads 142

Authors: B. S. Bamoallem, A. J. Wodehouse, G. M. Mair

Abstract:

Communication takes place not only through speech, but also by means of gestures such as facial expressions, gaze, head movements, hand movements and body posture, and though there has been rapid development, communication platforms still lack this type of behavior. We believe communication platforms need to fully achieve this verbal and non-verbal behavior in order to make interactions more engaging and more efficient. In this study we decided to focus our research on the head rather than any other body part as it is a rich source of information for speech-related movement Thus we aim to investigate the value of incorporating head movements into the use of telepresence robots as communication platforms; this will be done by investigating a system that reproduces head movement manually as closely as possible.

Keywords: engagement, nonverbal behaviours, head movements, face-to-face interaction, telepresence robot

Procedia PDF Downloads 455

Authors: Gintare Judzentyte

Abstract:

The present research is aimed to discuss semantics and pragmatics of spatial deictics (deictic adverbs of place and demonstrative pronouns) in the Baltic languages: in spoken Lithuanian and in spoken Latvian. The following objectives have been identified to achieve the aim: 1) to determine the usage of adverbs of place in spoken Lithuanian and Latvian and to verify their meanings in face-to-face communication; 2) to determine the usage of demonstrative pronouns in spoken Lithuanian and Latvian and to verify their meanings in face-to-face communication; 3) to compare the systems between the two spoken languages and to identify the main tendencies. As meanings of demonstratives (adverbs of place and demonstrative pronouns) are context-bound, it is necessary to verify their usage in spontaneous interaction. Besides, deictic gestures play a very important role in face-to-face communication. Therefore, an experimental method is necessary to collect the data. Video material representing spoken Lithuanian and spoken Latvian was recorded by means of the method of a qualitative interview (a semi-structured interview: an empirical research is all about asking right questions). The collected material was transcribed and evaluated taking into account several approaches: 1) physical distance (location of the referent, visual accessibility of the referent); 2) deictic gestures (the combination of language and gesture is especially characteristic of the exophoric use); 3) representation of mental spaces in physical space (a speaker sometimes wishes to mark something that is psychically close as psychologically distant and vice versa). The research of the collected data revealed that in face-to-face communication the participants choose deictic adverbs of place instead of demonstrative pronouns to locate/identify entities in situations where the demonstrative pronouns would be expected in spoken Lithuanian and in spoken Latvian. The analysis showed that visual accessibility of the referent is very important in face-to-face communication, but the main criterion while localizing objects and entities is the need for contrast: lith. čia ‘here’, šis ‘this’, latv. šeit ‘here’, šis ‘this’ usually identify distant entities and are used instead of distal demonstratives (lith. ten ‘there’, tas ‘that’, latv. tur ‘there’, tas ‘that’), because the referred objects/subjects contrast to further entities. Furthermore, the interlocutors in examples from a spontaneously situated interaction usually extend their space and can refer to a ‘distal’ object/subject with a ‘proximal’ demonstrative based on the psychological choice. As the research of the spoken Baltic languages confirmed, the choice of spatial deictics in face-to-face communication is strongly effected by a complex of criteria. Although there are some main tendencies, the exact meaning of spatial deictics in the spoken Baltic languages is revealed and is relevant only in a certain context.

Keywords: Baltic languages, face-to-face communication, pragmatics, semantics, spatial deictics

Procedia PDF Downloads 290

Authors: Ian A. Grout

Abstract:

In this paper, the design and development of a system to provide hand motion and gesture control of laboratory test equipment is considered and discussed. The Leap Motion controller is used to provide an input to control a laboratory power supply as part of an electronic circuit experiment. By suitable hand motions and gestures, control of the power supply is provided remotely and without the need to physically touch the equipment used. As such, it provides an alternative manner in which to control electronic equipment via a PC and is considered here within the field of human computer interaction (HCI).

Keywords: control, hand gesture, human computer interaction, test equipment

Procedia PDF Downloads 316

Authors: Keyi Wang

Abstract:

Similar to the touchscreen, hand gesture based human-computer interaction (HCI) is a technology that could allow people to perform a variety of tasks faster and more conveniently. This paper proposes a training method of an image-based hand gesture image and video clip recognition system using a CNN (Convolutional Neural Network) with a dataset. A dataset containing 6 hand gesture images is used to train a 2D CNN model. ~98% accuracy is achieved. Furthermore, a 3D CNN model is trained on a dataset containing 4 hand gesture video clips resulting in ~83% accuracy. It is demonstrated that a Cozmo robot loaded with pre-trained models is able to recognize static and dynamic hand gestures.

Keywords: deep learning, hand gesture recognition, computer vision, image processing

Procedia PDF Downloads 143

Authors: André Augusto Ceballos Melo

Abstract:

Design to facilitate the recognition of congruent prosthetic movements, context-to-motion translations guided by image, verbal prompt, users nonverbal communication such as facial expressions, gestures, paralinguistics, scene context, and object recognition contributes to this process though it can also be applied to other tasks, such as walking, Prosthetic limbs as assistive technology through gestures, sound codes, signs, facial, body expressions, and scene context The context-to-motion model is a machine learning approach that is designed to improve the control and dexterity of prosthetic limbs. It works by using sensory input from the prosthetic limb to learn about the dynamics of the environment and then using this information to generate smooth, stable movements. This can help to improve the performance of the prosthetic limb and make it easier for the user to perform a wide range of tasks. There are several key benefits to using the context-to-motion model for prosthetic limb control. First, it can help to improve the naturalness and smoothness of prosthetic limb movements, which can make them more comfortable and easier to use for the user. Second, it can help to improve the accuracy and precision of prosthetic limb movements, which can be particularly useful for tasks that require fine motor control. Finally, the context-to-motion model can be trained using a variety of different sensory inputs, which makes it adaptable to a wide range of prosthetic limb designs and environments. Stable diffusion is a machine learning method that can be used to improve the control and stability of movements in robotic and prosthetic systems. It works by using sensory feedback to learn about the dynamics of the environment and then using this information to generate smooth, stable movements. One key aspect of stable diffusion is that it is designed to be robust to noise and uncertainty in the sensory feedback. This means that it can continue to produce stable, smooth movements even when the sensory data is noisy or unreliable. To implement stable diffusion in a robotic or prosthetic system, it is typically necessary to first collect a dataset of examples of the desired movements. This dataset can then be used to train a machine learning model to predict the appropriate control inputs for a given set of sensory observations. Once the model has been trained, it can be used to control the robotic or prosthetic system in real-time. The model receives sensory input from the system and uses it to generate control signals that drive the motors or actuators responsible for moving the system. Overall, the use of the context-to-motion model has the potential to significantly improve the dexterity and performance of prosthetic limbs, making them more useful and effective for a wide range of users Hand Gesture Body Language Influence Communication to social interaction, offering a possibility for users to maximize their quality of life, social interaction, and gesture communication.

Keywords: stable diffusion, neural interface, smart prosthetic, augmenting

Procedia PDF Downloads 102

Authors: Madhusudan Hamirwasia, Sarang Barbarwar, Arshleen Kaur

Abstract:

The construction of memorials began thousands of years ago and the practice is still continuing. These memorials became a symbol to honor great people and events in the history. The aim of the study was to understand the evolution of memorials from an architectural design perspective. It is also concentrated on the similarities and differences between the memorials in Europe and those in the Indian subcontinent. The study shows how the design of a memorial has seen a considerable shift from the tribal Urasgattas to the contemporary commemorative structures. While they were somber symbolic gestures in the past, they have now transformed into a socio-cultural space in urban areas. Not only the memorials were inspired by the culture but the culture too got influenced by the memorials as with progressing time, they hold the vital link to our past. The study intends to encapsulate the essence of design elements in these memorials that convey the visitors the intangible messages held by the edifice in its tangible presence.

Keywords: evolution, emotion, memorials, symbolism

Procedia PDF Downloads 142

Authors: Parham Bakhtiari

Abstract:

Between 1910 and 1912, Claude Debussy recorded twelve of his solo piano pieces. Although Debussy frequently provided advice to his students on performing while they followed the written notes when performing, his personal recordings are characterized by creative liberties and unique freedom interpretations. Debussy's use of numerous interpretive gestures in these recordings is fascinating and corresponds with the techniques utilized by French Baroque keyboard performers. This paper will situate Debussy's presentation in the Baroque musical approach. Initially, we will discuss the recording by analyzing Welte-Mignon's used technology to guarantee the reliability of these recordings. Then, we will find commonalities in the intricate performances of harpsichord musicians who played in the 1600s and 1700s and recordings of Debussy. Finally, by drawing comparisons, we will review the patterns by contrasting Debussy's execution with recordings of the same pieces from the latter half of the 20th century as striving for improved presentations while limiting artistic freedom.

Keywords: music, Debussy, piano, performance, prelude

Procedia PDF Downloads 49

Authors: Stephen L. Green, Alexander N. Gorban, Ivan Y. Tyukin

Abstract:

Within the past decade, using Convolutional Neural Networks (CNN)’s to create Deep Learning systems capable of translating Sign Language into text has been a breakthrough in breaking the communication barrier for deaf-mute people. Conventional research on this subject has been concerned with training the network to recognize the fingerspelling gestures of a given language and produce their corresponding alphanumerics. One of the problems with the current developing technology is that images are scarce, with little variations in the gestures being presented to the recognition program, often skewed towards single skin tones and hand sizes that makes a percentage of the population’s fingerspelling harder to detect. Along with this, current gesture detection programs are only trained on one finger spelling language despite there being one hundred and forty-two known variants so far. All of this presents a limitation for traditional exploitation for the state of current technologies such as CNN’s, due to their large number of required parameters. This work aims to present a technology that aims to resolve this issue by combining a pretrained legacy AI system for a generic object recognition task with a corrector method to uptrain the legacy network. This is a computationally efficient procedure that does not require large volumes of data even when covering a broad range of sign languages such as American Sign Language, British Sign Language and Chinese Sign Language (Pinyin). Implementing recent results on method concentration, namely the stochastic separation theorem, an AI system is supposed as an operate mapping an input present in the set of images u ∈ U to an output that exists in a set of predicted class labels q ∈ Q of the alphanumeric that q represents and the language it comes from. These inputs and outputs, along with the interval variables z ∈ Z represent the system’s current state which implies a mapping that assigns an element x ∈ ℝⁿ to the triple (u, z, q). As all xi are i.i.d vectors drawn from a product mean distribution, over a period of time the AI generates a large set of measurements xi called S that are grouped into two categories: the correct predictions M and the incorrect predictions Y. Once the network has made its predictions, a corrector can then be applied through centering S and Y by subtracting their means. The data is then regularized by applying the Kaiser rule to the resulting eigenmatrix and then whitened before being split into pairwise, positively correlated clusters. Each of these clusters produces a unique hyperplane and if any element x falls outside the region bounded by these lines then it is reported as an error. As a result of this methodology, a self-correcting recognition process is created that can identify fingerspelling from a variety of sign language and successfully identify the corresponding alphanumeric and what language the gesture originates from which no other neural network has been able to replicate.

Keywords: convolutional neural networks, deep learning, shallow correctors, sign language

Procedia PDF Downloads 101

Authors: Mohd Noah A. Rahman, Afzaal H. Seyal, Norhafilah Bara

Abstract:

Developing a software application using an interface with computers and peripheral devices using gestures of human body such as hand movements keeps growing in interest. A review on this hand gesture detection and recognition based on computer vision technique remains a very challenging task. This is to provide more natural, innovative and sophisticated way of non-verbal communication, such as sign language, in human computer interaction. Nevertheless, this paper explores hand detection and hand gesture recognition applying a vision based approach. The hand detection and recognition used skin color spaces such as HSV and YCrCb are applied. However, there are limitations that are needed to be considered. Almost all of skin color space models are sensitive to quickly changing or mixed lighting circumstances. There are certain restrictions in order for the hand recognition to give better results such as the distance of user’s hand to the webcam and the posture and size of the hand.

Keywords: hand detection, hand gesture, hand recognition, sign language

Procedia PDF Downloads 307

Authors: Ruepert Jiel Cao

Abstract:

This study explores the renegotiation of bakla culture in Philippine media in the context of Kalye Serye segment of the popular Filipino noontime variety show Eat Bulaga. Although the term “bakla” is usually translated to “gay” or “homosexual male” in English, they do not mean the same. The western notion of a gay refers to a male person attracted to another male person but still retains the masculine physical attributes. However, the bakla embodies loudness, femininity, and transvestitism. Hence, a bakla is a gay man aspiring to be a woman by assuming feminine actions and appearance, a definition much closer to a transgender. The Philippine media usually employs the bakla culture in comedy programs. The bakla nowadays is usually associated with the people of lower economic strata and carries a pathological connotation. The Filipino television program Eat Bulaga, which has been airing for more than 36 years, is fond of using bakla in comedy. However, the recently launched segment entitled Kalye Serye (literally “Street [Television] Series”), while still employing drag performance to incorporate bakla culture in comedy, renegotiates the bakla culture by deviating from the stereotypical notion of bakla. In this study, this researcher asks: (1) How does Kalye Serye renegotiate the Filipino concept of bakla in terms of economic aspirations and social norms? (2) How does Kalye Serye reappropriate the bakla culture to fit non-comedic performances? The study examines 15 purposively selected Kalye Serye episodes. Seven were selected from the Thursday episodes, seven from Saturday episodes, and the Lenten special episode. These were selected to cover as many characters and different character roles as possible. Data was constructed by identifying and coding the roles, physical appearance and gestures, and key dialogs of the characters. A total of six female characters played by three different male actors were examined. Semiotic analysis using semiotics of Roland Barthes was performed to produce a reading of the characters. Findings show that through physical appearance, the characters associate bakla with the economic affluence through the use of expensive-looking clothes, jewelries, cars, and elaborate gestures. This represents a new economic but old western aspiration of the bakla. In terms of social norms, the characters try to revive the traditional concepts of femininity, courtship, and respect, values which are touted to be lost in the current generation of Filipinos. This is quite ironic because while there is a seemingly tolerant attitude towards all forms of queerness, the bakla is considered immoral and yet, the bakla is used to teach about morality and values. Finally, the characters break the traditional association of the bakla with slapstick comedy and their roles are reappropriated to suit dramatic roles. By refraining from portraying the bakla in ridiculous manner (physically and in terms of roles), the bakla lends itself well in the performance of dramatic roles and their ridiculous and pathological associations removed. Future research may include other Filipino or Asian portrayals of queerness to get a better understanding of how queerness is incorporated in contemporary popular culture.

Keywords: bakla, drag performance, popular culture, queer representation

Procedia PDF Downloads 302

Authors: Hervé Zénouda

Abstract:

This article seeks to shed light on an emerging creative field: "Audio games," at the crossroads between video games and computer music. Indeed, many applications, which propose entertaining audio-visual experiences with the objective of musical creation, are available today for different supports (game consoles, computers, cell phones). The originality of this field is the use of the gameplay of video games applied to music composition. Thus, composing music using interfaces but also cognitive logics that we qualify as "a-musicological" seem to us particularly interesting from the perspective of musical digital organology. This field raises questions about the representation of sound and musical structures and develops new instrumental gestures and strategies of musical composition. We will try in this article to define the characteristics of this field by highlighting some historical milestones (abstract cinema, game theory in music, actions, and graphic scores) as well as the novelties brought by digital technologies.

Keywords: audio-games, video games, computer generated music, gameplay, interactivity, synesthesia, sound interfaces, relationships image/sound, audiovisual music

Procedia PDF Downloads 113

Authors: Vaibhav Garg

Abstract:

The concept of guest engagement lies in the rigor of operational excellence and the emotional engagement of associates who perform their tasks genuinely from their hearts and hence in word and indeed, in intent and through gestures, a great hospitality is always genuine, attentive, passionate, caring and warm where engaged associates deliver exceptional service experiences and creates memories to last forever for the guests. One out of every five guests says that their decision of coming back to the same hotel is influenced by the opportunity to “experience and be engaged” A key question is what does a guest mean by experience and be engaged? Most hotels are highly concerned about the guest satisfaction. Therefore they have the brand standards which are a guide to the associate to ensure consistent implementation of set service and product standards to satisfy a guest. However, satisfaction of basic guest needs does not necessarily lead to engagement. For example an absolutely clean room and an in room dining order delivered on time can satisfy a guest but may not engage him. Absence of these standards can certainly lead to guest dissatisfaction however; the presence of these standards does not necessarily lead to guest engagement or guest delight.

Keywords: guest engagement, guest satisfaction, hospitality, hotel operations, operational excellence

Procedia PDF Downloads 240

Authors: Siddharth Satishchandran, Brian Khumalo

Abstract:

Humans are able to derive semantic content from syntactic and pragmatic sources. Multimodal evidence from signaling theory, which examines communication between individuals within and across species, suggests that non-human primates possess similar syntactic and pragmatic capabilities. However, the extent remains unknown because primate pragmatics are relatively under-examined. Our paper reviews research within communication theory amongst non-human primates to understand current theoretical trends. We examine evidence for primate pragmatic capacities through observational, experimental, and theoretical work on gestures. Given fragmented theoretical perspectives, we provide a unified framework of communication for future research that contextualizes the available research under code biology. To achieve this, we rely on biological semiotics (biosemiotics), the philosophy of biology investigating prelinguistic meaning-making as a function of signs and codes. We close by discussing areas of potential research for studying gestural pragmatics amongst non-human primates, particularly chimpanzees (Pan troglodytes), Diana monkeys (Cercopithecus diana), and other potential candidates.

Keywords: pragmatics, non-human primates, gestural communication, biological semiotics

Procedia PDF Downloads 42

Authors: Daniele Baldacci, Remo Pareschi

Abstract:

Augmented Reality and Augmented Intelligence are radically changing information technology. The path that starts from the keyboard and then, passing through milestones such as Siri, Alexa and other vocal avatars, reaches a more fluid and natural communication with computers, thus converting the dichotomy between man and machine into a harmonious interaction, now heads unequivocally towards a new IT paradigm, where holographic computing will play a key role. The OLOS® platform contributes substantially to this trend in that it infuses computers with human features, by transferring the gestures and expressions of persons of flesh and bones to anthropomorphic holographic interfaces which in turn will use them to interact with real-life humans. In fact, we could say, boldly but with a solid technological background to back the statement, that OLOS® gives reality to an altogether new entity, placed at the exact boundary between nature and technology, namely the holographic human being. Holographic humans qualify as the perfect carriers for the virtual reincarnation of characters handed down from history and tradition. Thus, they provide for an innovative and highly immersive way of experiencing our cultural heritage as something alive and pulsating in the present.

Keywords: digital cinematography, human-computer interfaces, holographic simulation, interactive museum exhibits

Procedia PDF Downloads 118