Search results for: multimodal metaphor

Authors: Xing Lyu

Abstract:

Multimodal metaphor promotes the further development and refinement of multimodal discourse study. Cultural aspects matter a lot not only in creating but also in comprehending multimodal metaphor. By analyzing the target domain and the source domain in 10 public service advertisements of China and Germany about environmental protection, this paper compares the source when the target is alike in each multimodal metaphor in order to seek similarities and differences across cultures. The findings are as follows: first, the multimodal metaphors center around three major topics: the earth crisis, consequences of environmental damage, and appeal for environmental protection; second, the multimodal metaphors mainly grounded in three universal conceptual metaphors which focused on high level is up; earth is mother and all lives are precious. However, there are five Chinese culture-specific multimodal metaphors which are not discovered in Germany ads: east is high leve; a purposeful life is a journey; a nation is a person; good is clean, and water is mother. Since metaphors are excellent instruments on studying ideology, this study can be helpful on intercultural/cross-cultural communication.

Keywords: multimodal metaphor, cultural aspects, public service advertising, cross-cultural communication

Procedia PDF Downloads 173

Authors: Manana Rusieshvili, Rusudan Dolidze

Abstract:

This paper aims at 1) investigating the ways in which a traditional, monomodal written verbal metaphor can be transposed as a monomodal non-verbal (visual) or multimodal (aural and -visual) filmic metaphor ; 2) exploring similarities and differences in the process of encoding and decoding of monomodal and multimodal metaphors. The empiric data, on which the research is based, embrace three sources: the novel by Harry Gray ‘The Hoods’, the script of the film ‘Once Upon a Time in America’ (English version by David Mills) and the resultant film by Sergio Leone. In order to achieve the above mentioned goals, the research focuses on the following issues: 1) identification of verbal and non-verbal monomodal and multimodal metaphors in the above-mentioned sources and 2) investigation of the ways and modes the specific written monomodal metaphors appearing in the novel and the script are enacted in the film and become visual, aural or visual-aural filmic metaphors ; 3) study of the factors which play an important role in contributing to the encoding and decoding of the filmic metaphor. The collection and analysis of the data were carried out in two stages: firstly, the relevant data, i.e. the monomodal metaphors from the novel, the script and the film were identified and collected. In the second, final stage the metaphors taken from all of the three sources were analysed, compared and two types of phenomena were selected for discussion: (1) the monomodal written metaphors found in the novel and/or in the script which become monomodal visual/aural metaphors in the film; (2) the monomodal written metaphors found in the novel and/or in the script which become multimodal, filmic (visual-aural) metaphors in the film.

Keywords: encoding, decoding, filmic metaphor, multimodality

Procedia PDF Downloads 526

Authors: Jie Zheng, Ruifeng Luo

Abstract:

The United States is the world’s most developed economy with the strongest military power. China is the fastest growing country with growing comprehensive strength and its economic strength is second only to the US. However, the conflict between them is getting serious in recent years. President’s address is the representative of a nation’s ideology. The paper has built up a small sized corpus of President Xi Jinping and Trump’s speech in Davos to investigate their respective use and types of metaphors and calculate the respective percentage of each type of metaphor. The result shows President Xi Jinping employs more metaphors than Trump. The metaphors of Xi includes “building” metaphor, “plant” metaphor, “journey” metaphor, “ship” metaphor, “traffic” metaphor, “nation is a person” metaphor, “show” metaphor, etc while Trump’s comprises “war” metaphor, “building” metaphor, “journey” metaphor, “traffic” metaphor, “tax” metaphor, “book” metaphor, etc. After investigating metaphor use differences, the paper makes an analysis of the underlying ideology between the two nations. China is willing to strengthen ties with all the countries all over the world and has built a platform of development for them and itself to go to the destination of social well being while the US pays much concern to itself, emphasizing its first leading position and is also willing to help its alliances to development. The paper’s comparison of the ideology difference between the two countries will help them get a better understanding and reduce the conflict to some extent.

Keywords: metaphor; corpus; ideology; conflict

Procedia PDF Downloads 147

Authors: Xuri Tang, Ting Pan

Abstract:

Metaphor institutionalization refers to the propagation of a metaphor that leads to its acceptance in speech community as a norm of the language. Such knowledge is important to both theoretical studies of metaphor and practical disciplines such as lexicography and language generation. This paper reports an empirical study of metaphor institutionalization of 14 Chinese metaphors. It first explores the pattern of metaphor institutionalization by fitting the logistic function (or S-shaped curve) to time series data of conventionality of the metaphors that are automatically obtained from a large-scale diachronic Chinese corpus. Then it reports a questionnaire-based survey on the propagation scale of each metaphor, which is measured by the average number of subjects that can easily understand the metaphorical expressions. The study provides two pieces of evidence supporting the hypothesis that metaphor institutionalization is a phrase transition: (1) the pattern of metaphor institutionalization is an S-shaped curve and (2) institutionalized metaphors generally do not propagate to the whole community but remain in equilibrium state. This conclusion helps distinguish metaphor institutionalization from topicalization and other types of semantic change.

Keywords: metaphor institutionalization, phase transition, propagation scale, s-shaped curve

Procedia PDF Downloads 169

Authors: Saulutė Juzelėnienė, Skirmantė Šarkauskienė

Abstract:

In this paper, it is aimed to analyse the multimodal representations of the concepts of LOVE and PASSION in Lithuanian graphic novel “Gertrūda”, by Gerda Jord. The research is based on the earlier findings by Forceville (2005), Eerden (2009) as well as insights made by Shihara and Matsunaka (2009) and Kövecses (2000). The domains of target and source of LOVE and PASSION metaphors in comics are expressed by verbal and non-verbal cues. The analysis of non-verbal cues adopts the concepts of rune and indexes. A pictorial rune is a graphic representation of an object that does not exist in reality in comics, such as lines, dashes, text "balloons", and pictorial index – a graphically represented object of reality, a real symptom expressing a certain emotion, such as a wide smile, furrowed eyebrows, etc. Indexes are often hyperbolized in comics. The research revealed that most frequent source domains are CLOSINESS/UNITY, NATURAL/ PHYSICAL FORCE, VALUABLE OBJECT, PRESSURE. The target is the emotion of LOVE/PASSION which belongs to a more abstract domain of psychological experience. In this kind of metaphor, the picture can be interpreted as representing the emotion of happiness. Data are taken from Lithuanian comic books and Internet sites, where comics have been presented. The data and the analysis we are providing in this article aims to reveal that there are pictorial metaphors that manifest conceptual metaphors that are also expressed verbally and that methodological framework constructed for the analysis in the papers by Forceville at all is applicable to other emotions and culture specific pictorial manifestations.

Keywords: multimodal metaphor, conceptual metaphor, comics, graphic novel, concept of love/passion

Procedia PDF Downloads 64

Authors: Mohammad Hamzah Alshehab

Abstract:

Metaphor is a substitute expression in everyday life in languages, thoughts and actions. It has an original value in language use with different conceptual, grammatical and properties. In addition, it is a central concept in literary studies. The present paper aims at investigating metaphor’s types imbedded in some Holy Verses (HV). For achieving the objectives of this paper, two English versions were chosen , the first is the Translation of the Meanings of the Noble Qura’n in the English Language by Mohammad AlHilali and Mohammad Khan, and the second version is the English Translation of the Holy Qura’n by Mohammad Ali were used. The researcher selected (20) Holy Verses include metaphors to be analyzed and investigated. Metaphor types were categorized by an assessment of the two translations followed by a discussion between the two versions of translation.

Keywords: metaphor, metaphor’s types, Holy Qura’n, Holy Verses

Procedia PDF Downloads 652

Authors: LuoJiaoyang, Yu Hongyang

Abstract:

In this paper, we experimented with a new approach to multimodal identification using RGB, RGB-D and voice data. The multimodal combination of RGB and voice data has been applied in tasks such as emotion recognition and has shown good results and stability, and it is also the same in identity recognition tasks. We believe that the data of different modalities can enhance the effect of the model through mutual reinforcement. We try to increase the three modalities on the basis of the dual modalities and try to improve the effectiveness of the network by increasing the number of modalities. We also implemented the single-modal identification system separately, tested the data of these different modalities under clean and noisy conditions, and compared the performance with the multimodal model. In the process of designing the multimodal model, we tried a variety of different fusion strategies and finally chose the fusion method with the best performance. The experimental results show that the performance of the multimodal system is better than that of the single modality, especially in dealing with noise, and the multimodal system can achieve an average improvement of 5%.

Keywords: multimodal, three modalities, RGB-D, identity verification

Procedia PDF Downloads 68

Authors: Tomoko Usui

Abstract:

In the field of linguistics, it has been said that concepts associated with space and motion systematically contribute structure to the temporal concept. This is the conceptual metaphor theory. conceptual metaphors typically employ a more abstract concept (time) as their target and a more concrete or physical concept as their source (space). This paper will examine two major temporal conceptual metaphors: Ego-centered Moving Time Metaphor and Time-RP Metaphor. Moving time generally receives a front-back orientation, however, Japanese shows a different orientation given to time. By means of Ego perspective, this paper will illustrate the paradox of a metaphorical orientation in time.

Keywords: Ego-centered Moving Time Metaphor, Japanese saki, temporal metaphors, Time RP Metaphor

Procedia PDF Downloads 496

Authors: Linette van der Merwe

Abstract:

It is known that visual artists, performing artists, and literary artists have inspired each other since time immemorial. The enduring, symbiotic relationship between the various art genres is evident where words, colours, lines, and sounds act as metaphors, a physical separation of the transcendental reality of art. Simonides of Keos (c. 556-468 BC) confirmed this, stating that a poem is a talking picture, or, in a more modern expression, a picture is worth a thousand words. It can be seen as an ancient relationship, originating from the epigram (tombstone or artefact inscriptions), the carmen figuratum (figure poem), and the ekphrasis (a description in the form of a poem of a work of art). Visual artists, including Michelangelo, Leonardo da Vinci, and Goethe, wrote poems and songs. Goya, Degas, and Picasso are famous for their works of art and for trying their hands at poetry. Afrikaans writers whose fine art is often published together with their writing, as in the case of Andries Bezuidenhout, Breyten Breytenbach, Sheila Cussons, Hennie Meyer, Carina Stander, and Johan van Wyk, among others, are not a strange phenomenon either. Imitating one art form into another art form is a form of translation, transposition, contemplation, and discovery of artistic impressions, showing parallel interpretations rather than physical comparison. It is especially about the harmony that exists between the different art genres, i.e., a poem that describes a painting or a visual text that portrays a poem that becomes a translation, interpretation, and rediscovery of the verbal text, or rather, from the word text to the image text. Poetry-film, as a form of such a translation of the word text into an image text, can be considered a hybrid, transdisciplinary art form that connects poetry and film. Poetry-film is regarded as an intertwined entity of word, sound, and visual image. It is an attempt to transpose and transform a poem into a new artwork that makes the poem more accessible to people who are not necessarily open to the written word and will, in effect, attract a larger audience to a genre that usually has a limited market. Poetry-film is considered a creative expression of an inverted ekphrastic inspiration, a visual description, interpretation, and expression of a poem. Research also emphasises that animated poetry-film is not widely regarded as a genre of anything and is thus severely under-theorized. This paper will focus on Afrikaans animated poetry-films as a multimodal transposition of a poem text to an animated poetry film, with specific reference to animated poetry-films in Filmverse I (2014) and Filmverse II (2016).

Keywords: poetry film, animated poetry film, poetic metaphor, conceptual metaphor, monomodal metaphor, multimodal metaphor, semiotic metaphor, multimodality, metaphor analysis, target domain, source domain

Procedia PDF Downloads 63

Authors: Svetlana L. Mishlanova, Evgeniia V. Ermakova, Mariia E. Timirkina

Abstract:

The paper is concerned with the study of metaphor production in essays written by Russian and English native speakers in the framework of cognitive metaphor theory. It considers metaphorical competence as individual’s ability to recognize, understand and use metaphors in speech. The work analyzes the influence of visual metaphor on production and density of conventional and novel verbal metaphors. The main methods of research include experiment connected with image interpretation, metaphor identification procedure (MIPVU) and visual conventional metaphors identification procedure proposed by VisMet group. The research findings will be used in the project aimed at comparing metaphorical competence of native and non-native English speakers.

Keywords: metaphor, metaphorical competence, conventional, novel

Procedia PDF Downloads 286

Authors: Marzieh Talebzadeh Shoushtari

Abstract:

The present study aimed to determine the effectiveness of Metaphor therapy on depression among female students. The sample included 60 female students with depression symptoms selected by simple sampling and randomly divided into two equal groups (experimental and control groups). Beck Depression Inventory was used to measure the variables. This was an experimental study with a pre-test/post-test design with control group. Eight metaphor therapy sessions were held for the experimental group. A post-test was administered to both groups. Data were analyzed using multivariate analysis of covariance (MANCOVA). Results showed that the Metaphor therapy decreased depression in the experimental group compared to the control group.

Keywords: metaphor therapy, depression, female, students

Procedia PDF Downloads 454

Authors: Amany ElShazly, Lubna A. Sherif

Abstract:

The entanglement of humans with the environment has always been inevitable and often causes destruction. In this respect, ‘Ecolinguistics’ helps humans to understand the link between languages and the environment. Stibbe (2014a) has indicated that ‘linguistics’, particularly, Critical Discourse Studies (CDS), provides an interpretation of language which shapes world views, while the ‘eco’ side maintains the life-sustaining interactions of humans and the physical environment. This paper considers two key ecological instances, namely: The Grand Ethiopian Renaissance Dam (GERD) as a focal point of political dispute and THE LINE project as well as Etthadar lel Akhdar (Go Green Initiative) as two examples of combating ecological degradation. ‘Ecosophy’ as explained by Naess (1996) is used to describe the ecolinguistic framework, which assesses discourse where the linguistic lens focuses on the use of metaphor, and ‘Positive Discourse’ framework, which resonates with respect and care for the natural world.

Keywords: ecosophy, critical discourse studies, metaphor, positive discourse, social semiotics, ecolinguistics

Procedia PDF Downloads 102

Authors: Wen-Hsien Huang, Hsu-Ting Hsu

Abstract:

While advertisers often use animal metaphors to promote product performance, representing through the use of a product image together with an animal-like messenger to imply the undesirable health states of not using the product, the effect of such metaphors on persuasion remains unclear. The current research addresses this issue by investigating how consumers perceive and react to animal metaphor advertising in the context of product promotion. Three studies are carried out using field and experimental data. The findings demonstrate that animal metaphor ads are less persuasive than non-metaphor ads and that ads with animal-like messengers (as opposed to human messengers) activate stronger dehumanization perceptions, which in turn lead to lower product choice, product evaluation and purchase intention, regardless of whether the animal metaphors are presented visually in the picture or verbally in the headline. Furthermore, when the metaphorical pairing includes a more disliked animal, consumer reaction was less favorable. The implications of the findings for advertisers considering the use of animalized messengers are discussed.

Keywords: animal metaphor, dehumanization, product evaluation, health communication

Procedia PDF Downloads 81

Authors: Andrey V. Timofeev, Dmitry V. Egorov

Abstract:

This paper introduces an original method of parametric optimization of the structure for multimodal decision-level fusion scheme which combines the results of the partial solution of the classification task obtained from assembly of the mono-modal classifiers. As a result, a multimodal fusion classifier which has the minimum value of the total error rate has been obtained.

Keywords: classification accuracy, fusion solution, total error rate, multimodal fusion classifier

Procedia PDF Downloads 465

Authors: Fleura Shkëmbi, Valbona Treska

Abstract:

In elementary school, we discover that a metaphor is a decorative linguistic device just for poets. But now that we know, it's also a crucial tactic that individuals employ to understand the universe, from fundamental ideas like time and causation to the most pressing societal challenges today. Metaphor is the use of language to refer to something other than what it was originally intended for or what it "literally" means in order to suggest a similarity or establish a connection between the two. People do not identify metaphors as relevant in their decisions, according to a study on metaphor and its effect on decision-making; instead, they refer to more "substantive" (typically numerical) facts as the basis for their problem-solving decision. Every day, metaphors saturate our lives via language, cognition, and action. They argue that our conceptions shape our views and interactions with others and that concepts define our reality. Metaphor is thus a highly helpful tool for both describing our experiences to others and forming notions for ourselves. In therapeutic contexts, their shared goal appears to be twofold. The cognitivist approach to metaphor regards it as one of the fundamental foundations of human communication. The benefits and disadvantages of utilizing the metaphor differ depending on the target domain that the metaphor portrays. The challenge of creating messages and surroundings that affect customers' notions of abstract ideas in a variety of industries, including health, hospitality, romance, and money, has been studied for decades in marketing and consumer psychology. The aim of this study is to examine, through a systematic literature review, the role of the metaphor in communication and in advertising. This study offers a selected analysis of this literature, concentrating on research on customer attitudes and product appraisal. The analysis of the data identifies potential research questions. With theoretical and applied implications for marketing, design, and persuasion, this study sheds light on how, when, and for whom metaphoric communications are powerful.

Keywords: metaphor, communication, advertising, cognition, action

Procedia PDF Downloads 97

Authors: Muhalim Muhalim

Abstract:

Drawing on Lakoff’s theory of metaphor conceptualization, this article explores the conceptualization of language two writing (L2W) of ten students-teachers in Indonesia via metaphors. The ten postgraduate English language teaching students and at the same time (former) English teachers received seven days of intervention in teaching and learning L2. Using introspective log and focus group discussion, the results illuminate us that all participants are unanimous on perceiving L2W as process-oriented rather than product-oriented activity. Specifically, the metaphor conceptualizations exhibit three categories of process-oriented L2W: deliberate process, learning process, and problem-solving process. However, it has to be clarified from the outset that this categorization is not rigid because some of the properties of metaphors might belong to other categories. Results of the study and implications for English language teaching will be further discussed.

Keywords: metaphor conceptualisation, second language, learning writing, teaching writing

Procedia PDF Downloads 411

Authors: Hadeer M. Sayed, Hesham E. El Deeb, Shereen A. Taie

Abstract:

In the big data era, we are facing a diversity of datasets from different sources in different domains that describe a single life event. These datasets consist of multiple modalities, each of which has a different representation, distribution, scale, and density. Multimodal fusion is the concept of integrating information from multiple modalities in a joint representation with the goal of predicting an outcome through a classification task or regression task. In this paper, multimodal fusion techniques are classified into two main classes: model-agnostic techniques and model-based approaches. It provides a comprehensive study of recent research in each class and outlines the benefits and limitations of each of them. Furthermore, the audiovisual speech recognition task is expressed as a case study of multimodal data fusion approaches, and the open issues through the limitations of the current studies are presented. This paper can be considered a powerful guide for interested researchers in the field of multimodal data fusion and audiovisual speech recognition particularly.

Keywords: multimodal data, data fusion, audio-visual speech recognition, neural networks

Procedia PDF Downloads 110

Authors: Nihar Kanta Patra, Akhtar Hussain

Abstract:

This comprehensive study, through the perspective of librarians, explores the library as a metaphor and its profound significance in representing knowledge and learning. It delves into how librarians perceive the library as a metaphor and the ways in which it symbolizes the acquisition, preservation, and dissemination of knowledge. The research investigates the most common metaphors used to describe libraries, as witnessed by librarians, and analyzes how these metaphors reflect the evolving role of libraries in society. Furthermore, the study examines how the library metaphor influences the perception of librarians regarding academic libraries as physical places and academic library websites as virtual spaces, exploring their potential for learning and exploration. It investigates the evolving nature of the library as a metaphor over time, as seen by librarians, considering the changing landscape of information and technology. The research explores the ways in which the library metaphor has expanded beyond its traditional representation, encompassing digital resources, online connectivity, and virtual realms, and provides insights into its potential evolution in the future. Drawing on the experiences of librarians in their interactions with library users, the study uncovers any specific cultural or generational differences in how people interpret or relate to the library as a metaphor. It sheds light on the diverse perspectives and interpretations of the metaphor based on cultural backgrounds, educational experiences, and technological familiarity. Lastly, the study investigates the evolving roles of libraries as observed by librarians and explores how these changing roles can influence the metaphors we use to represent them. It examines the dynamic nature of libraries as they adapt to societal needs, technological advancements, and new modes of information dissemination. By analyzing these various dimensions, this research provides a comprehensive understanding of the library as a metaphor through the lens of librarians, illuminating its significance, evolution, and its transformative impact on knowledge, learning, and the changing role of libraries in society.

Keywords: library, librarians, metaphor, perception

Procedia PDF Downloads 94

Authors: Uma Kanth, Amy Cook

Abstract:

Conceptual Metaphor Theory (CMT) proposes that metaphor is fundamental to human thought. CMT utilizes embodied cognition, in that emotions are conceptualized as effects on the body because of a coupling of one’s bodily experiences and one’s somatosensory system. Time perception is a function of embodied cognition and conceptual metaphor in that one’s experience of time is inextricably dependent on one’s perception of the world around them. A hallmark of depressive disorders is the distortion in one’s perception of time, such as neurological dysfunction and psychomotor retardation, and yet, to the author’s best knowledge, previous studies have not before linked CMT, embodied cognition, and depressive disorders. Therefore, the focus of this paper is the investigation of how the applications of CMT and embodied cognition (especially regarding time perception) have promise in improving current techniques to treat depressive disorders. This paper aimed to extend, through a thorough review of literature, the theoretical basis required to further research into CMT and embodied cognition’s application in treating time distortion related symptoms of depressive disorders. Future research could include the development of brain training technologies that capitalize on the principles of CMT, with the aim of promoting cognitive remediation and cognitive activation to mitigate symptoms of depressive disorder.

Keywords: depression, conceptual metaphor theory, embodied cognition, time

Procedia PDF Downloads 161

Authors: S. Pesina, T. Solonchak

Abstract:

An analysis of word semantics focusing on the invariance of advanced imagery in several pressing problems. Interest in the language of imagery is caused by the introduction, in the linguistics sphere, of a new paradigm, the center of which is the personality of the speaker (the subject of the language). Particularly noteworthy is the question of the place of the image when discussing the lexical, phraseological values and the relationship of imagery and metaphors. In part, the formation of a metaphor, as an interaction between two intellective entities, occurs at a cognitive level, and it is the category of the image, having cognitive roots, which aides in the correct interpretation of the results of this process on the lexical-semantic level.

Keywords: image, metaphor, concept, creation of a metaphor, cognitive linguistics, erased image, vivid image

Procedia PDF Downloads 360

Authors: Afef Badri

Abstract:

The role of metaphor in creating the coherence and the cohesion of discourse in online interactive talk has almost received no attention. This paper intends to provide some theoretical reflections on metaphorical coherence as a jointly constructed process that evolves in online, face-to-face interactions. It suggests that the presence of a global conceptual structure in a conversation makes it conceptually cohesive. Yet, coherence remains a process largely determined by other variables (shared goals, communicative intentions, and framework of understanding). Metaphorical coherence created by these variables can be useful in detecting bias in media reporting.

Keywords: coherence, cohesion, face-to-face interactions, metaphor

Procedia PDF Downloads 246

Authors: Stefanie Rukavina, Sascha Gruss, Steffen Walter, Holger Hoffmann, Harald C. Traue

Abstract:

OPEN-EmoRecII is an open multimodal corpus with experimentally induced emotions. In the first half of the experiment, emotions were induced with standardized picture material and in the second half during a human-computer interaction (HCI), realized with a wizard-of-oz design. The induced emotions are based on the dimensional theory of emotions (valence, arousal and dominance). These emotional sequences - recorded with multimodal data (mimic reactions, speech, audio and physiological reactions) during a naturalistic-like HCI-environment one can improve classification methods on a multimodal level. This database is the result of an HCI-experiment, for which 30 subjects in total agreed to a publication of their data including the video material for research purposes. The now available open corpus contains sensory signal of: video, audio, physiology (SCL, respiration, BVP, EMG Corrugator supercilii, EMG Zygomaticus Major) and mimic annotations.

Keywords: open multimodal emotion corpus, annotated labels, intelligent interaction

Procedia PDF Downloads 414

Authors: A. Kebbeb, M. Mostefai, F. Benmerzoug, Y. Chahir

Abstract:

The multimodal biometric identification is the combination of several biometric systems. The challenge of this combination is to reduce some limitations of systems based on a single modality while significantly improving performance. In this paper, we propose a new approach to the construction and the protection of a multimodal biometric database dedicated to an identification system. We use a topological watermarking to hide the relation between face image and the registered descriptors extracted from other modalities of the same person for more secure user identification.

Keywords: biometric databases, multimodal biometrics, security authentication, digital watermarking

Procedia PDF Downloads 388

Authors: Fuling Deng

Abstract:

Today’s children are frequently exposed to multimodal texts that adopt diverse modes to communicate myriad meanings within different cultural contexts. To respond to the new textual landscape, scholars have considered new literacy theories which propose picturebooks as important educational resources. Picturebooks are multimodal, with their meaning conveyed through the synchronisation of multiple modes, including linguistic, visual, spatial, and gestural acting as access to multimodal literacy. Picturebooks have been popular reading materials in primary educational settings in China. However, often viewed as “easy” texts directed at the youngest readers, picturebooks remain on the margins of Chinese upper primary classrooms, where they are predominantly used for linguistic tasks, with little value placed on their multimodal affordances. Practices with picturebooks in the upper grades in Chinese primary schools also encounter many challenges associated with the curation of texts for use, designing curriculum, and assessment. To respond to these issues, a qualitative study was conducted with a community of Chinese primary teachers using multi-methods such as interviews, focus groups, and documents. The findings showed the impact of the teachers’ increased awareness of picturebooks' multimodal affordances on their pedagogical decisions in using picturebooks as educational resources in upper primary classrooms.

Keywords: picturebook education, multimodal literacy, teachers' response to contemporary picturebooks, community of practice

Procedia PDF Downloads 136

Authors: Konstantinos Perifanos, Eirini Florou, Dionysis Goutsos

Abstract:

This paper presents and benchmarks a number of end-to-end Deep Learning based models for metaphor detection in Greek. We combine Convolutional Neural Networks and Recurrent Neural Networks with representation learning to bear on the metaphor detection problem for the Greek language. The models presented achieve exceptional accuracy scores, significantly improving the previous state-of-the-art results, which had already achieved accuracy 0.82. Furthermore, no special preprocessing, feature engineering or linguistic knowledge is used in this work. The methods presented achieve accuracy of 0.92 and F-score 0.92 with Convolutional Neural Networks (CNNs) and bidirectional Long Short Term Memory networks (LSTMs). Comparable results of 0.91 accuracy and 0.91 F-score are also achieved with bidirectional Gated Recurrent Units (GRUs) and Convolutional Recurrent Neural Nets (CRNNs). The models are trained and evaluated only on the basis of training tuples, the related sentences and their labels. The outcome is a state-of-the-art collection of metaphor detection models, trained on limited labelled resources, which can be extended to other languages and similar tasks.

Keywords: metaphor detection, deep learning, representation learning, embeddings

Procedia PDF Downloads 152

Authors: Victoria L. Malakhova

Abstract:

The research is devoted to multimodal content and its effectiveness in developing students’ linguistic and intercultural communicative competences as an indefeasible constituent of their future professional activity. Description of multimodal content both as a linguistic and didactic phenomenon makes the study relevant. The objective of the article is the analysis of creolized texts and the effect they have on fostering higher education students’ skills and their productivity. The main methods used are linguistic text analysis, qualitative and quantitative methods, deduction, generalization. The author studies texts with full and partial creolization, their features and role in composing multimodal textual space. The main verbal and non-verbal markers and paralinguistic means that enhance the linguo-pragmatic potential of creolized texts are covered. To reveal the efficiency of multimodal content application in English teaching, the author conducts an experiment among both undergraduate students and teachers. This allows specifying main functions of creolized texts in the process of language learning, detecting ways of enhancing students’ competences, and increasing their motivation. The described stages of using creolized texts can serve as an algorithm for work with multimodal content in teaching English as a foreign language. The findings contribute to improving the efficiency of the academic process.

Keywords: creolized text, English language learning, higher education, language and communication competences, multimodal content

Procedia PDF Downloads 110

Authors: Huang Yajing

Abstract:

Multimodal discourse refers to the phenomenon of using various senses such as hearing, vision, and touch to communicate through various means and symbolic resources such as language, images, sounds, and movements. With the development of modern technology and multimedia, language and technology have become inseparable, and foreign language teaching is becoming more and more modal. Teacher-student communication resorts to multiple senses and uses multiple symbol systems to construct and interpret meaning. The classroom is a semiotic space where multimodal discourses are intertwined. College English multi-modal teaching is to rationally utilize traditional teaching methods while mobilizing and coordinating various modern teaching methods to form a joint force to promote teaching and learning. Multimodal teaching makes full and reasonable use of various meaning resources and can maximize the advantages of multimedia and network environments. Based upon the above theories about multimodal discourse and multimedia technology, the present paper will propose a multi-modal teaching model for college English in China.

Keywords: multimodal discourse, multimedia technology, English education, applied linguistics

Procedia PDF Downloads 66

Authors: Dian Guan

Abstract:

With the wide application of multimedia and the Internet, the development of teaching theories, and the implementation of teaching reforms, many different university English classroom teaching modes have emerged. The university English teaching mode is changing from the traditional teaching mode based on conversation and text to the multimodal English teaching mode containing discussion, pictures, audio, film, etc. Applying university English teaching models is conducive to cultivating lifelong learning skills. In addition, lifelong learning skills can also be called learners' autonomous learning skills. Learners' independent learning ability has a significant impact on English learning. However, many university students, especially art and design students, don't know how to learn individually. When they become university students, their English foundation is a relative deficiency because they always remember the language in a traditional way, which, to a certain extent, neglects the cultivation of English learners' independent ability. As a result, the autonomous learning ability of most university students is not satisfactory. The participants in this study were 60 students and one teacher in their first year at a university in western China. Two observations and interviews were conducted inside and outside the classroom to understand the impact of a multimodal teaching model of university English on students' autonomous learning ability. The results were analyzed, and it was found that the multimodal teaching model of university English significantly affected learners' autonomy. Incorporating classroom presentations and poster exhibitions into multimodal teaching can increase learners' interest in learning and enhance their learning ability outside the classroom. However, further exploration is needed to develop multimodal teaching materials and evaluate multimodal teaching outcomes. Despite the limitations of this study, the study adopts a scientific research method to analyze the impact of the multimodal teaching mode of university English on students' independent learning ability. It puts forward a different outlook for further research on this topic.

Keywords: art university, EFL education, learner autonomy, multimodal pedagogy

Procedia PDF Downloads 99

Authors: Dayo Samuel Banjo, Connice Trimmingham, Niloofar Yousefi, Nitin Agarwal

Abstract:

Technological advancement and its omnipresent connection have pushed humans past the boundaries and limitations of a computer screen, physical state, or geographical location. It has provided a depth of avenues that facilitate human-computer interaction that was once inconceivable such as audio and body language detection. Given the complex modularities of emotions, it becomes vital to study human-computer interaction, as it is the commencement of a thorough understanding of the emotional state of users and, in the context of social networks, the producers of multimodal information. This study first acknowledges the accuracy of classification found within multimodal emotion detection systems compared to unimodal solutions. Second, it explores the characterization of multimedia content produced based on their emotions and the coherence of emotion in different modalities by utilizing deep learning models to classify emotion across different modalities.

Keywords: affective computing, deep learning, emotion recognition, multimodal

Procedia PDF Downloads 155

Authors: Dorit Lemberger

Abstract:

Wittgenstein directly used the term "metaphor" only infrequently and with reservations, but his writings include a number of metaphors that have become imprinted in the philosophical memory of Western thought. For example, the ladder in his book Tractatus, or in Philosophical investigations - the ancient city, the beetle in a box, the fly in the fly-bottle, and the duck-rabbit. In light of Wittgenstein's stressing, throughout his investigations, that the only language that exists is ordinary language, and that there is no "second-order" language, the question should be asked: How do these metaphors function, specifically, and in general, how are we to relate to language use that exceeds the normal? Wittgenstein did not disregard such phenomena, but he proposed viewing them in a different way, that would enable understanding them as uses in ordinary language, without necessarily exceeding such language. Two important terms that he coined in this context are "secondary sense" and "experience of meaning". Each denotes language use as reflective of a subjective element characteristic of the speaker, such as intent, experience, or emphasis of a certain aspect. More recent Wittgenstein scholars added the term "quasi-metaphor", that refers to his discussion of the possibility of aesthetic judgment. This paper will examine how, according to Wittgenstein, these terms function without exceeding ordinary language, and will illustrate how they can be applied, in an analysis of the poem "Butterfly" by Nelly Sachs.

Keywords: metaphor, quasi-metaphor, secondary sense, experience of meaning

Procedia PDF Downloads 437