Search results for: Bert Seither

Authors: Andrey Khalov

Abstract:

The rapid proliferation of unstructured data in IT infrastructure management demands innovative approaches for extracting actionable knowledge. This paper presents a framework for ontology-based knowledge extraction that combines relational graph neural networks (R-GNN) with large language models (LLMs). The proposed method leverages the DOLCE framework as the foundational ontology, extending it with concepts from ITSMO for domain-specific applications in IT service management and outsourcing. A key component of this research is the use of transformer-based models, such as DeBERTa-v3-large, for automatic entity and relationship extraction from unstructured texts. Furthermore, the paper explores how transfer learning techniques can be applied to fine-tune large language models (LLaMA) for using to generate synthetic datasets to improve precision in BERT-based entity recognition and ontology alignment. The resulting IT Ontology (ITO) serves as a comprehensive knowledge base that integrates domain-specific insights from ITIL processes, enabling more efficient decision-making. Experimental results demonstrate significant improvements in knowledge extraction and relationship mapping, offering a cutting-edge solution for enhancing cognitive computing in IT service environments.

Keywords: ontology mapping, R-GNN, knowledge extraction, large language models, NER, knowlege graph

Procedia PDF Downloads 16

Authors: Bowen Ding, Yihao Kuang

Abstract:

In recent years, large-scale pre-trained language models have achieved state-of-the-art performance on a variety of natural language processing tasks. The word vectors produced by these language models can be viewed as dense encoded presentations of natural language that in text form. However, it is unknown how much linguistic information is encoded and how. In this paper, we construct several corresponding probing tasks for multiple linguistic information to clarify the encoding capabilities of different language models and performed a visual display. We firstly obtain word presentations in vector form from different language models, including BERT, ELMo, RoBERTa and GPT. Classifiers with a small scale of parameters and unsupervised tasks are then applied on these word vectors to discriminate their capability to encode corresponding linguistic information. The constructed probe tasks contain both semantic and syntactic aspects. The semantic aspect includes the ability of the model to understand semantic entities such as numbers, time, and characters, and the grammatical aspect includes the ability of the language model to understand grammatical structures such as dependency relationships and reference relationships. We also compare encoding capabilities of different layers in the same language model to infer how linguistic information is encoded in the model.

Keywords: language models, probing task, text presentation, linguistic information

Procedia PDF Downloads 110

Authors: Tomás Speziale

Abstract:

This paper aims to analyse how Ian Hacking states the theoretical basis of his research on the classification of people. Although all his early philosophical education had been based in Foucault, it is also true that Erving Goffman’s perspective provided him with epistemological and methodological tools for understanding face-to-face relationships. Hence, all his works must be thought of as social science texts that combine the research on how the individuals are constituted ‘top-down’ (as in Foucault), with the inquiry into how people renegotiate ‘bottom-up’ the classifications about them. Thus, Hacking´s proposal constitutes a middle ground between the French Philosopher and the American Sociologist. Placing himself between both authors allows Hacking to build a frame that is expected to adjust to Social Sciences’ main particularity: the fact that they study interactive kinds. These are kinds of people, which imply that those who are classified can change in certain ways that prompt the need for changing previous classifications themselves. It is all about the interaction between the labelling of people and the people who are classified. Consequently, understanding the way in which Hacking uses Foucault’s and Goffman’s theories is essential to fully comprehend the social dynamic between individuals and concepts, what Bert Hansen had called dialectical realism. His theoretical proposal, therefore, is not only valuable because it combines diverse perspectives, but also because it constitutes an utterly original and relevant framework for Sociological theory and particularly for Criminology.

Keywords: classification of people, Foucault's archaeology, Goffman's interpersonal sociology, interactive kinds

Procedia PDF Downloads 343

Authors: Bert H. Jacobson, Taylor Monaghan, John Sellers

Abstract:

Intense resistance-type activity generally elicits delayed onset muscle soreness (DOMS) in individuals unaccustomed to such action. DOMS is a combination of contractile tissue microtrauma, osmotic pressure changes, alteration calcium regulation, and inflammation. Elevated muscle-specific enzyme creatine kinase (CK) is a marker of striated muscle damage. Avian immunoglobulin (IgY) mediates inflammation and may thereby reduce post-exercise DOMS. Purpose: The aim of this study was to compare the effect of oral IgY and placebo (Pl) on CK, serum relevels, and perceived pain following induced DOMS. Methods: Healthy college-aged females (N=16) were randomly divided into an experimental group (IgY) and a control group (PL). CK serum levels were recorded followed by 14 days of supplementation of either IgY or Pl at the following doses: days 1-2 =4.5 g, days 3-5 =9.0 g, and days 6-14 =13.5 g. Following the 14 d, lower limb DOMS was induced using two methods of resistance training. After 48 hours, subjects reported for a second blood draw. Results: One-way ANOVA resulted in the IgY group posting significantly less (p < 0.05) serum CK than the PL group. Furthermore, the IgY group experienced significantly less post-test perceived soreness than the Pl group. Conclusion: IgY supplementation lessens muscle CK levels and perceived muscle soreness following exercise, possibly due to an anti-inflammatory effect. It was suggested that IgY may serve as a buffer for DOMS thereby allowing the participant to continue vigorous exercise without discomfort.

Keywords: muscle, soreness, damage, serum

Procedia PDF Downloads 197

Authors: Qin Long, Li Xiaoge

Abstract:

The Knowledge Graphs have now become a new form of knowledge representation. However, there is no consensus in regard to a plausible and definition of entities and relationships in the domain-specific knowledge graph. Further, in conjunction with several limitations and deficiencies, various domain-specific entities and relationships recognition approaches are far from perfect. Specifically, named entity recognition in Chinese domain is a critical task for the natural language process applications. However, a bottleneck problem with Chinese named entity recognition in new domains is the lack of annotated data. To address this challenge, a domain distant supervised named entity recognition framework is proposed. The framework is divided into two stages: first, the distant supervised corpus is generated based on the entity linking model of graph attention neural network; secondly, the generated corpus is trained as the input of the distant supervised named entity recognition model to train to obtain named entities. The link model is verified in the ccks2019 entity link corpus, and the F1 value is 2% higher than that of the benchmark method. The re-pre-trained BERT language model is added to the benchmark method, and the results show that it is more suitable for distant supervised named entity recognition tasks. Finally, it is applied in the computer field, and the results show that this framework can obtain domain named entities.

Keywords: distant named entity recognition, entity linking, knowledge graph, graph attention neural network

Procedia PDF Downloads 93

Authors: Jinan Fiaidhi, Hashmath Shaik

Abstract:

Breaking the barrier for practicing evidence-based medicine relies on effective methods for rapidly identifying relevant evidence from the body of biomedical literature. An important challenge confronted by medical practitioners is the long time needed to browse, filter, summarize and compile information from different medical resources. Deep learning can help in solving this based on automatic question answering (Q&A) and transformers. However, Q&A and transformer technologies are not trained to answer clinical queries that can be used for evidence-based practice, nor can they respond to structured clinical questioning protocols like PICO (Patient/Problem, Intervention, Comparison and Outcome). This article describes the use of deep learning techniques for Q&A that are based on transformer models like BERT and GPT to answer PICO clinical questions that can be used for evidence-based practice extracted from sound medical research resources like PubMed. We are reporting acceptable clinical answers that are supported by findings from PubMed. Our transformer methods are reaching an acceptable state-of-the-art performance based on two staged bootstrapping processes involving filtering relevant articles followed by identifying articles that support the requested outcome expressed by the PICO question. Moreover, we are also reporting experimentations to empower our bootstrapping techniques with patch attention to the most important keywords in the clinical case and the PICO questions. Our bootstrapped patched with attention is showing relevancy of the evidence collected based on entropy metrics.

Keywords: automatic question answering, PICO questions, evidence-based medicine, generative models, LLM transformers

Procedia PDF Downloads 43

Authors: Shahad Nagoor, Lucy Hederman, Kevin Koidl, Annalina Caputo

Abstract:

Doctors’ notes reflect their impressions, attitudes, clinical sense, and opinions about patients’ conditions and progress, and other information that is essential for doctors’ daily clinical decisions. Despite their value, clinical notes are insufficiently researched within the language processing community. Automatically extracting information from unstructured text data is known to be a difficult task as opposed to dealing with structured information such as vital physiological signs, images, and laboratory results. The aim of this research is to investigate how Natural Language Processing (NLP) techniques and machine learning techniques applied to clinician notes can assist in doctors’ decision-making in Intensive Care Unit (ICU) for coronavirus disease 2019 (COVID-19) patients. The hypothesis is that clinical outcomes like survival or mortality can be useful in influencing the judgement of clinical sentiment in ICU clinical notes. This paper introduces two contributions: first, we introduce COVID_ICU_BERT, a fine-tuned version of clinical transformer models that can reliably predict clinical sentiment for notes of COVID patients in the ICU. We train the model on clinical notes for COVID-19 patients, a type of notes that were not previously seen by clinicalBERT, and Bio_Discharge_Summary_BERT. The model, which was based on clinicalBERT achieves higher predictive accuracy (Acc 93.33%, AUC 0.98, and precision 0.96 ). Second, we perform data augmentation using clinical contextual word embedding that is based on a pre-trained clinical model to balance the samples in each class in the data (survived vs. deceased patients). Data augmentation improves the accuracy of prediction slightly (Acc 96.67%, AUC 0.98, and precision 0.92 ).

Keywords: BERT fine-tuning, clinical sentiment, COVID-19, data augmentation

Procedia PDF Downloads 206

Authors: Ammarah Irum, Muhammad Ali Tahir

Abstract:

Document-level sentiment analysis in Urdu is a challenging Natural Language Processing (NLP) task due to the difficulty of working with lengthy texts in a language with constrained resources. Deep learning models, which are complex neural network architectures, are well-suited to text-based applications in addition to data formats like audio, image, and video. To investigate the potential of deep learning for Urdu sentiment analysis, we implemented five different deep learning models, including Bidirectional Long Short Term Memory (BiLSTM), Convolutional Neural Network (CNN), Convolutional Neural Network with Bidirectional Long Short Term Memory (CNN-BiLSTM), and Bidirectional Encoder Representation from Transformer (BERT). In this study, we developed a hybrid deep learning model called BiLSTM-Single Layer Multi Filter Convolutional Neural Network (BiLSTM-SLMFCNN) by fusing BiLSTM and CNN architecture. The proposed and baseline techniques are applied on Urdu Customer Support data set and IMDB Urdu movie review data set by using pre-trained Urdu word embedding that are suitable for sentiment analysis at the document level. Results of these techniques are evaluated and our proposed model outperforms all other deep learning techniques for Urdu sentiment analysis. BiLSTM-SLMFCNN outperformed the baseline deep learning models and achieved 83%, 79%, 83% and 94% accuracy on small, medium and large sized IMDB Urdu movie review data set and Urdu Customer Support data set respectively.

Keywords: urdu sentiment analysis, deep learning, natural language processing, opinion mining, low-resource language

Procedia PDF Downloads 72

Authors: Linda Blümel, Bettina Bert, Jan Brosda, Heidrun Fink, Melanie Hamann

Abstract:

Age-related cognitive decline can eventually lead to dementia, the most common mental illness in elderly people and an immense challenge for patients, their families and caregivers. Cholinesterase inhibitors constitute the most commonly used antidementia prescription medication. The standardized Ginkgo biloba leaf extract EGb 761® is approved for treating age-associated cognitive impairment and has been shown to improve the quality of life in patients suffering from mild dementia. A clinical trial with 96 Alzheimer´s disease patients indicated that the combined treatment with donepezil and EGb 761® had fewer side effects than donepezil alone. In an animal model of cognitive aging, we compared the effect of combined treatment with EGb 761® or donepezil monotherapy and vehicle. We compared the effect of chronic treatment (15 days of pretreatment) with donepezil (1.5 mg/kg p. o.), EGb 761® (100 mg/kg p. o.), or the combination of the two drugs, or vehicle in 18 – 20 month old male OFA rats. Learning and memory performance were assessed by Morris water maze testing, motor behavior in an open field paradigm. In addition to chronic treatment, the substances were administered orally 30 minutes before testing. Compared to the first day and to the control group, only the combination group showed a significant reduction in latency to reach the hidden platform on the second day of testing. Moreover, from the second day of testing onwards, the donepezil, the EGb 761® and the combination group required less time to reach the hidden platform compared to the first day. The control group did not reach the same latency reduction until day three. There were no effects on motor behavior. These results suggest a superiority of the combined treatment of donepezil with EGb 761® compared to monotherapy.

Keywords: age-related cognitive decline, dementia, ginkgo biloba leaf extract EGb 761®, learning and memory, old rats

Procedia PDF Downloads 368

Authors: Mst Shapna Akter, Hossain Shahriar

Abstract:

One of the most important challenges in the field of software code audit is the presence of vulnerabilities in software source code. Every year, more and more software flaws are found, either internally in proprietary code or revealed publicly. These flaws are highly likely exploited and lead to system compromise, data leakage, or denial of service. C and C++ open-source code are now available in order to create a largescale, machine-learning system for function-level vulnerability identification. We assembled a sizable dataset of millions of opensource functions that point to potential exploits. We developed an efficient and scalable vulnerability detection method based on deep neural network models that learn features extracted from the source codes. The source code is first converted into a minimal intermediate representation to remove the pointless components and shorten the dependency. Moreover, we keep the semantic and syntactic information using state-of-the-art word embedding algorithms such as glove and fastText. The embedded vectors are subsequently fed into deep learning networks such as LSTM, BilSTM, LSTM-Autoencoder, word2vec, BERT, and GPT-2 to classify the possible vulnerabilities. Furthermore, we proposed a neural network model which can overcome issues associated with traditional neural networks. Evaluation metrics such as f1 score, precision, recall, accuracy, and total execution time have been used to measure the performance. We made a comparative analysis between results derived from features containing a minimal text representation and semantic and syntactic information. We found that all of the deep learning models provide comparatively higher accuracy when we use semantic and syntactic information as the features but require higher execution time as the word embedding the algorithm puts on a bit of complexity to the overall system.

Keywords: cyber security, vulnerability detection, neural networks, feature extraction

Procedia PDF Downloads 89

Authors: Monica Gonzalez Machorro

Abstract:

Dementia is hard to diagnose because of the lack of early physical symptoms. Early dementia recognition is key to improving the living condition of patients. Speech technology is considered a valuable biomarker for this challenge. Recent works have utilized conventional acoustic features and machine learning methods to detect dementia in speech. BERT-like classifiers have reported the most promising performance. One constraint, nonetheless, is that these studies are either based on human transcripts or on transcripts produced by automatic speech recognition (ASR) systems. This research contribution is to explore a method that does not require transcriptions to detect early Alzheimer’s disease (AD) and mild cognitive impairment (MCI). This is achieved by fine-tuning a pre-trained ASR model for the downstream early AD and MCI tasks. To do so, a subset of the thoroughly studied Pitt Corpus is customized. The subset is balanced for class, age, and gender. Data processing also involves cropping the samples into 10-second segments. For comparison purposes, a baseline model is defined by training and testing a Random Forest with 20 extracted acoustic features using the librosa library implemented in Python. These are: zero-crossing rate, MFCCs, spectral bandwidth, spectral centroid, root mean square, and short-time Fourier transform. The baseline model achieved a 58% accuracy. To fine-tune HuBERT as a classifier, an average pooling strategy is employed to merge the 3D representations from audio into 2D representations, and a linear layer is added. The pre-trained model used is ‘hubert-large-ls960-ft’. Empirically, the number of epochs selected is 5, and the batch size defined is 1. Experiments show that our proposed method reaches a 69% balanced accuracy. This suggests that the linguistic and speech information encoded in the self-supervised ASR-based model is able to learn acoustic cues of AD and MCI.

Keywords: automatic speech recognition, early Alzheimer’s recognition, mild cognitive impairment, speech impairment

Procedia PDF Downloads 127

Authors: Lien Gysens, Bert Vanmechelen, Maarten Haspeslagh, Piet Maes, Ann Martens

Abstract:

Bovine papillomavirus (BPV) types 1 and 2 play a central role in the etiology of the most common neoplasm in horses, the equine sarcoid. The unknown mechanism behind the unique variety in a clinical presentation on the one hand and the host-dependent clinical outcome of BPV-1 infection, on the other hand, indicate the involvement of additional factors. Earlier studies have reported the potential functional significance of intratypic sequence variants, along with the existence of sarcoid-sourced BPV variants. Therefore, intratypic sequence variation seems to be an important emerging viral factor. This study aimed to give a broad insight in sarcoid-sourced BPV variation and explore its potential association with disease presentation. In order to do this, a nanopore sequencing approach was successfully optimized for screening a wide spectrum of clinical samples. Specimens of each tumour were initially screened for BPV-1/-2 by quantitative real-time PCR. A custom-designed primer set was used on BPV-positive samples to amplify the complete viral genome in two multiplex PCR reactions, resulting in a set of overlapping amplicons. For phylogenetic analysis, separate alignments were made of all available complete genome sequences for BPV-1/-2. The resulting alignments were used to infer Bayesian phylogenetic trees. We found substantial genetic variation among sarcoid-derived BPV-1, although this variation could not be linked to disease severity. Several of the BPV-1 genomes had multiple major deletions. Remarkably, the majority of the cluster within the region coding for late viral genes. Together with the extensiveness (up to 603 nucleotides) of the described deletions, this suggests an altered function of L1/L2 in disease pathogenesis. By generating a significant amount of complete-length BPV genomes, we succeeded in introducing next-generation sequencing into veterinary research focusing on the equine sarcoid, thus facilitating the first report of both nanopore-based sequencing of complete sarcoid-sourced BPV-1/-2 and the simultaneous nanopore sequencing of multiple complete genomes originating from a single clinical sample.

Keywords: Bovine papillomavirus, equine sarcoid, horse, nanopore sequencing, phylogenetic analysis

Procedia PDF Downloads 178

Authors: B. F. Espiritu

Abstract:

This paper examines the resistance of the Lumad people, the indigenous peoples in Mindanao, Southern Philippines, and of environmental and human rights activists to the Philippine government's neoliberal policies and their call for justice and a new economic paradigm that will uphold peoples' rights and environmental protection in two alternative media online sites. The study contributes to the body of knowledge on indigenous resistance to neoliberal globalization and the quest for a new economic paradigm that upholds social justice for the marginalized in society, empathy and compassion for those who depend on the land for their survival, and environmental sustainability. The study analyzes the discourses in selected news articles from Davao Today and Kalikasan (translated to English as 'Nature') People's Network for the Environment’s statements and advocacy articles for the Lumad and the environment from 2018 to February 2020. The study reveals that the alternative media news articles and the advocacy articles contain statements that expose the oppression and violation of human rights of the Lumad people, farmers, government environmental workers, and environmental activists as shown in their killings, illegal arrest and detention, displacement of the indigenous peoples, destruction of their schools by the military and paramilitary groups, and environmental plunder and destruction with the government's permit for the entry and operation of extractive and agribusiness industries in the Lumad ancestral lands. Anchored on Christian Fuch's theory of alternative media as critical media and Bert Cammaerts' theorization of alternative media as counterhegemonic media that are part of civil society and form a third voice between state media and commercial media, the study reveals the counterhegemonic discourses of the news and advocacy articles that oppose the dominant economic system of neoliberalism which oppresses the people who depend on the land for their survival. Furthermore, the news and advocacy articles seek to advance social struggles that transform society towards the realization of cooperative potentials or a new economic paradigm that upholds economic democracy, where the local people, including the indigenous people, are economically empowered their environment and protected towards the realization of self-sustaining communities. The study highlights the call for justice, empathy, and compassion for both the people and the environment and the need for a new economic paradigm wherein indigenous peoples and local communities are empowered towards becoming self-sustaining communities in a sustainable environment.

Keywords: alternative media, environmental sustainability, human rights, indigenous resistance

Procedia PDF Downloads 143

Authors: Shrabani Ghosh

Abstract:

Automatic offensive language detection in social media has become a stirring task in today's NLP. Manual Offensive language detection is tedious and laborious work where automatic methods based on machine learning are only alternatives. Previous works have done sentiment analysis over social media in different ways such as supervised, semi-supervised, and unsupervised manner. Domain adaptation in a semi-supervised way has also been explored in NLP, where the source domain and the target domain are different. In domain adaptation, the source domain usually has a large amount of labeled data, while only a limited amount of labeled data is available in the target domain. Pretrained transformers like BERT, RoBERTa models are fine-tuned to perform text classification in an unsupervised manner to perform further pre-train masked language modeling (MLM) tasks. In previous work, hate speech detection has been explored in Gab.ai, which is a free speech platform described as a platform of extremist in varying degrees in online social media. In domain adaptation process, Twitter data is used as the source domain, and Gab data is used as the target domain. The performance of domain adaptation also depends on the cross-domain similarity. Different distance measure methods such as L2 distance, cosine distance, Maximum Mean Discrepancy (MMD), Fisher Linear Discriminant (FLD), and CORAL have been used to estimate domain similarity. Certainly, in-domain distances are small, and between-domain distances are expected to be large. The previous work finding shows that pretrain masked language model (MLM) fine-tuned with a mixture of posts of source and target domain gives higher accuracy. However, in-domain performance of the hate classifier on Twitter data accuracy is 71.78%, and out-of-domain performance of the hate classifier on Gab data goes down to 56.53%. Recently self-supervised learning got a lot of attention as it is more applicable when labeled data are scarce. Few works have already been explored to apply self-supervised learning on NLP tasks such as sentiment classification. Self-supervised language representation model ALBERTA focuses on modeling inter-sentence coherence and helps downstream tasks with multi-sentence inputs. Self-supervised attention learning approach shows better performance as it exploits extracted context word in the training process. In this work, a self-supervised attention mechanism has been proposed to detect hate speech on Gab.ai. This framework initially classifies the Gab dataset in an attention-based self-supervised manner. On the next step, a semi-supervised classifier trained on the combination of labeled data from the first step and unlabeled data. The performance of the proposed framework will be compared with the results described earlier and also with optimized outcomes obtained from different optimization techniques.

Keywords: attention learning, language model, offensive language detection, self-supervised learning

Procedia PDF Downloads 105

Authors: Waisea Votadroka, Bert Van Bavel

Abstract:

The most commonly consumed shellfish species produced in the Pacific, shellfish and tuna fish, were investigated for the occurrence of a range of brominated and chlorinated contaminants in order to establish current levels. Polychlorinated biphenyls (PCBs), polybrominated diphenyl ethers (PBDEs) and polychlorinated dibenzo-p-dioxins and dibenzofurans (PCDD/Fs) were analysed in the muscle of tuna species Katsuwonis pelamis, yellow fin tuna, and shellfish species from the Fiji Islands. The investigation of polychlorinated biphenyls (PCBs), furans (PCDFs) and polybrominated diphenylethers (PBDE No.47) in tuna and shellfish in Kiribati, Solomon Islands and Fiji is necessary due to the lack of research data in the Pacific region. The health risks involved in the consumption of marine foods laced with toxic organo-chlorinated and brominated compounds makes in the analyses of these compounds in marine foods important particularly when Pacific communities rely on these resources as their main diet. The samples were homogenized in a motor with anhydrous sodium sulphate in the ratio of 1:3 (muscle) and 1:4-1:5 (roe and butter). The tuna and shellfish samples were homogenized and freeze dried at the sampling location at the Institute of Applied Science, Fiji. All samples were stored in amber glss jars at -18 ° C until extraction at Orebro University. PCDD/Fs, PCBs and pesticides were all analysed using an Autospec Ultina HRGC/HRMS operating at 10,000 resolutions with EI ionization at 35 eV. All the measurements were performed in the selective ion recording mode (SIR), monitoring the two most abundant ions of the molecular cluster (PCDD/Fs and PCBs). Results indicated that the Fiji Composite sample for Batissa violacea range 0.7-238.6 pg/g lipid; Fiji sample composite Anadara antiquate range 1.6 – 808.6 pg/g lipid; Solomon Islands Katsuwonis Pelamis 7.5-3770.7 pg/g lipid; Solomon Islands Yellow Fin tuna 2.1 -778.4 pg/g lipid; Kiribati Katsuwonis Pelamis 4.8-1410 pg/g lipids. The study has demonstrated that these species are good bio-indicators of the presence of these toxic organic pollutants in edible marine foods. Our results suggest that for pesticides levels, p,p-DDE is the most dominant for all the groups and seems to be highest at 565.48 pg/g lipid in composite Batissa violacea from Fiji. For PBDE no.47 in comparing all samples, the composite Batissa violacea from Fiji had the highest level of 118.20 pg/g lipid. Based upon this study, the contamination levels found in the study species were quite lower compared with levels reported in impacted ecosystems around the world

Keywords: polychlorinated biphenyl, polybrominated diphenylethers, pesticides, organoclorinated pesticides, PBDEs

Procedia PDF Downloads 383