Search results for: Swin transformer

Authors: Jimeng Shi, Mahek Jain, Giri Narasimhan

Abstract:

Time Series Forecasting (TSF) is used to predict the target variables at a future time point based on the learning from previous time points. To keep the problem tractable, learning methods use data from a fixed-length window in the past as an explicit input. In this paper, we study how the performance of predictive models changes as a function of different look-back window sizes and different amounts of time to predict the future. We also consider the performance of the recent attention-based Transformer models, which have had good success in the image processing and natural language processing domains. In all, we compare four different deep learning methods (RNN, LSTM, GRU, and Transformer) along with a baseline method. The dataset (hourly) we used is the Beijing Air Quality Dataset from the UCI website, which includes a multivariate time series of many factors measured on an hourly basis for a period of 5 years (2010-14). For each model, we also report on the relationship between the performance and the look-back window sizes and the number of predicted time points into the future. Our experiments suggest that Transformer models have the best performance with the lowest Mean Average Errors (MAE = 14.599, 23.273) and Root Mean Square Errors (RSME = 23.573, 38.131) for most of our single-step and multi-steps predictions. The best size for the look-back window to predict 1 hour into the future appears to be one day, while 2 or 4 days perform the best to predict 3 hours into the future.

Keywords: air quality prediction, deep learning algorithms, time series forecasting, look-back window

Procedia PDF Downloads 154

Authors: Minh-Thuan Nguyen, Phuong-Thai Nguyen, Van-Vinh Nguyen, Quang-Minh Nguyen

Abstract:

Research on automatically generating descriptions for e-commerce products is gaining increasing attention in recent years. However, the generated descriptions of their systems are often less informative and attractive because of lacking training datasets or the limitation of these approaches, which often use templates or statistical methods. In this paper, we explore a method to generate production descriptions by using the GPT-2 model. In addition, we apply text paraphrasing and task-adaptive pretraining techniques to improve the qualify of descriptions generated from the GPT-2 model. Experiment results show that our models outperform the baseline model through automatic evaluation and human evaluation. Especially, our methods achieve a promising result not only on the seen test set but also in the unseen test set.

Keywords: GPT-2, product description, transformer, task-adaptive, language model, pretraining

Procedia PDF Downloads 197

Authors: Andrey Khalov

Abstract:

The rapid proliferation of unstructured data in IT infrastructure management demands innovative approaches for extracting actionable knowledge. This paper presents a framework for ontology-based knowledge extraction that combines relational graph neural networks (R-GNN) with large language models (LLMs). The proposed method leverages the DOLCE framework as the foundational ontology, extending it with concepts from ITSMO for domain-specific applications in IT service management and outsourcing. A key component of this research is the use of transformer-based models, such as DeBERTa-v3-large, for automatic entity and relationship extraction from unstructured texts. Furthermore, the paper explores how transfer learning techniques can be applied to fine-tune large language models (LLaMA) for using to generate synthetic datasets to improve precision in BERT-based entity recognition and ontology alignment. The resulting IT Ontology (ITO) serves as a comprehensive knowledge base that integrates domain-specific insights from ITIL processes, enabling more efficient decision-making. Experimental results demonstrate significant improvements in knowledge extraction and relationship mapping, offering a cutting-edge solution for enhancing cognitive computing in IT service environments.

Keywords: ontology expansion, synthetic dataset, transformer fine-tuning, concept extraction, DOLCE, BERT, taxonomy, LLM, NER

Procedia PDF Downloads 14

Authors: Eki Farlen, Akas

Abstract:

Condition assessment of oil-paper-insulated power transformers, particularly of water content, is becoming increasingly important for aged transformers. As insulation ages, it can produce water, which reduces its dielectric strength, accelerates the cellulose ageing process, and causes gas bubbles to form at high temperatures. This paper mainly assesses the life condition of oil-paper insulation system of Inter Bus Transformer (IBT) 30 MVA, 150/30 kV in PT PLN-Substation Jelok that has been operating for 41 years, since 1974. Valuable information about the condition of high voltage insulation may be obtained by measuring its dielectric response. This paper describes in detail the interpretation of Dielectric Response Analysis (DIRANA) measurements and the test result compared to other insulation tests to get deep information for diagnostic, such as Tan delta test, oil characteristic test and Dissolve Gas Analysis (DGA) test. This paper mainly discusses the parameter relationship between moisture content, water content, acidity, oil conductivity and dissipation factor. The result and analysis show that IBT 30 MVA Jelok phase U and W had just been ageing due to high acidity level (>0.2 mgKOH/g) which cause high moisture in cellulose/paper (%) are in wet category about 4.7% and 5% and water content in oil (ppm) about 3.13 ppm and 3.33 ppm at temperature 20°C. High acidity level can make oxidation process and produce water in paper and particle which can decrease the value of Interfacial Tension (IFT) below 22 mN/m (poor category) for both phase U and W. Even if paper insulation of transformer are in wet condition, dissipation factor and capacitance at the same frequency (50 Hz) from both measurement DIRANA test and Tangent delta test give the same result (almost), the results are 0.69% and 0.71% (<1%), it may be acceptable and should not be investigated. The DGA results show that TDCG are in level one (1) condition and there are no found a Key Gases, it means that transformers had no failure during operation like arching, partial discharge and thermal in oil or cellulose.

Keywords: diagnostic, inter-bus transformer, oil-paper insulation, moisture, dissipation factor

Procedia PDF Downloads 279

Authors: Seong-Jin Cho, Jin Ho Kim

Abstract:

Energy harvesting is the technology which gathers and converts external energies such as light, vibration and heat which are disposed into reusable electrical energy and uses such electrical energy. The vibration energy harvesting is very interesting technology because it produces very high density of energy and unaffected by the climate. Vibration energy can be harvested by the electrostatic, electromagnetic and piezoelectric systems. The electrostatic system has low energy conversion efficiency, and the piezoelectric system is expensive and needs the frequent maintenance because it is made of piezoelectric ceramic. On the other hand, the electromagnetic system has a long life time and high harvesting efficiency, and it is relatively cheap. The electromagnetic harvesting system includes the linear generator and the rotary-type generator. The rotary-type generators require the additional mechanical conversion device if it uses linear motion of vibration. But, the linear generator uses directly linear motion of vibration without a mechanical conversion device, and it has uncomplicated structure and light weight compared with the rotary-type generator. Therefore, the linear electromagnetic generator can be useful in using vibration energy harvesting. The pole transformer systems need electricity sensor system for sending voltage and power information to administrator. Therefore, the battery is essential, and its regular maintenance of replacement is required. In case of the transformer of high location in mountainous areas, the person can’t easily access it resulting in high maintenance cost. To overcome these problems, we designed and developed the linear electromagnetic generator which can replace battery in electricity sensor system for sending voltage and power information of the pole transformer. And, it uses vibration energy of frequency more than 50 Hz by the pole transformer. In order to analyze the electromagnetic characteristics of small linear electric generator, a commercial electromagnetic finite element analysis program "MAXWELL" was used. Then, through the actual production and experiment of linear generator, we confirmed output power of linear generator.

Keywords: energy harvesting, frequency, linear generator, experiment

Procedia PDF Downloads 259

Authors: Javier Delgado-Gonzaga, Wilfrido Rivera, David Juárez-Romero

Abstract:

Cooling and water purification are processes that have contributed to the economic and social development of the modern world. However, these processes require a significant amount of energy globally. Nowadays, absorption heat pumps have been studied with great interest since they are capable of producing cooling and/or purifying water from low-temperature energy sources such as industrial waste heat or renewable energy. In addition, absorption heat pumps require negligible amounts of electricity for their operation and generally use working fluids that do not represent a risk to the environment. The objective of this work is to evaluate a system that integrates an absorption heat transformer and an absorption cooling system to produce fresh water and cooling from a low-temperature heat source. Both cycles operate with the working pair LiBr-H2O. The integration is possible through the interaction of the LiBr-H2O solution streams between both cycles and also by recycling heat from the absorption heat transformer to the absorption cooling system. Mathematical models were developed to compare the performance of four different configurations. The results showed that the configuration in which the hottest streams of LiBr-H2O solution preheated the coldest streams in the economizers of both cycles was one that achieved the best performance. The interaction of the solution currents and the heat recycling analyzed in this work serves as a record of the possibilities of integration between absorption cycles for cogeneration.

Keywords: absorption heat transformer, absorption cooling system, water desalination, integrated system

Procedia PDF Downloads 78

Authors: Jin-Lung Guan, Jyh-Cherng Gu, Chun-Wei Huang, Hsin-Hung Chang

Abstract:

This investigation develops a revisable method for estimating the estimate value of equivalent 10 Hz voltage flicker (DV10) of a DC Electric Arc Furnace (EAF). This study also discusses three 161kV DC EAFs by field measurement, with those results indicating that the estimated DV10 value is significantly smaller than the survey value. The key point is that the conventional means of estimating DV10 is inappropriate. There is a main cause as the assumed Qmax is too small. Although DC EAF is regularly operated in a constant MVA mode, the reactive power variation in the Main Transformer (MT) is more significant than that in the Furnace Transformer (FT). A substantial difference exists between estimated maximum reactive power fluctuation (DQmax) and the survey value from actual DC EAF operations. However, this study proposes a revisable method that can obtain a more accurate DV10 estimate than the conventional method.

Keywords: voltage flicker, dc EAF, estimate value, DV10

Procedia PDF Downloads 448

Authors: Ahmed A. Al-Mudhafar, Ali A. Abduljabar, Hayder Jawad Albattat

Abstract:

This work suggests a new microwave resonator sensor (MRS) device for measuring the oil’s breakdown voltage of high voltage transformers. A precise high-sensitivity sensor is designed and manufactured based on a microstrip split ring resonator (SRR). To improve the sensor sensitivity, a RF amplifier of 30 dB gain is linked through a transmission line of 50Ω.The sensor operates at a microwave band (L) with a quality factor of 1.35x105 when it is loaded with an empty tube. In this work, the sensor has been tested with three samples of high voltage transformer oil of different ages (new, middle, and damaged) where the quality factor differs with each sample. A mathematical model was built to calculate the breakdown voltage of the transformer oils and the accuracy of the results was higher than 90%.

Keywords: active resonator sensor, oil breakdown voltage, transformers oils, quality factor

Procedia PDF Downloads 269

Authors: Virender Singh, Mathew Rees, Simon Hampton, Sivaram Annadurai

Abstract:

Plant identification is a challenging task that aims to identify the family, genus, and species according to plant morphological features. Automated deep learning-based computer vision algorithms are widely used for identifying plants and can help users narrow down the possibilities. However, numerous morphological similarities between and within species render correct classification difficult. In this paper, we tested custom convolution neural network (CNN) and vision transformer (ViT) based models using the PyTorch framework to classify plants. We used a large dataset of 88,000 provided by the Royal Horticultural Society (RHS) and a smaller dataset of 16,000 images from the PlantClef 2015 dataset for classifying plants at genus and species levels, respectively. Our results show that for classifying plants at the genus level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420 and other state-of-the-art CNN-based models suggested in previous studies on a similar dataset. ViT model achieved top accuracy of 83.3% for classifying plants at the genus level. For classifying plants at the species level, ViT models perform better compared to CNN-based models ResNet50 and ResNet-RS-420, with a top accuracy of 92.5%. We show that the correct set of augmentation techniques plays an important role in classification success. In conclusion, these results could help end users, professionals and the general public alike in identifying plants quicker and with improved accuracy.

Keywords: plant identification, CNN, image processing, vision transformer, classification

Procedia PDF Downloads 104

Authors: Rahul Ganpat Mapari, D. G. Wakde

Abstract:

In this paper, a proposed approach to improve the power factor of single-phase rectifiers and to regulate the output voltage against the change in grid voltage and load is presented. This converter topology is evaluated on the basis of performance and its salient features like simplicity, low cost and high performance are discussed to analyze its applicability. The proposed control strategy is bridgeless, transformer-less and output current sensor-less and consists of only two Bi-directional IGBTs and two diodes. The voltage regulation is achieved by a simple voltage divider to communicate to a controller to control the duty cycles of PWM. A control technique and operational procedure are also developed, both theoretically and experimentally. The experimental results clearly verify the theoretical analysis from the prototype connected to grid unity.

Keywords: Active Rectifier (AC-DC), power factor, single phase, voltage regulation

Procedia PDF Downloads 580

Authors: Jinan Fiaidhi, Hashmath Shaik

Abstract:

Breaking the barrier for practicing evidence-based medicine relies on effective methods for rapidly identifying relevant evidence from the body of biomedical literature. An important challenge confronted by medical practitioners is the long time needed to browse, filter, summarize and compile information from different medical resources. Deep learning can help in solving this based on automatic question answering (Q&A) and transformers. However, Q&A and transformer technologies are not trained to answer clinical queries that can be used for evidence-based practice, nor can they respond to structured clinical questioning protocols like PICO (Patient/Problem, Intervention, Comparison and Outcome). This article describes the use of deep learning techniques for Q&A that are based on transformer models like BERT and GPT to answer PICO clinical questions that can be used for evidence-based practice extracted from sound medical research resources like PubMed. We are reporting acceptable clinical answers that are supported by findings from PubMed. Our transformer methods are reaching an acceptable state-of-the-art performance based on two staged bootstrapping processes involving filtering relevant articles followed by identifying articles that support the requested outcome expressed by the PICO question. Moreover, we are also reporting experimentations to empower our bootstrapping techniques with patch attention to the most important keywords in the clinical case and the PICO questions. Our bootstrapped patched with attention is showing relevancy of the evidence collected based on entropy metrics.

Keywords: automatic question answering, PICO questions, evidence-based medicine, generative models, LLM transformers

Procedia PDF Downloads 44

Authors: Shweta Singh, Sudaman Katti

Abstract:

The most well-known sequence models make use of complex recurrent neural networks in an encoder-decoder configuration. The model used in this research makes use of a transformer, which is based purely on a self-attention mechanism, without relying on recurrence at all. More specifically, encoders and decoders which make use of self-attention and operate based on a memory, are used. In this research work, results for various 3D visual and non-visual reinforcement learning tasks designed in Unity software were obtained. Convolutional neural networks, more specifically, nature CNN architecture, are used for input processing in visual tasks, and comparison with standard long short-term memory (LSTM) architecture is performed for both visual tasks based on CNNs and non-visual tasks based on coordinate inputs. This research work combines the transformer architecture with the proximal policy optimization technique used popularly in reinforcement learning for stability and better policy updates while training, especially for continuous action spaces, which are used in this research work. Certain tasks in this paper are long horizon tasks that carry on for a longer duration and require extensive use of memory-based functionalities like storage of experiences and choosing appropriate actions based on recall. The transformer, which makes use of memory and self-attention mechanism in an encoder-decoder configuration proved to have better performance when compared to LSTM in terms of exploration and rewards achieved. Such memory based architectures can be used extensively in the field of cognitive robotics and reinforcement learning.

Keywords: convolutional neural networks, reinforcement learning, self-attention, transformers, unity

Procedia PDF Downloads 136

Authors: Saeed Khandan Siar, Stefan Tenbohlen, Christian Breuer, Raphael Lebreton

Abstract:

The goal of this article is to investigate the detailed temperature distribution and the fluid flow of an oil cooled winding of a power transformer by means of computational fluid dynamics (CFD). The experimental setup consists of three passes of a zig-zag cooled disc type winding, in which losses are modeled by heating cartridges in each winding segment. A precise temperature sensor measures the temperature of each turn. The laboratory setup allows the exact control of the boundary conditions, e.g. the oil flow rate and the inlet temperature. Furthermore, a simulation model is solved using the open source computational fluid dynamics solver OpenFOAM and validated with the experimental results. The model utilizes the laminar and turbulent flow for the different mass flow rate of the oil. The good agreement of the simulation results with experimental measurements validates the model.

Keywords: CFD, conjugated heat transfer, power transformers, temperature distribution

Procedia PDF Downloads 422

Authors: Helena M. Wilhelm, Paulo O. Fernandes, Laís P. Dill, Kethlyn G. Moscon

Abstract:

The use of vegetable oil (or natural ester) as an insulating fluid in electrical transformers is a trend that aims to contribute to environmental preservation since it is biodegradable and non-toxic. Besides, vegetable oil has high flash and combustion points, being considered a fire safety fluid. However, vegetable oil is usually less stable towards oxidation than mineral oil. Both insulating fluids, mineral and vegetable oils, need to be tested periodically according to specific standards. Oxidation stability can be determined by the induction period measured by conductivity method (Rancimat) by monitoring the effectivity of oil’s antioxidant additives, a methodology already developed for food application and biodiesel but still not standardized for insulating fluids. Besides adequate oxidation stability, fluids must be compatible with transformer's construction materials under normal operating conditions to ensure that damage to the oil and parts of the transformer does not occur. ASTM standard and Brazilian normative differ in parameters evaluated, which reveals the need to regulate tests for each oil type. The aim of this study was to assess oxidation stability and compatibility of vegetable oils to suggest the best way to assure a viable performance of vegetable oil as transformer insulating fluid. The determination of the induction period for several vegetable insulating oils from the local market by using Rancimat was carried out according to BS EN 14112 standard, at different temperatures (110, 120, and 130 °C). Also, the compatibility of vegetable oil was assessed according to ASTM and ABNT NBR standards. The main results showed that the best temperature for use in the Rancimat test is 130 °C, which allows a better observation of conductivity change. The compatibility test results presented differences between vegetable and mineral oil standards that should be taken into account in oil testing since materials compatibility and oxidation stability are essential for equipment reliability.

Keywords: compatibility, Rancimat, natural ester, vegetable oil

Procedia PDF Downloads 210

Authors: Murad Almadani, Khalil Abu-Hantash, Xinyu Wang, Herbert Jelinek, Kinda Khalaf

Abstract:

Gait analysis is used by healthcare professionals as a tool to gain a better understanding of the movement impairment and track progress. In most circumstances, monitoring patients in their real-life environments with low-cost equipment such as cameras and wearable sensors is more important. Inertial sensors, on the other hand, cannot provide enough information on angular dynamics. This research offers a method for tracking 2D joint coordinates using cutting-edge vision algorithms and a single RGB camera. We provide an end-to-end comprehensive deep learning pipeline for marker-less gait parameter estimation, which, to our knowledge, has never been done before. To make our pipeline function in real-time for real-world applications, we leverage the AlphaPose human posture prediction model and a deep learning transformer. We tested our approach on the well-known GPJATK dataset, which produces promising results.

Keywords: gait analysis, human pose estimation, deep learning, real time gait estimation, AlphaPose, transformer

Procedia PDF Downloads 118

Authors: Sean Paulsen, Michael Casey

Abstract:

In this work we present a self-supervised pretraining framework for transformers on functional Magnetic Resonance Imaging (fMRI) data. First, we pretrain our architecture on two self-supervised tasks simultaneously to teach the model a general understanding of the temporal and spatial dynamics of human auditory cortex during music listening. Our pretraining results are the first to suggest a synergistic effect of multitask training on fMRI data. Second, we finetune the pretrained models and train additional fresh models on a supervised fMRI classification task. We observe significantly improved accuracy on held-out runs with the finetuned models, which demonstrates the ability of our pretraining tasks to facilitate transfer learning. This work contributes to the growing body of literature on transformer architectures for pretraining and transfer learning with fMRI data, and serves as a proof of concept for our pretraining tasks and multitask pretraining on fMRI data.

Keywords: transfer learning, fMRI, self-supervised, brain decoding, transformer, multitask training

Procedia PDF Downloads 90

Authors: Yasunori Kobori, Li Quan, Shu Wu, Nizam Mohyar, Zachary Nosker, Nobukazu Tsukiji, Nobukazu Takai, Haruo Kobayashi

Abstract:

This paper proposes an electrolytic capacitor-less transformer-less AC-DC LED driver with a current ripple canceller. The proposed LED driver includes a diode bridge, a buck-boost converter, a negative feedback controller and a current ripple cancellation circuit. The current ripple canceller works as a bi-directional current converter using a sub-inductor, a sub-capacitor and two switches for controlling current flow. LED voltage is controlled in order to regulate LED current by the negative feedback controller using a current sense resistor. There are two capacitors which capacitance of 5 uF. We describe circuit topologies, operation principles and simulation results for our proposed circuit. In addition, we show the line regulation for input voltage variation from 85V to 130V. The output voltage ripple is 2V and the LED current ripple is 65 mA which is less than 20% of the typical current of 350 mA. We are now making the proposed circuit on a universal board in order to measure the experimental characteristics.

Keywords: LED driver, electrolytic, capacitor-less, AC-DC converter, buck-boost converter, current ripple canceller

Procedia PDF Downloads 473

Authors: Antonis Constantinides, Christos Yiallouras, Christakis Damianou

Abstract:

The coaxial transformer-coupled push-pull circuitry has been used widely in HF and VHF amplifiers for many decades without significant changes in the topology of the transformers. Basic changes over the years concerned the construction and turns ratio of the transformers as has been imposed upon the newer technologies active devices demands. The balun transmission line transformers applied in push-pull amplifiers enable input/output impedance transformation, but are mainly used to convert the balanced output into unbalanced and the input unbalanced into balanced. A simple and affordable alternative solution over the traditional coaxial transformer is the coreless planar balun. A key advantage over the traditional approach lies in the high specifications repeatability; simplifying the amplifier construction requirements as the planar balun constitutes an integrated part of the PCB copper layout. This paper presents the performance analysis of a planar LDMOS MRFE6VP5600 Push-Pull amplifier that enables robust operation in Band III, DVB-T, DVB-T2 standards but functions equally well in Band II, for DRM+ new generation transmitters.

Keywords: amplifier, balun, complex impedance, LDMOS, planar-transformers

Procedia PDF Downloads 440

Authors: Arvind Dhingra, Tejinder Singh Saggu

Abstract:

As the energy demands rise and the pollution levels grow, it becomes imperative for us to save energy in all the fields in which it is used. The industrial sector is the major commercial energy consuming sector in India, where electrical energy is the most common and widely used type of energy. As the demand and price of energy are increasing day by day, therefore, the subject of energy conservation is a concern for most energy users particularly industry. Judicious use of energy becomes imperative for third world developing country being presence of energy crisis. This paper provides some measure for energy saving that can be commonly recommended for a rolling unit of steel industry. A case of hot rolling unit in JSL Stainless Ltd., Hisar for energy conservation is given. Overall improvement in energy consumption in light of the stated recommendation is illustrated along with the proposed utilization of the techniques and their applications. Energy conservation in conventional motor with replacement or use of star delta star converter, reduction in cable losses, replacement of filament of LED lamps, replacement of conventional transformer with cast resin dry type transformer and provision of energy management system for energy conservation and per unit production cost reduction are elaborated in this paper.

Keywords: energy audit, energy conservation, energy efficient motors

Procedia PDF Downloads 531

Authors: Atuhurra Jesse, N'guessan Yves-Roland Douha, Pabitra Lenka

Abstract:

The detection of diseases within plants has attracted a lot of attention from computer vision enthusiasts. Despite the progress made to detect diseases in many plants, there remains a research gap to train image classifiers to detect the cacao swollen shoot virus disease or CSSVD for short, pertinent to cacao plants. This gap has mainly been due to the unavailability of high quality labeled training data. Moreover, institutions have been hesitant to share their data related to CSSVD. To fill these gaps, image classifiers to detect CSSVD-infected cacao plants are presented in this study. The classifiers are based on VGG16, ResNet50 and Vision Transformer (ViT). The image classifiers are evaluated on a recently released and publicly accessible KaraAgroAI Cocoa dataset. The best performing image classifier, based on ResNet50, achieves 95.39\% precision, 93.75\% recall, 94.34\% F1-score and 94\% accuracy on only 20 epochs. There is a +9.75\% improvement in recall when compared to previous works. These results indicate that the image classifiers learn to identify cacao plants infected with CSSVD.

Keywords: CSSVD, image classification, ResNet50, vision transformer, KaraAgroAI cocoa dataset

Procedia PDF Downloads 103

Authors: Tingwei Shu, Dong Zhou, Chengjun Guo

Abstract:

Semantic communication is an emerging form of communication that realize intelligent communication by extracting semantic information of data at the source and transmitting it, and recovering the data at the receiving end. It can effectively solve the problem of data transmission under the situation of large data volume, low SNR and restricted bandwidth. With the development of Deep Learning, semantic communication further matures and is gradually applied in the fields of the Internet of Things, Uumanned Air Vehicle cluster communication, remote sensing scenarios, etc. We propose an improved semantic communication system for the situation where the data volume is huge and the spectrum resources are limited during the transmission of remote sensing images. At the transmitting, we need to extract the semantic information of remote sensing images, but there are some problems. The traditional semantic communication system based on Convolutional Neural Network cannot take into account the global semantic information and local semantic information of the image, which results in less-than-ideal image recovery at the receiving end. Therefore, we adopt the improved vision-Transformer-based structure as the semantic encoder instead of the mainstream one using CNN to extract the image semantic features. In this paper, we first perform pre-processing operations on remote sensing images to improve the resolution of the images in order to obtain images with more semantic information. We use wavelet transform to decompose the image into high-frequency and low-frequency components, perform bilinear interpolation on the high-frequency components and bicubic interpolation on the low-frequency components, and finally perform wavelet inverse transform to obtain the preprocessed image. We adopt the improved Vision-Transformer structure as the semantic coder to extract and transmit the semantic information of remote sensing images. The Vision-Transformer structure can better train the huge data volume and extract better image semantic features, and adopt the multi-layer self-attention mechanism to better capture the correlation between semantic features and reduce redundant features. Secondly, to improve the coding efficiency, we reduce the quadratic complexity of the self-attentive mechanism itself to linear so as to improve the image data processing speed of the model. We conducted experimental simulations on the RSOD dataset and compared the designed system with a semantic communication system based on CNN and image coding methods such as BGP and JPEG to verify that the method can effectively alleviate the problem of excessive data volume and improve the performance of image data communication.

Keywords: semantic communication, transformer, wavelet transform, data processing

Procedia PDF Downloads 78

Authors: Ling Qi, Matloob Khushi, Josiah Poon

Abstract:

This paper introduces a pre-trained Masked Candlestick Model (MCM) for trading time-series data. The pre-trained model is based on three core designs. First, we convert trading price data at each data point as a set of normalized elements and produce embeddings of each element. Second, we generate a masked sequence of such embedded elements as inputs for self-supervised learning. Third, we use the encoder mechanism from the transformer to train the inputs. The masked model learns the contextual relations among the sequence of embedded elements, which can aid downstream classification tasks. To evaluate the performance of the pre-trained model, we fine-tune MCM for three different downstream classification tasks to predict future price trends. The fine-tuned models achieved better accuracy rates for all three tasks than the baseline models. To better analyze the effectiveness of MCM, we test the same architecture for three currency pairs, namely EUR/GBP, AUD/USD, and EUR/JPY. The experimentation results demonstrate MCM’s effectiveness on all three currency pairs and indicate the MCM’s capability for signal extraction from trading data.

Keywords: masked language model, transformer, time series prediction, trading prediction, embedding, transfer learning, self-supervised learning

Procedia PDF Downloads 128

Authors: D. Subedi, S. Pradhan

Abstract:

Current transformers are an integral part of power system because it provides a proportional safe amount of current for protection and measurement applications. However when the power system experiences an abnormal situation leading to huge current flow, then this huge current is proportionally injected to the protection and metering circuit. Since the protection and metering equipment’s are designed to withstand only certain amount of current with respect to time, these high currents pose a risk to man and equipment. Therefore during such instances, the CT saturation characteristics have a huge influence on the safety of both man and equipment and also on the reliability of the protection and metering system. This paper shows the effect of burden on the Accuracy Limiting factor/ Instrument security factor of current transformers and also the change in saturation characteristics of the CT’s. The response of the CT to varying levels of overcurrent at different connected burden will be captured using the data acquisition software LabVIEW. Analysis is done on the real time data gathered using LabVIEW. Variation of current transformer saturation characteristics with changes in burden will be discussed.

Keywords: accuracy limiting factor, burden, current transformer, instrument security factor, saturation characteristics

Procedia PDF Downloads 415

Authors: Awais Ahmed, Javed Iqbal

Abstract:

Pakistan has been for many years facing extreme challenges in energy deficit due to the shortage of power generation compared to increasing demand. A part of this energy deficit is also contributed by the power lost in transmission and distribution network. Unfortunately, distribution companies are not equipped with modern technologies and methods to identify and eliminate these losses. According to estimate, total energy lost in early 2000 was between 20 to 26 percent. To address this issue the present research study was designed with the objectives of developing a standalone GIS application for distribution companies having the capability of loss calculation as well as identification of overloaded transformers. For this purpose, Hilal Road feeder in Faisalabad Electric Supply Company (FESCO) was selected as study area. An extensive GPS survey was conducted to identify each consumer, linking it to the secondary pole of the transformer, geo-referencing equipment and documenting conductor sizes. To identify overloaded transformer, accumulative kWH reading of consumer on transformer was compared with threshold kWH. Technical losses of 11kV and 220V lines were calculated using the data from substation and resistance of the network calculated from the geo-database. To automate the process a standalone GIS application was developed using ArcObjects with engineering analysis capabilities. The application uses GIS database developed for 11kV and 220V lines to display and query spatial data and present results in the form of graphs. The result shows that about 14% of the technical loss on both high tension (HT) and low tension (LT) network while about 4 out of 15 general duty transformers were found overloaded. The study shows that GIS can be a very effective tool for distribution companies in management and planning of their distribution network.

Keywords: geographical information system, GIS, power distribution, distribution transformers, technical losses, GPS, SDSS, spatial decision support system

Procedia PDF Downloads 376

Authors: Paffrath Meinhard, Zhou Yayun, Guo Yiqing, Ertl Harald

Abstract:

Winding geometry design is an important part of power transformer electrical design. Conventionally, the winding geometry is designed manually, which is a time-consuming job because it involves many iteration steps in order to meet all cost, manufacturing and electrical requirements. Here a method is presented which automatically generates the winding geometry for given user parameters and allows the user to interactively set and change parameters. To achieve this goal, the winding problem is transferred to a mixed integer nonlinear optimization problem. The relevant geometrical design parameters are defined as optimization variables. The cost and other requirements are modeled as constraints. For the solution, a stochastic ant colony optimization algorithm is applied. It is well-known, that an optimizer can get stuck in a local minimum. For the winding problem, we present efficient strategies to come out of local minima, furthermore a reduced variable search range helps to accelerate the solution process. Numerical examples show that the optimization result is delivered within seconds such that the user can interactively change the variable search area and constraints to improve the design.

Keywords: ant colony optimization, mixed integer nonlinear programming, power transformer, winding design

Procedia PDF Downloads 380

Authors: Shamm Kidane, Ibrahim Abdella, Fitsum Gaim, Simon Mulugeta, Sirak Asmerom, Natnael Ambasager, Yoel Ghebrihiwot

Abstract:

The call for natural language processing (NLP) systems for low-resource languages has become more apparent than ever in the past few years, with the arduous challenges still present in preparing such systems. This paper presents an improved dataset version of the Nagaoka Tigrinya Corpus for Parts-of-Speech (POS) classification system in the Tigrinya language. The size of the initial Nagaoka dataset was incremented, totaling the new tagged corpus to 118K tokens, which comprised the 12 basic POS annotations used previously. The additional content was also annotated manually in a stringent manner, followed similar rules to the former dataset and was formatted in CONLL format. The system made use of the novel approach in NLP tasks and use of the monolingually pre-trained TiELECTRA, TiBERT and TiRoBERTa transformer models. The highest achieved score is an impressive weighted F1-score of 94.2%, which surpassed the previous systems by a significant measure. The system will prove useful in the progress of NLP-related tasks for Tigrinya and similarly related low-resource languages with room for cross-referencing higher-resource languages.

Keywords: Tigrinya POS corpus, TiBERT, TiRoBERTa, conditional random fields

Procedia PDF Downloads 103

Authors: S. Oladoyinbo, A. A. Tijani

Abstract:

Multilevel inverters (MLI) are used at high power application based on their operation. There are 3 main types of multilevel inverters (MLI); diode clamped, flying capacitor and cascaded MLI. A cascaded MLI requires the least number of components to achieve same number of voltage levels when compared to other types of MLI while the flying capacitor has the minimum harmonic distortion. However, maximizing the advantage of cascaded H-bridge MLI and flying capacitor MLI, an improved MLI can be achieved with fewer components and better performance. In this paper an improved MLI is presented by asymmetrically integrating a flying capacitor to a cascaded H-bridge MLI also integrating an auxiliary transformer to the main transformer to decrease the total harmonics distortion (THD) with increased number of output voltage levels. Furthermore, the system is incorporated with a hybrid time and climate based solar panel cleaner and power selector which intelligently manage the input of the MLI and clean the solar panel weekly ensuring the environmental factor effect on the panel is reduced to minimum.

Keywords: multilevel inverter, total harmonics distortion, cascaded h-bridge inverter, flying capacitor

Procedia PDF Downloads 366

Authors: Xinyuan Chen, Mohd Nizam Husen, Zhongmei Zhou, Gongde Guo, Wei Gao

Abstract:

Temporal Knowledge Graph Completion (TKGC) has attracted increasing attention for its enormous value; however, existing models lack capabilities to capture both local interactions and global dependencies simultaneously with evolutionary dynamics, while the latest achievements in convolutions and Transformers haven't been employed in this area. What’s more, periodic patterns in TKGs haven’t been fully explored either. To this end, a multi-stage hybrid architecture with convolution-backed Transformers is introduced in TKGC tasks for the first time combining the Hawkes process to model evolving event sequences in a continuous-time domain. In addition, the seasonal-trend decomposition is adopted to identify periodic patterns. Experiments on six public datasets are conducted to verify model effectiveness against state-of-the-art (SOTA) methods. An extensive ablation study is carried out accordingly to evaluate architecture variants as well as the contributions of independent components in addition, paving the way for further potential exploitation. Besides complexity analysis, input sensitivity and safety challenges are also thoroughly discussed for comprehensiveness with novel methods.

Keywords: temporal knowledge graph completion, convolution, transformer, Hawkes process, periodicity

Procedia PDF Downloads 78

Authors: Ma. Gracia Corazon Cayanan, Kai Yuen Cheong, Li Sha

Abstract:

Training a language model for a minority language has been a challenging task. The lack of available corpora to train and fine-tune state-of-the-art language models is still a challenge in the area of Natural Language Processing (NLP). Moreover, the need for high computational resources and bulk data limit the attainment of this task. In this paper, we presented the following contributions: (1) we introduce and used a translation pair set of Tagalog and English (TL-EN) in pre-training a language model to a minority language resource; (2) we fine-tuned and evaluated top-ranking and pre-trained semantic textual similarity binary task (STSB) models, to both TL-EN and STS dataset pairs. (3) then, we reduced the size of the model to offset the need for high computational resources. Based on our results, the models that were pre-trained to translation pairs and STS pairs can perform well for STSB task. Also, having it reduced to a smaller dimension has no negative effect on the performance but rather has a notable increase on the similarity scores. Moreover, models that were pre-trained to a similar dataset have a tremendous effect on the model’s performance scores.

Keywords: semantic matching, semantic textual similarity binary task, low resource minority language, fine-tuning, dimension reduction, transformer models

Procedia PDF Downloads 211

Authors: Babak Forouraghi

Abstract:

A genetic circuit is a collection of interacting genes and proteins that enable individual cells to implement and perform vital biological functions such as cell division, growth, death, and signaling. In cell engineering, synthetic gene circuits are engineered networks of genes specifically designed to implement functionalities that are not evolved by nature. These engineered networks enable scientists to tackle complex problems such as engineering cells to produce therapeutics within the patient's body, altering T cells to target cancer-related antigens for treatment, improving antibody production using engineered cells, tissue engineering, and production of genetically modified plants and livestock. Construction of computational models to realize genetic circuits is an especially challenging task since it requires the discovery of flow of genetic information in complex biological systems. Building synthetic biological models is also a time-consuming process with relatively low prediction accuracy for highly complex genetic circuits. The primary goal of this study was to investigate the utility of a pre-trained bidirectional encoder transformer that can accurately predict gene expressions in genetic circuit designs. The main reason behind using transformers is their innate ability (attention mechanism) to take account of the semantic context present in long DNA chains that are heavily dependent on spatial representation of their constituent genes. Previous approaches to gene circuit design, such as CNN and RNN architectures, are unable to capture semantic dependencies in long contexts as required in most real-world applications of synthetic biology. For instance, RNN models (LSTM, GRU), although able to learn long-term dependencies, greatly suffer from vanishing gradient and low-efficiency problem when they sequentially process past states and compresses contextual information into a bottleneck with long input sequences. In other words, these architectures are not equipped with the necessary attention mechanisms to follow a long chain of genes with thousands of tokens. To address the above-mentioned limitations of previous approaches, a transformer model was built in this work as a variation to the existing DNA Bidirectional Encoder Representations from Transformers (DNABERT) model. It is shown that the proposed transformer is capable of capturing contextual information from long input sequences with attention mechanism. In a previous work on genetic circuit design, the traditional approaches to classification and regression, such as Random Forrest, Support Vector Machine, and Artificial Neural Networks, were able to achieve reasonably high R2 accuracy levels of 0.95 to 0.97. However, the transformer model utilized in this work with its attention-based mechanism, was able to achieve a perfect accuracy level of 100%. Further, it is demonstrated that the efficiency of the transformer-based gene expression classifier is not dependent on presence of large amounts of training examples, which may be difficult to compile in many real-world gene circuit designs.

Keywords: transformers, generative ai, gene expression design, classification

Procedia PDF Downloads 59