Search results for: data backup

Authors: Xinyan Zhang, Zhipeng Tan, Shan Fan

Abstract:

The first layer of defense against data loss is the backup data. This paper implements an agent-based network backup system used the backup, server-storage and server-backup agent these tripartite construction, and we realize the snapshot and hierarchical index in the NSBS. It realizes the control command and data flow separation, balances the system load, thereby improving the efficiency of the system backup and recovery. The test results show the agent-based network backup system can effectively improve the task-based concurrency, reasonably allocate network bandwidth, the system backup performance loss costs smaller and improves data recovery efficiency by 20%.

Keywords: agent, network backup system, three architecture model, NSBS

Procedia PDF Downloads 428

Authors: Abdullah Alsaeed

Abstract:

Virtualized computing and cloud computing infrastructures are no longer fuzz or marketing term. They are a core reality in today’s corporate Information Technology (IT) organizations. Hence, developing an effective and efficient methodologies for data backup and data recovery is required more than any time. The purpose of data backup and recovery techniques are to assist the organizations to strategize the business continuity and disaster recovery approaches. In order to accomplish this strategic objective, a variety of mechanism were proposed in the recent years. This research paper will explore and examine the latest techniques and solutions to provide data backup and restoration for the cloud computing platforms.

Keywords: data backup, data recovery, cloud computing, business continuity, disaster recovery, cost-effective, data encryption.

Procedia PDF Downloads 53

Authors: Nihed Bahria El Asghar, Imen Jouili, Mounir Frikha

Abstract:

Inter-datacenter transport network is very bandwidth and delay demanding. The data transferred over such a network is also highly QoS-exigent mostly because a huge volume of data should be transported transparently with regard to the application user. To avoid the data transfer failure, a backup path should be reserved. No re-routing delay should be observed. A dedicated 1+1 protection is however not applicable in inter-datacenter transport network because of the huge spare capacity. In this context, we propose a survivable virtual network with minimal backup based on network coding (1 ⊕ 1) and solve it using a modified Dijkstra-based heuristic.

Keywords: network coding, dedicated protection, spare capacity, inter-datacenters transport network

Procedia PDF Downloads 418

Authors: Ahmad Shukri Mohd Noor, Emma Ahmad Sirajudin, Rabiei Mamat

Abstract:

Pertaining to a particular Marine interest, the process of data sampling could take years before a study can be concluded. Therefore, the need for a robust backup system for the data is invariably implicit. In recent advancement of Marine applications, more functionalities and tools are integrated to assist the work of the researchers. It is anticipated that this modality will continue as research scope widens and intensifies and at the same to follow suit with current technologies and lifestyles. The convenience to collect and share information these days also applies to the work in Marine research. Therefore, Marine system designers should be aware that high availability is a necessary attribute in Marine repository applications as well as a robust backup system for the data. In this paper, the approach to high availability is related both to hardware and software but the focus is more on software. We consider a NABTIC repository system that is primitively built on a single server and does not have replicated components. First, the system is decomposed into separate modules. The modules are placed on multiple servers to create a distributed system. Redundancy is added by placing the copies of the modules on different servers using Neighbor Caring Environment System(NCES) technique. NCER is utilizing parallel replication components mechanism. A background monitoring is established to check servers’ heartbeats to confirm their aliveness. At the same time, a critical adaptive threshold is maintained to make sure a failure is timely detected using Adaptive Fault Detection (AFD). A confirmed failure will set the recovery mode where a selection process will be done before a fail-over server is instructed. In effect, the Marine repository service is continued as the fail-over masks a recent failure. The performance of the new prototype is tested and is confirmed to be more highly available. Furthermore, the downtime is not noticeable as service is immediately restored automatically. The Marine repository system is said to have achieved fault tolerance.

Keywords: availability, fault detection, replication, fault tolerance, marine application

Procedia PDF Downloads 285

Authors: Frie Ayalew, Seada Hussen

Abstract:

Ethio-telecom holds a variety of telecom devices that needs a consistent power source to be operational. The company got this power mainly from the national grid and used this power source alone or with a generator and/or batteries as a backup. In addition, for off-grid or remote areas, the company commonly uses generators and batteries. But unstable diesel prices, huge expenses of fuel and transportation, and high carbon emissions are the main problems associated with fuel energy. So, the design of solar power with battery backup is a highly recommended and advantageous source for the next coming years. This project designs the AC and DC standalone photovoltaic supply to Ethio-telecom access layer devices for the case of multi-service access gateway in Adama. The design is done by using Homer software for both AC and DC loads. The project shows that the design of a solar based microgrid is the best option for the designed area.

Keywords: solar power, battery, inverter, Ethio-telecom, solar radiation

Procedia PDF Downloads 44

Authors: Yusuf Ziya Ayık, Ferhat Kahveci

Abstract:

Currently, database management systems have various tools such as backup and maintenance, and also provide statistical information such as resource usage and security. In terms of query performance, this paper covers query optimization, views, indexed tables, pre-computation materialized view, query performance analysis in which query plan alternatives can be created and the least costly one selected to optimize a query. Indexes and views can be created for related table columns. The literature review of this study showed that, in the course of time, despite the growing capabilities of the database management system, only database administrators are aware of the need for dealing with archival and transactional data types differently. These data may be constantly changing data used in everyday life, and also may be from the completed questionnaire whose data input was completed. For both types of data, the database uses its capabilities; but as shown in the findings section, instead of repeating similar heavy calculations which are carrying out same results with the same query over a survey results, using materialized view results can be in a more simple way. In this study, this performance difference was observed quantitatively considering the cost of the query.

Keywords: cost of query, database management systems, materialized view, query performance

Procedia PDF Downloads 247

Authors: Masoud Taghavi, Gholamreza Salehi, Ali Lohrasbi Nichkoohi

Abstract:

Despite the growing use of renewable energies in different fields of application of this technology in the field of water supply has been less attention. Photovoltaic and wind hybrid system is that new topics in renewable energy, including photovoltaic arrays, wind turbines, a set of batteries as a storage system and a diesel generator as a backup system is. In this investigation, first climate data including average wind speed and solar radiation at any time during the year, data collection and analysis are performed in the energy. The wind turbines in four models, photovoltaic panels at the 6 position of relative power, batteries and diesel generator capacity in seven states in the two models are combined hours of operation with renewables, diesel generator and battery bank check and a hybrid system of solar power generation-wind, which is optimized conditions, are presented.

Keywords: renewable energy, wind and solar energy, hybrid systems, cloning station

Procedia PDF Downloads 362

Authors: Zhou Luchen, Wu Yubing, Burra Venkata Durga Kumar

Abstract:

As the scale of the network becomes larger and more complex than before, the density of user devices is also increasing. The development of Unmanned Aerial Vehicle (UAV) networks is able to collect and transform data in an efficient way by using software-defined networks (SDN) technology. This paper proposed a three-layer distributed and dynamic cluster architecture to manage UAVs by using an AI-based resource allocation calculation algorithm to address the overloading network problem. Through separating services of each UAV, the UAV hierarchical cluster system performs the main function of reducing the network load and transferring user requests, with three sub-tasks including data collection, communication channel organization, and data relaying. In this cluster, a head node and a vice head node UAV are selected considering the Central Processing Unit (CPU), operational (RAM), and permanent (ROM) memory of devices, battery charge, and capacity. The vice head node acts as a backup that stores all the data in the head node. The k-means clustering algorithm is used in order to detect high load regions and form the UAV layered clusters. The whole process of detecting high load areas, forming and selecting UAV clusters, and moving the selected UAV cluster to that area is proposed as offloading traffic algorithm.

Keywords: k-means, resource allocation, SDN, UAV network, unmanned aerial vehicles

Procedia PDF Downloads 74

Authors: Sapan Bhatnagar

Abstract:

Micron size particles emitted from different sources and produced by combustion have serious negative effects on human health and environment. They can penetrate deep into our lungs through the respiratory system. Determination of the amount of particulates present in the atmosphere per cubic meter is necessary to monitor, regulate and model atmospheric particulate levels. Cascade impactor is used to collect the atmospheric particulates and by gravimetric analysis, their concentration in the atmosphere of different size ranges can be determined. Cascade impactors have been used for the classification of particles by aerodynamic size. They operate on the principle of inertial impaction. It consists of a number of stages each having an impaction plate and a nozzle. Collection plates are connected in series with smaller and smaller cutoff diameter. Air stream passes through the nozzle and the plates. Particles in the stream having large enough inertia impact upon the plate and smaller particles pass onto the next stage. By designing each successive stage with higher air stream velocity in the nozzle, smaller diameter particles will be collected at each stage. Particles too small to be impacted on the last collection plate will be collected on a backup filter. Impactor consists of 4 stages each made of steel, having its cut-off diameters less than 10 microns. Each stage is having collection plates, soaked with oil to prevent bounce and allows the impactor to function at high mass concentrations. Even after the plate is coated with particles, the incoming particle will still have a wet surface which significantly reduces particle bounce. The particles that are too small to be impacted on the last collection plate are then collected on a backup filter (microglass fiber filter), fibers provide larger surface area to which particles may adhere and voids in filter media aid in reducing particle re-entrainment.

Keywords: aerodynamic diameter, cascade, environment, particulates, re-entrainment

Procedia PDF Downloads 289

Authors: S. Hameed, S. Anwer, M. Saad, M. Saady

Abstract:

For autonomic recovery with server virtualization, a cogent plan that includes recovery techniques and backups with virtualized servers can be developed instead of assigning an idle server to backup operations. In addition to hardware cost reduction and data center trail, the disaster recovery plan can ensure system uptime and to meet objectives of high availability, recovery time, recovery point, server provisioning, and quality of services. This autonomic solution would also support disaster management, testing, and development of the recovery site. In this research, a workflow plan is proposed for supporting disaster recovery with virtualization providing virtual monitoring, requirements engineering, solution decision making, quality testing, and disaster management. This recovery model would make disaster recovery a lot easier, faster, and less error prone.

Keywords: autonomous intelligence, disaster recovery, cloud computing, server virtualization

Procedia PDF Downloads 130

Authors: Elmahdi Mohammadine, Ahouzi Esmail, Najid Abdellah

Abstract:

This contribution aims to present a new protected hybrid WDM/TDM PON architecture using Wavelength Selective Switches and Optical Line Protection devices. The objective from using these technologies is to improve flexibility and enhance the protection of GPON networks.

Keywords: Wavlenght Division Multiplexed Passive Optical Network (WDM-PON), Time Division Multiplexed PON (TDM-PON), architecture, Protection, Wavelength Selective Switches (WSS), Optical Line Protection (OLP)

Procedia PDF Downloads 507

Authors: Suleiman Musa, Shuaibu Sidi Safiyanu

Abstract:

This study examined the digital preservation in Nigeria university libraries. A comparison between the university of Nigeria Nsukka (UNN) and Ahmadu Bello University Zaria (ABU, Zaria). The study utilized primary source of data obtained from two selected institution librarians. Finding revealed varying results in terms of skills acquired by librarians before and after digitization of the two institutions. The study reports that journals publication, text book, CD-ROMS, conference papers and proceedings, theses, dissertations and seminar papers are among the information resources available for digitization. The study further documents that copyright issue, power failure, and unavailability of needed materials are among the challenges facing the digitization of library of the institution. On the basis of the finding, the study concluded that digitization of library enhances efficiency in organization and retrieval of information services. The study therefore recommended that software should be upgraded with backup, training of the librarians on digital process, installation of antivirus and enhancement of technical collaboration between the library and MIS.

Keywords: digitalization, preservation, libraries, comparison

Procedia PDF Downloads 304

Authors: C. Matasane, C. Dwarika, R. Naidoo

Abstract:

The mathematical analysis on radiation obtained and the development of the solar photovoltaic (PV) array groundwater pumping is needed in the rural areas of Thohoyandou, Limpopo Province for sizing and power performance subject to the climate conditions within the area. A simple methodology approach is developed for the directed coupled solar, controller and submersible ground water pump system. The system consists of a PV array, pump controller and submerged pump, battery backup and charger controller. For this reason, the theoretical solar radiation obtained for optimal predictions and system performance in order to achieve different design and operating parameters. Here the examination of the PV schematic module in a Direct Current (DC) application is used for obtainable maximum solar power energy for water pumping. In this paper, a simple efficient photovoltaic water pumping system is presented with its theoretical studies and mathematical modeling of photovoltaics (PV) system.

Keywords: renewable energy sources, solar groundwater pumping, theoretical and mathematical analysis of photovoltaic (PV) system, theoretical solar radiation

Procedia PDF Downloads 342

Authors: Saeed Ullah Jan

Abstract:

This study has examined the e-readiness in libraries of public sector universities in Khyber Pakhtunkhwa. Efforts were made to evaluate the availability of human resources, electronic infrastructure, and network services and programs in the public sector university libraries. The population of the study was the twenty-seven public sector university libraries of Khyber Pakhtunkhwa. A quantitative approach was adopted, and a questionnaire-based survey was conducted to collect data from the librarian/in charge of public sector university libraries. The collected data were analyzed using Statistical Package for Social Sciences version 22 (SPSS). The mean score of the knowledge component interpreted magnitudes below three which indicates that the respondents are poorly or moderately satisfied regards knowledge of libraries. The satisfaction level of the respondents about the other components, such as electronic infrastructure, network services and programs, and enhancers of the networked world, was rated as average or below. The study suggested that major aspects of existing public-sector university libraries require significant transformation. For this purpose, the government should provide all the required resources and facilities to meet the population's informational and recreational demands. The Information Communication Technology (ICT) infrastructure of public university libraries needs improvement in terms of the availability of computer equipment, databases, network servers, multimedia projectors, digital cameras, uninterruptible power supply, scanners, and backup devices such as hard discs and Digital Video Disc/Compact Disc.

Keywords: ICT-libraries, e-readiness-libraries, e-readiness-university libraries, e-readiness-Pakistan

Procedia PDF Downloads 55

Authors: Sean Sloan

Abstract:

Probability Risk Assessments (PRA) can be a difficult concept for students to grasp. So in searching for different ways to describe PRA to relate it to their lives; COVID-19 came up. The parallels are amazing. Soon students began analyzing acceptable risk with the virus. This helped them to quantify just how dangerous is dangerous. The original lesson was dismissed and for the remainder of the period, the probability of risk, and the lethality of risk became the topic. Spreading events such as a COVID carrier on an airline became analogous to single fault casualties such as a Tsunami. Odds of spreading became odds of backup-diesel-generator failure – like with Fukashima Daiichi. Fatalities of the disease became expected fatalities due to radiation spread. Quantification from this discussion took it from hyperbole and emotion into one where we could rationally base guidelines. It has been one of the most effective educational devices observed.

Keywords: COVID, education, probability, risk

Procedia PDF Downloads 128

Authors: Farzane Sadatnia, Bahram Fathi

Abstract:

Today, all public and private organizations have found that the management of the world for key information related to the activities of a staff and its main essence and philosophy, though they constitute the management information systems are very helpful in this respect the right to apply systems can save a lot in terms of economic organizations including reducing the time decision - making, improve the quality of decision making, and cost savings to bring information systems is a backup system that can never be instead of logic and human reasoning, which can be used in the series is spreading, providing resources, and provide the necessary facilities, provide better services for users, balanced budget allocation, determine strengths and weaknesses and previous plans to review the current decisions and especially the decision . Hence; in this study attempts to the effect of an information system on a review of the organization.

Keywords: information system, planning, organization, coordination, control

Procedia PDF Downloads 442

Authors: Tareq Oshan

Abstract:

Facility location and size decisions are important to any company because they affect profitability and success. However, warehouses are exposed to various risks of failure that affect their activity. This paper presents a mixed-integer non-linear mathematical model that can be used to determine optimal warehouse locations and sizes, which warehouses to fortify, and which branches should be assigned to specific warehouses when there is a risk of warehouse failure. Every branch is assigned to a fortified primary warehouse or a nonfortified primary warehouse and a fortified backup warehouse. The standard method and an introduced method, based on the average probabilities, for linearizing this mathematical model were used. A Canadian case study was used to demonstrate the developed mathematical model, followed by some sensitivity analysis.

Keywords: supply chain network design, fortified warehouse, mixed-integer mathematical model, warehouse failure risk

Procedia PDF Downloads 207

Authors: V. Ziegler, F. Schneider, M. Pesch

Abstract:

With the EDM264, GRIMM offers a solution for mobile short- and long-term measurements in outdoor areas and at production sites. For research as well as permanent areal observations on a near reference quality base. The model EDM264 features a powerful and robust measuring cell based on optical particle counting (OPC) principle with all the advantages that users of GRIMM's portable aerosol spectrometers are used to. The system is embedded in a compact weather-protection housing with all-weather sampling, heated inlet system, data logger, and meteorological sensor. With TSP, PM10, PM4, PM2.5, PM1, and PMcoarse, the EDM264 provides all fine dust fractions real-time, valid for outdoor applications and calculated with the proven GRIMM enviro-algorithm, as well as six additional dust mass fractions pm10, pm2.5, pm1, inhalable, thoracic and respirable for IAQ and workplace measurements. This highly versatile instrument performs real-time monitoring of particle number, particle size and provides information on particle surface distribution as well as dust mass distribution. GRIMM's EDM264 has 31 equidistant size channels, which are PSL traceable. A high-end data logger enables data acquisition and wireless communication via LTE, WLAN, or wired via Ethernet. Backup copies of the measurement data are stored in the device directly. The rinsing air function, which protects the laser and detector in the optical cell, further increases the reliability and long term stability of the EDM264 under different environmental and climatic conditions. The entire sample volume flow of 1.2 L/min is analyzed by 100% in the optical cell, which assures excellent counting efficiency at low and high concentrations and complies with the ISO 21501-1standard for OPCs. With all these features, the EDM264 is a world-leading dust monitor for precise monitoring of particulate matter and particle number concentration. This highly reliable instrument is an indispensable tool for many users who need to measure aerosol levels and air quality outdoors, on construction sites, or at production facilities.

Keywords: aerosol research, aerial observation, fence line monitoring, wild fire detection

Procedia PDF Downloads 115

Authors: Erdianta S, Chastiti M. Wulolo, IDK Kerta Widana

Abstract:

Perang Semesta strategy is a national security system used by Republic of Indonesia. It comes from local wisdom, cultural, and hereditary of Indonesia itself. This system involves all people and all nation resources, and it is early prepared by government and conducted totality, integratedly, directly, and continously to enforce a sovereignty of country, teritorial integrity and the safety of the whole nation from threats. This study uses a qualitative content analysis method by studying, recording, and analyzing government policy. The Perang Semesta strategy divided into main, backup, and supporting components. Every component has its function and responsibility in security perspective. So when an attack comes, all people of Indonesia will voluntary to defend the country. Perang Semesta strategy is a national security system which becomes the most reliable strategy toward geography and demography of Indonesia.

Keywords: Indonesia, Perang Semesta strategy, national security, local wisdom

Procedia PDF Downloads 416

Authors: Nikat Parveen, M. Ananthi

Abstract:

Big data is one of the emerging technology, which collects the data from various sensors and those data will be used in many fields. Data retrieval is one of the major issue where there is a need to extract the exact data as per the need. In this paper, large amount of data set is processed by using the feature selection. Feature selection helps to choose the data which are actually needed to process and execute the task. The key value is the one which helps to point out exact data available in the storage space. Here the available data is streamed and R-Center is proposed to achieve this task.

Keywords: big data, key value, feature selection, retrieval, performance

Procedia PDF Downloads 304

Authors: Ahmadu Maidorawa, Kamalrulnizam Abu Bakar

Abstract:

This paper proposed an Enhanced Connectivity Aware Routing (ECAR) protocol for Vehicular Ad hoc Network (VANET). The protocol uses a control broadcast to reduce the number of overhead packets needed in a route discovery process. It is also equipped with an alternative backup route that is used whenever a primary path to destination failed, which highly reduces the frequent launching and re-launching of the route discovery process that waste useful bandwidth and unnecessarily prolonging the average packet delay. NS2 simulation results show that the performance of ECAR protocol outperformed the original connectivity aware routing (CAR) protocol by reducing the average packet delay by 28%, control overheads by 27% and increased the packet delivery ratio by 22%.

Keywords: alternative path, primary path, protocol, routing, VANET, vehicular ad hoc networks

Procedia PDF Downloads 369

Authors: Lefteris Tsipis, Demosthenes Vouyioukas, George Loumos, Antonis Kargas, Dimitris Varoutas

Abstract:

The employment of technology and digitization is crucial for cultural organizations to establish and sustain digital repositories for their cultural heritage artefacts. This utilization is also essential in facilitating the presentation of cultural works and exhibits to a broader audience. Consequently, in this work, we propose a digital repository that functions as Software as a Service (SaaS), primarily promoting the safe storage, display, and sharing of cultural materials, enhancing accessibility, and fostering a deeper understanding and appreciation of cultural heritage. Moreover, the proposed digital repository service is designed as a multitenant architecture, which enables organizations to expand their reach, enhance accessibility, foster collaboration, and ensure the preservation of their content. Specifically, this project aims to assist each cultural institution in organizing its digital cultural assets into collections and feeding other digital platforms, including educational, museum, pedagogical, and games, through appropriate interfaces. Moreover, the creation of this digital repository offers a cutting-edge and effective open-access laboratory solution. It allows organizations to have a significant influence on their audiences by fostering cultural understanding and appreciation. Additionally, it facilitates the connection between different digital repositories and national/European aggregators, promoting collaboration and information sharing. By embracing this solution, cultural institutions can benefit from shared resources and features, such as system updates, backup and recovery services, and data analytics tools, that are provided by the platform.

Keywords: cultural technologies, gaming technologies, web sharing, digital repository

Procedia PDF Downloads 43

Authors: Faisal Kalota

Abstract:

Big Data and analytics have gained a huge momentum in recent years. Big Data feeds into the field of Learning Analytics (LA) that may allow academic institutions to better understand the learners’ needs and proactively address them. Hence, it is important to have an understanding of Big Data and its applications. The purpose of this descriptive paper is to provide an overview of Big Data, the technologies used in Big Data, and some of the applications of Big Data in education. Additionally, it discusses some of the concerns related to Big Data and current research trends. While Big Data can provide big benefits, it is important that institutions understand their own needs, infrastructure, resources, and limitation before jumping on the Big Data bandwagon.

Keywords: big data, learning analytics, analytics, big data in education, Hadoop

Procedia PDF Downloads 380

Authors: Mustafa Dagbasi, Olusola Bamisile, Adii Chinedum

Abstract:

The objective of this work is to examine and compare the economic and environmental feasibility of 40MW photovoltaic (PV) power plant and 40MW parabolic trough (PT) power plant to be installed in two different cities, namely Nicosia and Famagusta in Turkish Republic of Northern Cyprus (TRNC). The need for using solar power technology around the world is also emphasized. Solar radiation and sunshine data for Nicosia and Famagusta are considered and analyzed to assess the distribution of solar radiation, sunshine duration, and air temperature. Also, these two different technologies with same rated power of 40MW will be compared with the performance of the proposed Solar Power Plant at Bari, Italy. The project viability analysis is performed using System Advisor Model (SAM) through Annual Energy Production and economic parameters for both cities. It is found that for the two cities; Nicosia and Famagusta, the investment is feasible for both 40MW PV power plant and 40MW PT power plant. From the techno-economic analysis of these two different solar power technologies having same rated power and under the same environmental conditions, PT plants produce more energy than PV plant. It is also seen that if a PT plant is installed near an existing steam turbine power plant, the steam from the PT system can be used to run this turbine which makes it more feasible to invest. The high temperatures that are used to produce steam for the turbines in the PT plant system can be supplemented with a secondary plant based on natural gas or other biofuels and can be used as backup. Although the initial investment of PT plant is higher, it has higher economic return and occupies smaller area compared to PV plant of the same capacity.

Keywords: solar power, photovoltaic plant, parabolic trough plant, techno-economic analysis

Procedia PDF Downloads 255

Authors: Sandeep Sharma, Sarabjit Singh

Abstract:

As per the user demand and growth trends of large free data the storage solutions are now becoming more challenge-able to protect, store and to retrieve data. The days are not so far when the storage companies and organizations are start saying 'no' to store our valuable data or they will start charging a huge amount for its storage and protection. On the other hand as per the environmental conditions it becomes challenge-able to maintain and establish new data warehouses and data centers to protect global warming threats. A challenge of small data is over now, the challenges are big that how to manage the exponential growth of data. In this paper we have analyzed the growth trend of big data and its future implications. We have also focused on the impact of the unstructured data on various concerns and we have also suggested some possible remedies to streamline big data.

Keywords: big data, unstructured data, volume, variety, velocity

Procedia PDF Downloads 510

Authors: Yang Bao, Shi Wei Deng, WangQun Lin

Abstract:

This paper introduces the concept and principle of data cleaning, analyzes the types and causes of dirty data, and proposes several key steps of typical cleaning process, puts forward a well scalability and versatility data cleaning framework, in view of data with attribute dependency relation, designs several of violation data discovery algorithms by formal formula, which can obtain inconsistent data to all target columns with condition attribute dependent no matter data is structured (SQL) or unstructured (NoSQL), and gives 6 data cleaning methods based on these algorithms.

Keywords: data cleaning, dependency rules, violation data discovery, data repair

Procedia PDF Downloads 530

Authors: Yujiang Wu

Abstract:

As electrical machine (e-machine) power density re-querulents become more stringent in vehicle electrification, mounting a temperature sensor for e-machine stator windings becomes increasingly difficult. This can lead to higher manufacturing costs, complicated harnesses, and reduced reliability. In this paper, we propose a deep-learning method for predicting electric machine winding temperature, which can either replace the sensor entirely or serve as a backup to the existing sensor. We compare the performance of our method, the stateful long short-term memory networks (LSTM) with truncated backpropagation through time (TBTT), with that of linear regression, as well as stateless LSTM with/without residual connection. Our results demonstrate the strength of combining stateful LSTM and TBTT in tackling nonlinear time series prediction problems with long sequence lengths. Additionally, in industrial applications, high-temperature region prediction accuracy is more important because winding temperature sensing is typically used for derating machine power when the temperature is high. To evaluate the performance of our algorithm, we developed a temperature-stratified MSE. We propose a simple but effective data preprocessing trick to improve the high-temperature region prediction accuracy. Our experimental results demonstrate the effectiveness of our proposed method in accurately predicting winding temperature, particularly in high-temperature regions, while also reducing manufacturing costs and improving reliability.

Keywords: deep learning, electrical machine, functional safety, long short-term memory networks (LSTM), thermal management, time series prediction

Procedia PDF Downloads 58

Authors: Hoda A. Abdel Hafez

Abstract:

Mining big data represents a big challenge nowadays. Many types of research are concerned with mining massive amounts of data and big data streams. Mining big data faces a lot of challenges including scalability, speed, heterogeneity, accuracy, provenance and privacy. In telecommunication industry, mining big data is like a mining for gold; it represents a big opportunity and maximizing the revenue streams in this industry. This paper discusses the characteristics of big data (volume, variety, velocity and veracity), data mining techniques and tools for handling very large data sets, mining big data in telecommunication and the benefits and opportunities gained from them.

Keywords: mining big data, big data, machine learning, telecommunication

Procedia PDF Downloads 365

Authors: Theertha Chandroth

Abstract:

This paper presents a comparative study on how to check JSON (JavaScript Object Notation) data against XML (eXtensible Markup Language) data from a software testing point of view. JSON and XML are widely used data interchange formats, each with its unique syntax and structure. The objective is to explore various techniques and methodologies for validating comparison and integration between JSON data to XML and vice versa. By understanding the process of checking JSON data against XML data, testers, developers and data practitioners can ensure accurate data representation, seamless data interchange, and effective data validation.

Keywords: XML, JSON, data comparison, integration testing, Python, SQL

Procedia PDF Downloads 83

Authors: Bolin Hua

Abstract:

In city governance, various data are involved, including city component data, demographic data, housing data and all kinds of business data. These data reflects different aspects of people, events and activities. Data generated from various systems are different in form and data source are different because they may come from different sectors. In order to reflect one or several facets of an event or rule, data from multiple sources need fusion together. Data from different sources using different ways of collection raised several issues which need to be resolved. Problem of data fusion include data update and synchronization, data exchange and sharing, file parsing and entry, duplicate data and its comparison, resource catalogue construction. Governments adopt statistical analysis, time series analysis, extrapolation, monitoring analysis, value mining, scenario prediction in order to achieve pattern discovery, law verification, root cause analysis and public opinion monitoring. The result of Multi-source data fusion is to form a uniform central database, which includes people data, location data, object data, and institution data, business data and space data. We need to use meta data to be referred to and read when application needs to access, manipulate and display the data. A uniform meta data management ensures effectiveness and consistency of data in the process of data exchange, data modeling, data cleansing, data loading, data storing, data analysis, data search and data delivery.

Keywords: multi-source data fusion, urban comprehensive management, information fusion, government data

Procedia PDF Downloads 347