Search results for: data lake

Authors: Vahid Bairami Rad

Abstract:

Extracting knowledge from spatial data like GIS data is important to reduce the data and extract information. Therefore, the development of new techniques and tools that support the human in transforming data into useful knowledge has been the focus of the relatively new and interdisciplinary research area ‘knowledge discovery in databases’. Thus, we introduce a set of database primitives or basic operations for spatial data mining which are sufficient to express most of the spatial data mining algorithms from the literature. This approach has several advantages. Similar to the relational standard language SQL, the use of standard primitives will speed-up the development of new data mining algorithms and will also make them more portable. We introduced a database-oriented framework for spatial data mining which is based on the concepts of neighborhood graphs and paths. A small set of basic operations on these graphs and paths were defined as database primitives for spatial data mining. Furthermore, techniques to efficiently support the database primitives by a commercial DBMS were presented.

Keywords: spatial data base, knowledge discovery database, data mining, spatial relationship, predictive data mining

Procedia PDF Downloads 430

Authors: B. Suraj Aravind, M. H. M. Krishna Prasad

Abstract:

There exist emerging applications of data streams that require association rule mining, such as network traffic monitoring, web click streams analysis, sensor data, data from satellites etc. Data streams typically arrive continuously in high speed with huge amount and changing data distribution. This raises new issues that need to be considered when developing association rule mining techniques for stream data. This paper proposes to introduce an improved data stream association rule mining algorithm by eliminating the limitation of resources. For this, the concept of cloud computing is used. Inclusion of this may lead to additional unknown problems which needs further research.

Keywords: data stream, association rule mining, cloud computing, frequent itemsets

Procedia PDF Downloads 474

Authors: Azzouzi Fadoua

Abstract:

This study focuses on the diagnosis of the dynamics of natural resources in a semi-arid mountainous weakened by natural vulnerability and anthropogenic action. This is evident in the forms of hydraulic erosion and degradation of agricultural land. The rate of this damaged land is 53%, with a strong presence of concentrated erosion; this shows that balanced and semi-balanced environments are less apparent to the Watershed, representing 47%. The results revealed the crucial role of the slopes and the density of the hydraulic networks to facilitate the transport of fine elements, at the level of the slopes with low vegetation intensity, to the lake of the dam. Something that endangers the siltation of the latter. After the study of natural and anthropogenic elements, it turned out that natural vulnerability is an integral part of the current dynamic, especially when it coincides with the overexploitation of natural resources, in this case, the exploitation of steep slopes for the cultivation of cereals and overgrazing. This causes the soil to pile up and increase the rate of runoff.

Keywords: watershed, erosion, natural vulnerability, anthropogenic

Procedia PDF Downloads 119

Authors: Tosin Ige

Abstract:

Ethics must be a condition of the world, like logic. (Ludwig Wittgenstein, 1889-1951). As important as data mining is, it possess a significant threat to ethics, privacy, and legality, since data mining makes it difficult for an individual or consumer (in the case of a company) to control the accessibility and usage of his data. This research focuses on Current issues and the latest research and development on Privacy preserving data mining methods as at year 2022. It also discusses some advances in those techniques while at the same time highlighting and providing a new technique as a solution to an existing technique of privacy preserving data mining methods. This paper also bridges the wide gap between Data mining and the Web Application Programing Interface (web API), where research is urgently needed for an added layer of security in data mining while at the same time introducing a seamless and more efficient way of data mining.

Keywords: data, privacy, data mining, association rule, privacy preserving, mining technique

Procedia PDF Downloads 133

Authors: A. Alexandru, C. A. Alexandru, D. Coardos, E. Tudora

Abstract:

Big Data (BD) is associated with a new generation of technologies and architectures which can harness the value of extremely large volumes of very varied data through real time processing and analysis. It involves changes in (1) data types, (2) accumulation speed, and (3) data volume. This paper presents the main concepts related to the BD paradigm, and introduces architectures and technologies for BD and BD sets. The integration of BD with the Hadoop Framework is also underlined. BD has attracted a lot of attention in the public sector due to the newly emerging technologies that allow the availability of network access. The volume of different types of data has exponentially increased. Some applications of BD in the public sector in Romania are briefly presented.

Keywords: big data, big data analytics, Hadoop, cloud

Procedia PDF Downloads 284

Authors: Sakshi Saxena

Abstract:

Bangalore is an IT industry-based metropolitan city in the state of Karnataka in India. It has experienced tremendous urbanization at the expense of the environment. The reasons behind development over and near ecologically sensitive areas have been raised by several instances of disappearing lakes. Lakes in Bangalore can be considered commons on both a local and a regional scale and these water bodies are becoming less interconnected because of encroachment in the catchment area. Other sociocultural environmental risks that have led to social issues are now a source of concern. They serve as an example of the transformations in commons, a dilemma that as is transformed from rural to urban areas, as well as the complicated institutional issues associated with governance. According to some scholarly work and ecologists, a nexus of public and commercial institutions is primarily responsible for the depletion of water tanks and the inefficiency of the planning process. It is said that Bangalore's growth as an urban centre, together with the demands it created, particularly on land and water, resulted in the emergence of a middle and upper class that was demanding and self-assured. For the report in focus, it is evident to understand the issues and problems which led to these encroachments and captured violations if any around these lakes and tanks which arose during these decades. To claim watersheds and lake edges as properties, institutional arrangements (organizations, laws, and policies) intersect with planning authorities. Because of unregulated or indiscriminate forms of urbanization, it is claimed that the engagement of actors and negotiations of the process, including government ignorance, are allowing this problem to flourish. In general, the governance of natural resources in India is largely state-based. This is due to the constitutional scheme, which since the Government of India Act, of 1935 has in principle given the power to the states to legislate in this area. Thus, states have the exclusive power to regulate water supplies, irrigation and canals, drainage and embankments, water storage, hydropower, and fisheries. Thus, The main aim is to understand institutional arrangements and the master planning processes behind these arrangements. To understand the ambiguity through an example, it is noted that, Custodianship alone is a role divided between two state and two city-level bodies. This creates regulatory ambiguity and the effects on the environment are such as changes in city temperature, urban flooding, etc. As established, the main kinds of issues around lakes/tanks in Bangalore are encroachment and depletion. This study will further be enhanced by doing a physical survey of three of these lakes focusing on the Bellandur site and the stakeholders involved. According to the study's findings thus far, corrupt politicians and dubious land transaction tools are involved in the real estate industry. It appears that some destruction could have been stopped or at least mitigated in this case if there had been a robust system of urban planning processes involved along with strong institutional arrangements to protect lakes.

Keywords: wetlands, lakes, urbanization, bangalore, politics, reservoirs, municipal jurisdiction, lake connections, institutions

Procedia PDF Downloads 60

Authors: Aïcha Ben Salem, Faouzi Boufares, Sebastiao Correia

Abstract:

The subject covered in this paper aims at assisting the user in its quality approach. The goal is to better extract, mix, interpret and reuse data. It deals with the semantic schema recognition of a data source. This enables the extraction of data semantics from all the available information, inculding the data and the metadata. Firstly, it consists of categorizing the data by assigning it to a category and possibly a sub-category, and secondly, of establishing relations between columns and possibly discovering the semantics of the manipulated data source. These links detected between columns offer a better understanding of the source and the alternatives for correcting data. This approach allows automatic detection of a large number of syntactic and semantic anomalies.

Keywords: schema recognition, semantic data profiling, meta-categorisation, semantic dependencies inter columns

Procedia PDF Downloads 397

Authors: Luojie Tang, Libin Ouyang, Yihang Gao

Abstract:

The natural landscape and urban-rural structure, with their attractiveness in the Dali area around Erhai Lake, exhibit striking similarities with Howard's Garden City. With the emergence of the unique phenomenon of the first large-scale gathering of digital nomads in China in Dali, an analysis of Dali's natural, economic, and cultural representations and structures reveals that the Garden City model can no longer fully explain the current overall human living environment in Dali. By interpreting the bottom-up local construction process in Dali based on landscape identity, the transformation of production and lifestyle under new technologies such as ICT(Information and Communication Technology), and the values and lifestyle reshaping embodied in the "reverse urbanization" phenomenon of the middle class in Dali, it is believed that Dali has moved towards a "contemporary garden city influenced by new technology." The article summarizes the characteristics and connotations of this Garden City and provides corresponding strategies for its continued healthy development.

Keywords: dali, ICT, rural-urban relationship, garden city model

Procedia PDF Downloads 47

Authors: Winfred Okoe Addy, Jean Jacques Dominique Beraud

Abstract:

Access control systems (ACs) are some of the most important components in safety areas. Inaccuracies of regulatory frameworks make personal policies and remedies more appropriate than standard models or protocols. This problem is exacerbated by the increasing complexity of software, such as integrated Big Data (BD) software for controlling large volumes of encrypted data and resources embedded in a dedicated BD production system. This paper proposes a general access control strategy system for the diffusion of Big Data domains since it is crucial to secure the data provided to data consumers (DC). We presented a general access control circulation strategy for the Big Data domain by describing the benefit of using designated access control for BD units and performance and taking into consideration the need for BD and AC system. We then presented a generic of Big Data access control system to improve the dissemination of Big Data.

Keywords: access control, security, Big Data, domain

Procedia PDF Downloads 111

Authors: Michael Gidey Gebru

Abstract:

Most of Data Envelopment Analysis models operate in a static environment with input and output parameters that are chosen by deterministic data. However, due to ambiguity brought on shifting market conditions, input and output data are not always precisely gathered in real-world scenarios. Fuzzy numbers can be used to address this kind of ambiguity in input and output data. Therefore, this work aims to expand crisp Data Envelopment Analysis into Data Envelopment Analysis with fuzzy environment. In this study, the input and output data are regarded as fuzzy triangular numbers. Then, the Data Envelopment Analysis model with fuzzy environment is solved using a multi-objective method to gauge the Decision Making Units' efficiency. Finally, the developed Data Envelopment Analysis model is illustrated with an application on real data 50 educational institutions.

Keywords: efficiency, Data Envelopment Analysis, fuzzy, higher education, input, output

Procedia PDF Downloads 22

Authors: Anja Jutraz, Tadeja Zupancic

Abstract:

This article explores urban laboratory, which presents a combination of different physical and digital methods and tools for public participation in urban design. The city consists of built and unbuilt environments, which can be defined as a community of people, who live there. Communities should have the option to express opinions and decide about the future of their city, from the early stages of the design process onwards. In this paper, we presented the possibility of involving community into renewal of Banska Štiavnica in Slovakia (more exactly the old mining shaft and lake Michal Štolna) and the methods to promote the community building. As a case study we presented the eTHNo project, Education about Technical, Historical and Natural opportunities of Michal Štolna. Moreover, we discussed the possibility of using virtual digital tools for public participation in urban design, where we especially focused on Virtual Urban Laboratory, VuLab.

Keywords: community building, digital tools, public participation, urban design

Procedia PDF Downloads 544

Authors: Namubiru Zula, Janice Desire Busingue

Abstract:

Pro-active educational approaches have lately been adapted Globally in the Conservation of Natural Resources. This led to the introduction of the co-management system, which worked for some European Countries on the conservation of sharks and other Natural resources. However, this approach has drastically failed in the Fishery sector on Lake Victoria; and the punitive education approach has been re-instated. Literature is readily available about the punitive educational approaches and scanty with the pro-active one. This article analyses the pro-active approach adopted by the Department of Fisheries for the orientation of BMU leaders in a co-management system. The study is interpreted using the social constructivist lens for co-management of the fishery resource to ensure that fishers are also back to fishing sustainably. It highlights some of the education methods used, methodological challenges that included the power and skills gap of the facilitators and program designers, and some implications to practice.

Keywords: beach management units, fishers, education methods, proactive approach, punitive approach

Procedia PDF Downloads 99

Authors: Mentor Sulollari

Abstract:

Albania, according to (UNU-EHS) United Nations University, Institute for Environment and Human Security studies for 2015, is listed as the number one country in Europe for the possibility to be caught by natural catastrophes. This is conditioned by unstudied human activity, which has seriously damaged the environment. Albanian part of Ohrid graben that lies in Southeast of Albania, is endangered by landslides and floods, as a result of uncontrolled urban development and low level of investment in infrastructure, rugged terrain in its western part and capricious climate caused by global warming. To be dealt with natural disasters, which cause casualties and material damage, it is important to study them in order to anticipate and reduce damages in future. As part of this study is the construction of natural hazards map, which show us where they are distributed, and which are the vulnerable areas. This article will also be dealing with socio-economic and environmental costs of those events and what are the measures to be taken to reduce them.

Keywords: flooding, landslides, natural catastrophes mapping, Pogradec, lake Ohrid, Albanian part of Ohrid graben

Procedia PDF Downloads 270

Authors: Kacper Tomasz Kröber-Mulawa

Abstract:

This paper will address the topic of data ownership from an economic perspective, and examples of economic limitations of data property rights will be provided, which have been identified using methods and approaches of economic analysis of law. To properly build a background for the economic focus, in the beginning a short perspective of data and data ownership in the EU’s legal system will be provided. It will include a short introduction to its political and social importance and highlight relevant viewpoints. This will stress the importance of a Single Market for data but also far-reaching regulations of data governance and privacy (including the distinction of personal and non-personal data, data held by public bodies and private businesses). The main discussion of this paper will build upon the briefly referred to legal basis as well as methods and approaches of economic analysis of law.

Keywords: antitrust, data, data ownership, digital economy, property rights

Procedia PDF Downloads 51

Authors: Getalem E. Haylia

Abstract:

The Ribb dam is one of the irrigation projects in the Upper Blue Nile basin, Ethiopia, to irrigate the Fogera plain. Reservoir sedimentation is a major problem because it reduces the useful reservoir capacity by the accumulation of sediments coming from the watersheds. Estimates of sediment yield are needed for studies of reservoir sedimentation and planning of soil and water conservation measures. The objective of this study was to simulate the Ribb dam catchment sediment yield using SWAT model and to estimate Ribb reservoir effective life according to trap efficiency methods. The Ribb dam catchment is found in North Western part of Ethiopia highlands, and it belongs to the upper Blue Nile and Lake Tana basins. Soil and Water Assessment Tool (SWAT) was selected to simulate flow and sediment yield in the Ribb dam catchment. The model sensitivity, calibration, and validation analysis at Ambo Bahir site were performed with Sequential Uncertainty Fitting (SUFI-2). The flow data at this site was obtained by transforming the Lower Ribb gauge station (2002-2013) flow data using Area Ratio Method. The sediment load was derived based on the sediment concentration yield curve of Ambo site. Stream flow results showed that the Nash-Sutcliffe efficiency coefficient (NSE) was 0.81 and the coefficient of determination (R²) was 0.86 in calibration period (2004-2010) and, 0.74 and 0.77 in validation period (2011-2013), respectively. Using the same periods, the NS and R² for the sediment load calibration were 0.85 and 0.79 and, for the validation, it became 0.83 and 0.78, respectively. The simulated average daily flow rate and sediment yield generated from Ribb dam watershed were 3.38 m³/s and 1772.96 tons/km²/yr, respectively. The effective life of Ribb reservoir was estimated using the developed empirical methods of the Brune (1953), Churchill (1948) and Brown (1958) methods and found to be 30, 38 and 29 years respectively. To conclude, massive sediment comes from the steep slope agricultural areas, and approximately 98-100% of this incoming annual sediment loads have been trapped by the Ribb reservoir. In Ribb catchment, as well as reservoir systematic and thorough consideration of technical, social, environmental, and catchment managements and practices should be made to lengthen the useful life of Ribb reservoir.

Keywords: catchment, reservoir effective life, reservoir sedimentation, Ribb, sediment yield, SWAT model

Procedia PDF Downloads 154

Authors: Abdullah Alsaeed

Abstract:

Virtualized computing and cloud computing infrastructures are no longer fuzz or marketing term. They are a core reality in today’s corporate Information Technology (IT) organizations. Hence, developing an effective and efficient methodologies for data backup and data recovery is required more than any time. The purpose of data backup and recovery techniques are to assist the organizations to strategize the business continuity and disaster recovery approaches. In order to accomplish this strategic objective, a variety of mechanism were proposed in the recent years. This research paper will explore and examine the latest techniques and solutions to provide data backup and restoration for the cloud computing platforms.

Keywords: data backup, data recovery, cloud computing, business continuity, disaster recovery, cost-effective, data encryption.

Procedia PDF Downloads 59

Authors: JaeHwan Yang, Da-Woon Jeong, Seung-Young Kho, Dong-Kyu Kim

Abstract:

In terms of ITS, information on link characteristic is an essential factor for plan or operation. But in practical cases, not every link has installed sensors on it. The link that does not have data on it is called “Missing Link”. The purpose of this study is to impute data of these missing links. To get these data, this study applies the machine learning method. With the machine learning process, especially for the deep learning process, missing link data can be estimated from present link data. For deep learning process, this study uses “Recurrent Neural Network” to take time-series data of road. As input data, Dedicated Short-range Communications (DSRC) data of Dalgubul-daero of Daegu Metropolitan Area had been fed into the learning process. Neural Network structure has 17 links with present data as input, 2 hidden layers, for 1 missing link data. As a result, forecasted data of target link show about 94% of accuracy compared with actual data.

Keywords: data estimation, link data, machine learning, road network

Procedia PDF Downloads 488

Authors: Monica Lia

Abstract:

This article presents a customer data analysis model using business intelligence tools for data modelling, transforming, data visualization and dynamic reports building. Economic organizational customer’s analysis is made based on the information from the transactional systems of the organization. The paper presents how to develop the data model starting for the data that companies have inside their own operational systems. The owned data can be transformed into useful information about customers using business intelligence tool. For a mature market, knowing the information inside the data and making forecast for strategic decision become more important. Business Intelligence tools are used in business organization as support for decision-making.

Keywords: customer analysis, business intelligence, data warehouse, data mining, decisions, self-service reports, interactive visual analysis, and dynamic dashboards, use cases diagram, process modelling, logical data model, data mart, ETL, star schema, OLAP, data universes

Procedia PDF Downloads 400

Authors: Kaiping Xu, Aiju You, Lei Hua

Abstract:

In view of the ecological environmental problems and protection needs of mountainous rivers in Zhejiang province, a suitable ecological water demand evaluation system was established based on investigation and monitoring. Taking the Kaihua river as an example, the research on ecological water demand and the current situation evaluation were carried out. The main types of ecological water demand in Majin River are basic ecological flow and lake wetland outside the river, and instream flow and water demands for water quality in Zhongcun river. In the wet season, each ecological water demand is 18.05m3/s and 2.56m3 / s, and in the dry season is 3.00m3/s and 0.61m3/s. Three indexes of flow, duration and occurrence time are used to evaluate the ecological water demand. The degree of ecological water demand in the past three years is low level of satisfaction. Meanwhile, the existing problems are analyzed, and put forward reasonable and operable safeguards and suggestions.

Keywords: Zhejiang province, mountainous river, ecological water demand, Kaihua river, evaluation

Procedia PDF Downloads 200

Authors: S. Begum, T. Biswas, M. A. Islam

Abstract:

The present research investigates the distribution and contamination of heavy metals in core sediments collected from three locations of the Sundarbans mangrove forest. In this research, quality of the analysis is evaluated by analyzing certified reference materials IAEA-SL-1 (lake sediment), IAEA-Soil-7, and NIST-1633b (coal fly ash). Total concentrations of 28 heavy metals (Na, Al, K, Ca, Sc, Ti, V, Cr, Mn, Fe, Co, Zn, Ga, As, Sb, Cs, La, Ce, Sm, Eu, Tb, Dy, Ho, Yb, Hf, Ta, Th, and U) have determined in core sediments of the Sundarbans mangrove by neutron activation analysis (NAA) technique. When compared with upper continental crustal (UCC) values, it is observed that mean concentrations of K, Ti, Zn, Cs, La, Ce, Sm, Hf, and Th show elevated values in the research area is high. In this research, the assessments of metal contamination levels using different environmental contamination indices (EF, Igeo, CF) indicate that Ti, Sb, Cs, REEs, and Th have minor enrichment of the sediments of the Sundarbans. The modified degree of contamination (mCd) of studied samples of the Sundarbans ecosystem show low contamination. The pollution load index (PLI) values for the cores suggested that sampling points are moderately polluted. The possible sources of the deterioration of the sediment quality can be attributed to the different chemical carrying cargo accidents, port activities, ship breaking, agricultural and aquaculture run-off of the area. Pearson correlation matrix (PCM) established relationships among elements. The PCM indicates that most of the metal's distributions have been controlled by the same factors such as Fe-oxy-hydroxides and clay minerals, and also they have a similar origin. The poor correlations of Ca with most of the elements in the sediment cores indicate that calcium carbonate has a less significant role in this mangrove sediment. Finally, the data from this research will be used as a benchmark for future research and help to quantify levels of metal pollutions, as well as to manage future ecological risks of the vulnerable mangrove ecosystem, the Sundarbans.

Keywords: contamination, core sediment, trace element, sundarbans, vulnerable

Procedia PDF Downloads 98

Authors: K. Hardy, A. Maurushat

Abstract:

Policy makers are increasingly looking to make evidence-based decisions. Evidence-based decisions have historically used rigorous methodologies of empirical studies by research institutes, as well as less reliable immediate survey/polls often with limited sample sizes. As we move into the era of Big Data analytics, policy makers are looking to different methodologies to deliver reliable empirics in real-time. The question is not why did these people do this for the last 10 years, but why are these people doing this now, and if the this is undesirable, and how can we have an impact to promote change immediately. Big data analytics rely heavily on government data that has been released in to the public domain. The open data movement promises greater productivity and more efficient delivery of services; however, Australian government agencies remain reluctant to release their data to the general public. This paper considers the barriers to releasing government data as open data, and how these barriers might be overcome.

Keywords: big data, open data, productivity, data governance

Procedia PDF Downloads 344

Authors: Abeer Ahmed Abdehameed

Abstract:

Background: Diabetic sensorimotor polyneuropathy (DSP) is one of the most common micro vascular complications of type 2 diabetes. Loss of sensation is thought to contribute to lake of static and dynamic stability and increased risk of falling. Purpose: The purpose of this study was to compare the effects of low level laser (LLL) and monochromatic near infrared photo energy (MIRE) on pain , cutaneous sensation, static stability and index of lower limb blood flow in diabetic with peripheral neuropathy. Methods: Forty subjects with diabetic peripheral neuropathy were recruited for study. They were divided into two groups: The ( MIRE) group that included (20) patients and (LLL) group included (20) patients. All patients in the study had been subjected to various physical assessment procedures including pain, cutaneous sensation, Doppler flow meter and static stability assessments. The baseline measurements were followed by treatment sessions that conducted twice a week for 6 successive weeks. Results: The statistical analysis of the data had revealed significant improvement of the pain in both groups, with significant improvement in cutaneous sensation and static balance in (MIRE) group compared to (LLL) group; on the other hand results showed no significant differences on lower limb blood flow in both groups. Conclusion: Low level laser and monochromatic near infrared therapy can improve painful symptoms in patients with diabetic neuropathy. On the other hand (MIRE) is useful in improving cutaneous sensation and static stability in patients with diabetic neuropathy.

Keywords: diabetic neuropathy, doppler flow meter, low level laser, monochromatic near infrared photo energy

Procedia PDF Downloads 289

Authors: Birtukan Atinkut Asmare

Abstract:

Climate change disproportionately affects many Africans who heavily rely on climate-exposed sectors such as rain-fed agriculture and fishing, rendering them highly vulnerable. Gender plays a significant role, as men and women experience unequal impacts and vulnerabilities due to gender norms, labor divisions, resource access, and power dynamics. Drawing on an integrated framework, this study sheds light on the gendered impacts of climate change on household’s livelihood, their vulnerability, and adaptation in rural Ethiopia's Lake Tana Basin. This study utilized mixed research methods, integrating diverse qualitative techniques such as focus group discussions, key informant interviews, and field observations, along with quantitative data gathered through household surveys. The findings reveal that women-headed households were more vulnerable to climate change than male-headed households. Flood was the major climate-induced hazards in the area that threatened the lives and livelihoods of households. In response to climate change, households undertook different adaptation measures such as agroforestry practices, crop diversification, seasonal migration, petty trading, charcoal and fuel wood sales. However, the adaptation strategies were slightly varied based on the gender of the household head. Women-headed households specifically engaged in fuelwood collection and selling and petty trading activities. The main constraints for adaptation were limited access to technologies, extension services, information, and financial services. Therefore, this research urges attention from research, policy, and advisory services on rural households who are trying to survive in the face of climate change.

Keywords: agriculture, climate change impacts, ethiopia, gender

Procedia PDF Downloads 34

Authors: Hema Bhardwaj, D. Srinivasa Rao

Abstract:

Technology for analysing, processing, and extracting meaningful data from enormous and complicated datasets can be termed as "big data." The technique of big data mining and big data analysis is extremely helpful for business movements such as making decisions, building organisational plans, researching the market efficiently, improving sales, etc., because typical management tools cannot handle such complicated datasets. Special computational and statistical issues, such as measurement errors, noise accumulation, spurious correlation, and storage and scalability limitations, are brought on by big data. These unique problems call for new computational and statistical paradigms. This research paper offers an overview of the literature on big data mining, its process, along with problems and difficulties, with a focus on the unique characteristics of big data. Organizations have several difficulties when undertaking data mining, which has an impact on their decision-making. Every day, terabytes of data are produced, yet only around 1% of that data is really analyzed. The idea of the mining and analysis of data and knowledge discovery techniques that have recently been created with practical application systems is presented in this study. This article's conclusion also includes a list of issues and difficulties for further research in the area. The report discusses the management's main big data and data mining challenges.

Keywords: big data, data mining, data analysis, knowledge discovery techniques, data mining challenges

Procedia PDF Downloads 85

Authors: Rimmy Yadav, Anmol Preet Kaur

Abstract:

Big Data has demonstrated the vast potential in streamlining, deciding, spotting business drifts in different fields, for example, producing, fund, Information Technology. This paper gives a multi-disciplinary diagram of the research issues in enormous information and its procedures, instruments, and system identified with the privacy, data storage management, network and energy utilization, adaptation to non-critical failure and information representations. Other than this, result difficulties and openings accessible in this Big Data platform have made.

Keywords: big data, privacy, data management, network and energy consumption

Procedia PDF Downloads 280

Authors: Mansoureh Ghiasabadi Farahani, Samira Kalantary, Sara Taghi-Pour, Mahboubeh Shamsi

Abstract:

Nowadays, the development of computers technology and its recent applications provide access to new types of data, which have not been considered by the traditional data analysts. Two particularly interesting characteristics of such data sets include their huge size and streaming nature .Incremental learning techniques have been used extensively to address the data stream classification problem. This paper presents a concise survey on the obstacles and the requirements issues classifying data streams with using decision tree. The most important issue is to maintain a balance between accuracy and efficiency, the algorithm should provide good classification performance with a reasonable time response.

Keywords: big data, data streams, classification, decision tree

Procedia PDF Downloads 490

Authors: Aishwarya Shekhar, Himanshu Sharma

Abstract:

Data deduplication is one of important data compression techniques for eliminating duplicate copies of repeating data, and has been widely used in cloud storage to reduce the amount of storage space and save bandwidth. In this process, duplicate data is expunged, leaving only one copy means single instance of the data to be accumulated. Though, indexing of each and every data is still maintained. Data deduplication is an approach for minimizing the part of storage space an organization required to retain its data. In most of the company, the storage systems carry identical copies of numerous pieces of data. Deduplication terminates these additional copies by saving just one copy of the data and exchanging the other copies with pointers that assist back to the primary copy. To ignore this duplication of the data and to preserve the confidentiality in the cloud here we are applying the concept of hybrid nature of cloud. A hybrid cloud is a fusion of minimally one public and private cloud. As a proof of concept, we implement a java code which provides security as well as removes all types of duplicated data from the cloud.

Keywords: confidentiality, deduplication, data compression, hybridity of cloud

Procedia PDF Downloads 360

Authors: Devatha Kalyan Kumar, Aravindraj D., Sadathulla A.

Abstract:

Big data are now rapidly expanding in all engineering and science and many other domains. The potential of large or massive data is undoubtedly significant, make sense to require new ways of thinking and learning techniques to address the various big data challenges. Machine learning is continuously unleashing its power in a wide range of applications. In this paper, the latest advances and advancements in the researches on machine learning for big data processing. First, the machine learning techniques methods in recent studies, such as deep learning, representation learning, transfer learning, active learning and distributed and parallel learning. Then focus on the challenges and possible solutions of machine learning for big data.

Keywords: active learning, big data, deep learning, machine learning

Procedia PDF Downloads 410

Authors: Tomy Prihananto, Damar Apri Sudarmadi

Abstract:

Indonesia recognizes the right to privacy as a human right. Indonesia provides legal protection against data management activities because the protection of personal data is a part of human rights. This paper aims to describe the arrangement of data management and data management in Indonesia. This paper is a descriptive research with qualitative approach and collecting data from literature study. Results of this paper are comprehensive arrangement of data that have been set up as a technical requirement of data protection by encryption methods. Arrangements on encryption and protection of personal data are mutually reinforcing arrangements in the protection of personal data. Indonesia has two important and immediately enacted laws that provide protection for the privacy of information that is part of human rights.

Keywords: Indonesia, protection, personal data, privacy, human rights, encryption

Procedia PDF Downloads 157

Authors: Pooria Pasbakhsh, Rangika T. De Silva, Vahdat Vahedi, Hanafi Ismail

Abstract:

The influence of three different types of halloysite nanotubes (HNTs) with different dimensions, namely as camel lake (CLA), Jarrahdale (JA) and Matauri Bay (MB), on their reinforcing ability of ethylene propylene dine monomer (EPDM) were investigated by varying the HNTs loading (from 0-15 phr). Mechanical properties of the nanocomposites improved with addition of all three HNTs, but CLA based nanocomposites exhibited a significant enhancement compared to the other HNTs. For instance, tensile properties of EPDM nanocomposites increased by 120%, 256% and 340% for MB, JA, and CLA, respectively with addition of 15 phr of HNTs. This could be due to the higher aspect ratio and higher surface area of CLA compared to others. Scanning electron microscopy (SEM) of nanocomposites at 15 phr of HNT loadings showed low amounts of pulled-out nanotubes which confirmed the presence of more embedded nanotubes inside the EPDM matrix, as well as aggregates within the fracture surface of EPDM/HNT nanocomposites.

Keywords: aspect ratio, halloysite nanotubes (HNTs), mechanical properties, rubber/clay nanocomposites

Procedia PDF Downloads 352