Search results for: multi-agent reinforcement learning (MARL)

Authors: Xiaohua Zou, Yongxin Su

Abstract:

The reasonable operation of heating, ventilation, and air conditioning (HVAC) is of great significance in improving the security, stability, and economy of power system operation. However, the uncertainty of the operating environment, thermal comfort varies by users and rapid decision-making pose challenges for HVAC demand response optimization. In this regard, this paper proposes a reinforcement learning-based method for HVAC demand response optimization considering few-shot personalized thermal comfort (PTC). First, an HVAC DR optimization framework based on few-shot PTC model and DRL is designed, in which the output of few-shot PTC model is regarded as the input of DRL. Then, a few-shot PTC model that distinguishes between awake and asleep states is established, which has excellent engineering usability. Next, based on soft actor criticism, an HVAC DR optimization algorithm considering the user’s PTC is designed to deal with uncertainty and make decisions rapidly. Experiment results show that the proposed method can efficiently obtain use’s PTC temperature, reduce energy cost while ensuring user’s PTC, and achieve rapid decision-making under uncertainty.

Keywords: HVAC, few-shot personalized thermal comfort, deep reinforcement learning, demand response

Procedia PDF Downloads 78

Authors: Farzaneh Sarbandi Farahani

Abstract:

Drought and decrease in the level of lakes in recent years due to global warming and excessive use of water resources feeding lakes are of great importance, and this research has provided a structure to investigate this issue. First, the information required for simulating lake drought is provided with strong references and necessary assumptions. Entity-Component-System (ECS) structure has been used for simulation, which can consider assumptions flexibly in simulation. Three major users (i.e., Industry, agriculture, and Domestic users) consume water from groundwater and surface water (i.e., streams, rivers and lakes). Lake Mead has been considered for simulation, and the information necessary to investigate its drought has also been provided. The results are presented in the form of a scenario-based design and optimal strategy selection. For optimal strategy selection, a deep reinforcement algorithm is developed to select the best set of strategies among all possible projects. These results can provide a better view of how to plan to prevent lake drought.

Keywords: drought simulation, Mead lake, entity component system programming, deep reinforcement learning

Procedia PDF Downloads 84

Authors: Alica Höpken, Hergen Pargmann

Abstract:

The resource-efficient planning of the complex production planning processes in the meat industry and the reduction of food waste is a permanent challenge. The complexity of the production planning process occurs in every part of the supply chain, from agriculture to the end consumer. It arises from long and uncertain planning phases. Uncertainties such as stochastic yields, fluctuations in demand, and resource variability are part of this process. In the meat industry, waste mainly relates to incorrect storage, technical causes in production, or overproduction. The high amount of food waste along the complex supply chain in the meat industry could not be reduced by simple solutions until now. Therefore, resource-efficient production planning by conventional methods is currently only partially feasible. The realization of intelligent, automated production planning is basically possible through the application of machine learning algorithms, such as those of reinforcement learning. By applying the adapted design thinking method, machine learning methods (especially reinforcement learning algorithms) are used for the complex production planning process in the meat industry. This method represents a concretization to the application area. A resource-efficient production planning process is made available by adapting the design thinking method. In addition, the complex processes can be planned efficiently by using this method, since this standardized approach offers new possibilities in order to challenge the complexity and the high time consumption. It represents a tool to support the efficient production planning in the meat industry. This paper shows an elegant adaption of the design thinking method to apply the reinforcement learning method for a resource-efficient production planning process in the meat industry. Following, the steps that are necessary to introduce machine learning algorithms into the production planning of the food industry are determined. This is achieved based on a case study which is part of the research project ”REIF - Resource Efficient, Economic and Intelligent Food Chain” supported by the German Federal Ministry for Economic Affairs and Climate Action of Germany and the German Aerospace Center. Through this structured approach, significantly better planning results are achieved, which would be too complex or very time consuming using conventional methods.

Keywords: change management, design thinking method, machine learning, meat industry, reinforcement learning, resource-efficient production planning

Procedia PDF Downloads 123

Authors: Rui Wu

Abstract:

In the volatile modern manufacturing environment, new orders randomly occur at any time, while the pre-emptive methods are infeasible. This leads to a real-time scheduling method that can produce a reasonably good schedule quickly. The dynamic Flexible Job Shop problem is an NP-hard scheduling problem that hybrid the dynamic Job Shop problem with the Parallel Machine problem. A Flexible Job Shop contains different work centres. Each work centre contains parallel machines that can process certain operations. Many algorithms, such as genetic algorithms or simulated annealing, have been proposed to solve the static Flexible Job Shop problems. However, the time efficiency of these methods is low, and these methods are not feasible in a dynamic scheduling problem. Therefore, a dynamic rule selection scheduling system based on the reinforcement learning method is proposed in this research, in which the dynamic Flexible Job Shop problem is divided into several parallel machine problems to decrease the complexity of the dynamic Flexible Job Shop problem. Firstly, the features of jobs, machines, work centres, and flexible job shops are selected to describe the status of the dynamic Flexible Job Shop problem at each decision point in each work centre. Secondly, a framework of reinforcement learning algorithm using a double-layer deep Q-learning network is applied to select proper composite dispatching rules based on the status of each work centre. Then, based on the selected composite dispatching rule, an available operation is selected from the waiting buffer and assigned to an available machine in each work centre. Finally, the proposed algorithm will be compared with well-known dispatching rules on objectives of mean tardiness, mean flow time, mean waiting time, or mean percentage of waiting time in the real-time Flexible Job Shop problem. The result of the simulations proved that the proposed framework has reasonable performance and time efficiency.

Keywords: dynamic scheduling problem, flexible job shop, dispatching rules, deep reinforcement learning

Procedia PDF Downloads 100

Authors: Migena Mana, Ahmed Khalid Syed, Abdul Malik, Nikhil Cherian

Abstract:

In recent years, advancements in deep learning enabled researchers to tackle the problem of self-driving cars. Car companies use huge datasets to train their deep learning models to make autonomous cars a reality. However, this approach has certain drawbacks in that the state space of possible actions for a car is so huge that there cannot be a dataset for every possible road scenario. To overcome this problem, the concept of reinforcement learning (RL) is being investigated in this research. Since the problem of autonomous driving can be modeled in a simulation, it lends itself naturally to the domain of reinforcement learning. The advantage of this approach is that we can model different and complex road scenarios in a simulation without having to deploy in the real world. The autonomous agent can learn to drive by finding the optimal policy. This learned model can then be easily deployed in a real-world setting. In this project, we focus on three RL algorithms: Q-learning, Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO). To model the environment, we have used TORCS (The Open Racing Car Simulator), which provides us with a strong foundation to test our model. The inputs to the algorithms are the sensor data provided by the simulator such as velocity, distance from side pavement, etc. The outcome of this research project is a comparative analysis of these algorithms. Based on the comparison, the PPO algorithm gives the best results. When using PPO algorithm, the reward is greater, and the acceleration, steering angle and braking are more stable compared to the other algorithms, which means that the agent learns to drive in a better and more efficient way in this case. Additionally, we have come up with a dataset taken from the training of the agent with DDPG and PPO algorithms. It contains all the steps of the agent during one full training in the form: (all input values, acceleration, steering angle, break, loss, reward). This study can serve as a base for further complex road scenarios. Furthermore, it can be enlarged in the field of computer vision, using the images to find the best policy.

Keywords: autonomous driving, DDPG (deep deterministic policy gradient), PPO (proximal policy optimization), reinforcement learning

Procedia PDF Downloads 140

Authors: Francisco José Plácido da Cunha, Tassio Ferenzini Martins Sirqueira, Marx Leles Viana, Carlos José Pereira de Lucena

Abstract:

Open Multiagent Systems (MASs) are societies in which heterogeneous and independently designed entities (agents) work towards similar, or different ends. Software agents are autonomous and the diversity of interests among different members living in the same society is a fact. In order to deal with this autonomy, these open systems use mechanisms of social control (norms) to ensure a desirable social order. This paper considers the following types of norms: (i) obligation — agents must accomplish a specific outcome; (ii) permission — agents may act in a particular way, and (iii) prohibition — agents must not act in a specific way. All of these characteristics mean to encourage the fulfillment of norms through rewards and to discourage norm violation by pointing out the punishments. Once the software agent decides that its priority is the satisfaction of its own desires and goals, each agent must evaluate the effects associated to the fulfillment of one or more norms before choosing which one should be fulfilled. The same applies when agents decide to violate a norm. This paper also introduces a framework for the development of MASs that provide support mechanisms to the agent’s decision-making, using norm-based reasoning. The applicability and validation of this approach is demonstrated applying a traffic intersection scenario.

Keywords: BDI agent, BDI4JADE framework, multiagent systems, normative agents

Procedia PDF Downloads 228

Authors: Zheni S. Stefanova, Kandethody M. Ramachandran

Abstract:

With the increasing dependency on our computer devices, we face the necessity of adequate, efficient and effective mechanisms, for protecting our network. There are two main problems that Intrusion Detection Systems (IDS) attempt to solve. 1) To detect the attack, by analyzing the incoming traffic and inspect the network (intrusion detection). 2) To produce a prompt response when the attack occurs (intrusion prevention). It is critical creating an Intrusion detection model that will detect a breach in the system on time and also challenging making it provide an automatic and with an acceptable delay response at every single stage of the monitoring process. We cannot afford to adopt security measures with a high exploiting computational power, and we are not able to accept a mechanism that will react with a delay. In this paper, we will propose an intrusion response mechanism that is based on artificial intelligence, and more precisely, reinforcement learning techniques (RLT). The RLT will help us to create a decision agent, who will control the process of interacting with the undetermined environment. The goal is to find an optimal policy, which will represent the intrusion response, therefore, to solve the Reinforcement learning problem, using a Q-learning approach. Our agent will produce an optimal immediate response, in the process of evaluating the network traffic.This Q-learning approach will establish the balance between exploration and exploitation and provide a unique, self-learning and strategic artificial intelligence response mechanism for IDS.

Keywords: cyber security, intrusion prevention, optimal policy, Q-learning

Procedia PDF Downloads 232

Authors: C. H. S. Vidyasagar, D. B. Karunakar

Abstract:

In this present work, five different composite samples with AA2024 as matrix and varying amounts of yttrium (0.1-0.5 wt.%) as reinforcement are developed through cold compaction. The microstructures of the developed composite samples revealed that the yttrium reinforcement caused grain refinement up to 0.3 wt.% and beyond which the refinement is not effective. The microstructure revealed Al2Cu precipitation which strengthened the composite up to 0.3 wt.% yttrium reinforcement. Upon further increase in yttrium reinforcement, the intermetallics and the precipitation coarsen and their corresponding strengthening effect decreases. The mechanical characterization revealed that the composite sample reinforced with 0.3 wt.% yttrium showed highest mechanical properties like 82 HV of hardness, 276 MPa Ultimate Tensile Strength (UTS), 229 MPa Yield Strength (YS) and an elongation (EL) of 18.9% respectively. However, the relative density of the developed composites decreased with the increase in yttrium reinforcement.

Keywords: mechanical properties, AA 2024 matrix, yttrium reinforcement, cold compaction, precipitation

Procedia PDF Downloads 147

Authors: Lifar M. S., Tereshchenko A. A., Bulgakov A. N., Guda S. A., Guda A. A., Soldatov A. V.

Abstract:

Metal nanoparticles are widely used as heterogeneous catalysts to activate adsorbed molecules and reduce the energy barrier of the reaction. Reaction product yield depends on the interplay between elementary processes - adsorption, activation, reaction, and desorption. These processes, in turn, depend on the inlet feed concentrations, temperature, and pressure. At stationary conditions, the active surface sites may be poisoned by reaction byproducts or blocked by thermodynamically adsorbed gaseous reagents. Thus, the yield of reaction products can significantly drop. On the contrary, the dynamic control accounts for the changes in the surface properties and adjusts reaction parameters accordingly. Therefore dynamic control may be more efficient than stationary control. In this work, a reinforcement learning algorithm has been applied to control the simulation of CO oxidation on a catalyst. The policy gradient algorithm is learned to maximize the CO₂ production rate based on the CO and O₂ flows at a given time step. Nonstationary solutions were found for the regime with surface deactivation. The maximal product yield was achieved for periodic variations of the gas flows, ensuring a balance between available adsorption sites and the concentration of activated intermediates. This methodology opens a perspective for the optimization of catalytic reactions under nonstationary conditions.

Keywords: artificial intelligence, catalyst, co oxidation, reinforcement learning, dynamic control

Procedia PDF Downloads 123

Authors: Sahar El Kasmi, Ayoub Aziz, Saadia Lharti, Mohammed El Janati, Boubker Boukili, Nacer El Motawakil, Mayom Chol Luka Awan

Abstract:

Two types of geomaterials (Red Triassic clay from the Maaziz region and Yellow Pliocene clay from the Akrach region) were used to create different mixtures for the fabrication of ceramic bricks. This study investigated the influence of the Pliocene clay on the overall composition and mechanical properties of the Triassic clay. The red Triassic clay, sourced from Maaziz, underwent various mechanical processes and treatments to facilitate its transformation into ceramic bricks for construction. The triassic clay was subjected to a drying chamber and a heating chamber at 100°C to remove moisture. Subsequently, the dried clay samples were processed using a Planetary Babs ll Mill to reduce particle size and improve homogeneity. The resulting clay material was sieved, and the fine particles below 100 mm were collected for further analysis. In parallel, the Miocene marl obtained from the Akrach region was fragmented into finer particles and subjected to similar drying, grinding, and sieving procedures as the triassic clay. The two clay samples are then amalgamated and homogenized in different proportions. Precise measurements were taken using a weighing balance, and mixtures of 90%, 80%, and 70% Triassic clay with 10%, 20%, and 30% yellow clay were prepared, respectively. To evaluate the impact of Pliocene marl on the composition, the prepared clay mixtures were spread evenly and treated with a water modifier to enhance plasticity. The clay was then molded using a brick-making machine, and the initial manipulation process was observed. Additional batches were prepared with incremental amounts of Pliocene marl to further investigate its effect on the fracture behavior of the clay, specifically their resistance. The molded clay bricks were subjected to compression tests to measure their strength and resistance to deformation. Additional tests, such as water absorption tests, were also conducted to assess the overall performance of the ceramic bricks fabricated from the different clay mixtures. The results were analyzed to determine the influence of the Pliocene marl on the strength and durability of the Triassic clay bricks. The results indicated that the incorporation of Pliocene clay reduced the fracture of the triassic clay, with a noticeable reduction observed at 10% addition. No fractures were observed when 20% and 30% of yellow clay are added. These findings suggested that yellow clay can enhance the mechanical properties and structural integrity of red clay-based products.

Keywords: triassic clay, pliocene clay, mineralogical composition, geo-materials, ceramics, akach region, maaziz region, morocco.

Procedia PDF Downloads 78

Authors: Ahmed Negm, George Aggidis, Xiandong Ma

Abstract:

With the diverse nature of urban cities, customer demand patterns, landscape topologies or even seasonal weather trends; managing our water distribution networks (WDNs) has proved a complex task. These unpredictable circumstances manifest as pipe failures, intermittent supply and burst events thus adding to water loss, energy waste and increased carbon emissions. Whilst these events are unavoidable, advanced pressure management has proved an effective tool to control and mitigate them. Henceforth, water utilities have struggled with developing a real-time control method that is resilient when confronting the challenges of water distribution. In this paper we use deep reinforcement learning (DRL) algorithms as a novel pressure control strategy to minimise pressure violations and leakage under both burst and background leakage conditions. Agents based on asynchronous actor critic (A2C) and recurrent proximal policy optimisation (Recurrent PPO) were trained and compared to benchmarked optimisation algorithms (differential evolution, particle swarm optimisation. A2C manages to minimise leakage by 32.48% under burst conditions and 67.17% under background conditions which was the highest performance in the DRL algorithms. A2C and Recurrent PPO performed well in comparison to the benchmarks with higher processing speed and lower computational effort.

Keywords: deep reinforcement learning, pressure management, water distribution networks, leakage management

Procedia PDF Downloads 83

Authors: Rodrigo S. Piera, Jailson Sales Ara´ujo, Gabriela B. Lemos, Matthew B. Weiss, John B. DeBrota, Gabriel H. Aguilar, Jacques L. Pienaar

Abstract:

The Born rule was historically viewed as an independent axiom of quantum mechanics until Gleason derived it in 1957 by assuming the Hilbert space structure of quantum measurements [1]. In subsequent decades there have been diverse proposals to derive the Born rule starting from even more basic assumptions [2]. In this work, we demonstrate that a simple reinforcement-learning algorithm, having no pre-programmed assumptions about quantum theory, will nevertheless converge to a behaviour pattern that accords with the Born rule, when tasked with predicting the output of a quantum optical implementation of a symmetric informationally-complete measurement (SIC). Our findings support a hypothesis due to QBism (the subjective Bayesian approach to quantum theory), which states that the Born rule can be thought of as a normative rule for making decisions in a quantum world [3].

Keywords: quantum Bayesianism, quantum theory, quantum information, quantum measurement

Procedia PDF Downloads 102

Authors: Taylan Kabbani, Ekrem Duman

Abstract:

The design of adaptive systems that take advantage of financial markets while reducing the risk can bring more stagnant wealth into the global market. However, most efforts made to generate successful deals in trading financial assets rely on Supervised Learning (SL), which suffered from various limitations. Deep Reinforcement Learning (DRL) offers to solve these drawbacks of SL approaches by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with its environment to make optimal decisions through trial and error. In this paper, a continuous action space approach is adopted to give the trading agent the ability to gradually adjust the portfolio's positions with each time step (dynamically re-allocate investments), resulting in better agent-environment interaction and faster convergence of the learning process. In addition, the approach supports the managing of a portfolio with several assets instead of a single one. This work represents a novel DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate the trading problem, or what is referred to as The Agent Environment as Partially observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market, such as liquidity and transaction costs. More specifically, we design an environment that simulates the real-world trading process by augmenting the state representation with ten different technical indicators and sentiment analysis of news articles for each stock. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, which can learn policies in high-dimensional and continuous action spaces like those typically found in the stock market environment. From the point of view of stock market forecasting and the intelligent decision-making mechanism, this paper demonstrates the superiority of deep reinforcement learning in financial markets over other types of machine learning such as supervised learning and proves its credibility and advantages of strategic decision-making.

Keywords: the stock market, deep reinforcement learning, MDP, twin delayed deep deterministic policy gradient, sentiment analysis, technical indicators, autonomous agent

Procedia PDF Downloads 175

Authors: Siti Rossidatul Munawaroh, Siti Khusnul Khowatim

Abstract:

This study is determined to find how is the influence of family attention of students in provides guidance of the student learning. The increasing of student’s learning motivation can be increased made up in various ways, one of them are through students social guidance in their relation with the family. The family not only provides the matter and the learning time but also be supervise for the learning time and guide his children to overcome a learning disability. The character of physics subject in their science experiences at junior high schools has demanded that student’s ability is to think symbolically and understand something in a meaningful manner. Therefore, the reinforcement of the physics learning motivation is clearly necessary not only by the school are related, but the family environment and the society. As for the role of family which includes maintenance, parenting, coaching, and educating both of physically and spiritually, this way is expected to give spirit impulsion in studying physics subject in order to increase student learning achievements.

Keywords: physics subject, the influence of family attention, learning motivation, the Student care

Procedia PDF Downloads 427

Authors: Huan Ting Liao

Abstract:

In a typical manufacturing environment, computer numerical control (CNC) machining is essential for automating production through precise computer-controlled tool operations, significantly enhancing efficiency and ensuring consistent product quality. However, traditional CNC production lines often rely on manual loading and unloading, limiting operational efficiency and scalability. Although automated loading systems have been developed, they frequently lack sufficient intelligence and configuration efficiency, requiring extensive setup adjustments for different products and impacting overall productivity. This research addresses the job shop scheduling problem (JSSP) in CNC machining environments, aiming to minimize total completion time (makespan) and maximize CNC machine utilization. We propose a novel approach using reinforcement learning (RL), specifically the Q-learning algorithm, to optimize scheduling decisions. The study simulates the JSSP, incorporating robotic arm operations, machine processing times, and work order demand allocation to determine optimal processing sequences. The Q-learning algorithm enhances machine utilization by dynamically balancing workloads across CNC machines, adapting to varying job demands and machine states. This approach offers robust solutions for complex manufacturing environments by automating decision-making processes for job assignments. Additionally, we evaluate various layout configurations to identify the most efficient setup. By integrating RL-based scheduling optimization with layout analysis, this research aims to provide a comprehensive solution for improving manufacturing efficiency and productivity in CNC-based job shops. The proposed method's adaptability and automation potential promise significant advancements in tackling dynamic manufacturing challenges.

Keywords: job shop scheduling problem, reinforcement learning, operations sequence, layout optimization, q-learning

Procedia PDF Downloads 12

Authors: Hakan Ozturk, Aydin Demir, Kemal Edip, Marta Stojmanovska, Julijana Bojadjieva

Abstract:

There are many factors that affect the behavior of reinforced concrete beams. These can be listed as concrete compressive and reinforcement yield strength, amount of tension, compression and confinement bars, and strain hardening of reinforcement. In the study, support condition of short beams is selected statically indeterminate to first degree. Experimental and numerical analysis are carried for reinforcement concrete (RC) short beams. Dimensions of cross sections are selected as 250mm width and 500 mm height. The length of RC short beams is designed as 2250 mm and these values are constant in all beams. After verifying accurately finite element model, a numerical parametric study is performed with varied diameter of tension reinforcement. Effect of change in diameter is investigated on behavior of RC short beams. As a result of the study, ductility ratios and failure modes are determined, and load-displacement graphs are obtained in order to understand the behavior of short beams. It is deduced that diameter of tension reinforcement plays very important role on the behavior of RC short beams in terms of ductility and brittleness.

Keywords: short beam, reinforced concrete, finite element analysis, longitudinal reinforcement

Procedia PDF Downloads 206

Authors: Rahul Paul, Kedar Nath Das

Abstract:

The paradigm of Reinforcement Learning (RL) has become prominent in training intelligent agents to make decisions in environments that are both dynamic and uncertain. The primary objective of RL is to optimize the policy of an agent in order to maximize the cumulative reward it receives throughout a given period. Nevertheless, the process of optimization presents notable difficulties as a result of the inherent trade-off between exploration and exploitation, the presence of extensive state-action spaces, and the intricate nature of the dynamics involved. Evolutionary Optimization Methods (EOMs) have garnered considerable attention as a supplementary approach to tackle these challenges, providing distinct capabilities for optimizing RL policies and value functions. The ongoing advancement of research in both RL and EOMs presents an opportunity for significant advancements in autonomous decision-making systems. The convergence of these two fields has the potential to have a transformative impact on various domains of artificial intelligence (AI) applications. This article highlights the considerable influence of EOMs in enhancing the capabilities of RL. Taking advantage of evolutionary principles enables RL algorithms to effectively traverse extensive action spaces and discover optimal solutions within intricate environments. Moreover, this paper emphasizes the practical implementations of EOMs in the field of RL, specifically in areas such as robotic control, autonomous systems, inventory problems, and multi-agent scenarios. The article highlights the utilization of EOMs in facilitating RL agents to effectively adapt, evolve, and uncover proficient strategies for complex tasks that may pose challenges for conventional RL approaches.

Keywords: machine learning, reinforcement learning, loss function, optimization techniques, evolutionary optimization methods

Procedia PDF Downloads 78

Authors: A. Ardianto, M. A. Putera Agung, S. Pramusandi

Abstract:

Power waterway is one of dam structures and as an intake vertical tunnel or well function for hydroelectric power plants in Jatigede area, Sumedang, West Java. Gate shaft is also one of parts the power waterway system. The paper concerns some consideration in determining a critical state parameter on the back stability analysis of gate shaft or excavation wall stability during excavation. Study analysis was carried out using without and with reinforcement system. Results study showed that reinforcement shaft could reduce the total displacement and safety factor could increases significantly. Based on the back calculation results, it was recommended to install some reinforcement materials and drainage system to reduce pore water pressure.

Keywords: power waterway, reinforcement, displacement, safety

Procedia PDF Downloads 405

Authors: Faris A. Uriayer, Mehtab Alam

Abstract:

The concept of using a hybrid composite by combining two or more different materials to produce bilinear stress–strain behaviour has become a subject of interest. Having studied the mechanical properties of steel-CFRP specimens (CFRP Laminate Sandwiched between Mild Steel Strips), full size steel-CFRP composite reinforcement were fabricated and used as a new reinforcing material inside beams in lieu of traditional steel bars. Four beams, three beams reinforced with steel-CFRP composite reinforcement and one beam reinforced with traditional steel bars were cast, cured and tested under quasi-static loading. The flexural test results of the beams reinforced with this composite reinforcement showed that the beams with steel-CFRP composite reinforcement had comparable flexural strength and flexural ductility with beams reinforced with traditional steel bars.

Keywords: CFRP laminate, steel strip, flexural behaviour, modified model, concrete beam

Procedia PDF Downloads 681

Authors: Najmeh Abedzadeh, Matthew Jacobs

Abstract:

An intrusion detection system (IDS) is a software application that monitors malicious activities and generates alerts if any are detected. However, most network activities in IDS datasets are normal, and the relatively few numbers of attacks make the available data imbalanced. Consequently, cyber-attacks can hide inside a large number of normal activities, and machine learning algorithms have difficulty learning and classifying the data correctly. In this paper, a comprehensive literature review is conducted on different types of algorithms for both implementing the IDS and methods in correcting the imbalanced IDS dataset. The most famous algorithms are machine learning (ML), deep learning (DL), synthetic minority over-sampling technique (SMOTE), and reinforcement learning (RL). Most of the research use the CSE-CIC-IDS2017, CSE-CIC-IDS2018, and NSL-KDD datasets for evaluating their algorithms.

Keywords: IDS, imbalanced datasets, sampling algorithms, big data

Procedia PDF Downloads 319

Authors: S. K. Khosrowshahi, E. Güler

Abstract:

This study investigates the reinforcement function of geosynthetics on the shear strength and strain profile of sand. Conducting a series of simple shear tests, the shearing behavior of the samples under static and cyclic loads was evaluated. Three different types of geosynthetics including geotextile and geonets were used as the reinforcement materials. An image processing analysis based on the optical flow method was performed to measure the lateral displacements and estimate the shear strains. It is shown that besides improving the shear strength, the geosynthetic reinforcement leads a remarkable reduction on the shear strains. The improved layer reduces the required thickness of the soil layer to resist against shear stresses. Consequently, the geosynthetic reinforcement can be considered as a proper approach for the sustainable designs, especially in the projects with huge amount of geotechnical applications like subgrade of the pavements, roadways, and railways.

Keywords: image processing, soil reinforcement, geosynthetics, simple shear test, shear strain profile

Procedia PDF Downloads 214

Authors: Seyed-Ali Sadegh-Zadeh, Kaveh Kavianpour, Hamed Atashbar, Elham Heidari, Saeed Shiry Ghidary, Amir M. Hajiyavand

Abstract:

Evaluation metrics play a critical role in assessing the performance of machine learning models. In this review paper, we provide a comprehensive overview of performance measurement approaches for machine learning models. For each category, we discuss the most widely used metrics, including their mathematical formulations and interpretation. Additionally, we provide a comparative analysis of performance measurement approaches for metric combinations. Our review paper aims to provide researchers and practitioners with a better understanding of performance measurement approaches and to aid in the selection of appropriate evaluation metrics for their specific applications.

Keywords: evaluation metrics, performance measurement, supervised learning, unsupervised learning, reinforcement learning, model robustness and stability, comparative analysis

Procedia PDF Downloads 64

Authors: Guray Arslan, Riza Secer Orkun Keskin

Abstract:

Numerical investigations were conducted to study the influence of flexural reinforcement ratio on the diagonal cracking strength and ultimate shear strength of reinforced concrete (RC) beams without stirrups. Three-dimensional nonlinear finite element analyses (FEAs) of the beams with flexural reinforcement ratios ranging from 0.58% to 2.20% subjected to a mid-span concentrated load were carried out. It is observed that the load-deflection and load-strain curves obtained from the numerical analyses agree with those obtained from the experiments. It is concluded that flexural reinforcement ratio has a significant effect on the shear strength and deflection capacity of RC beams without stirrups. The predictions of the diagonal cracking strength and ultimate shear strength of beams obtained by using the equations defined by a number of codes and researchers are compared with each other and with the experimental values.

Keywords: finite element, flexural reinforcement, reinforced concrete beam, shear strength

Procedia PDF Downloads 326

Authors: Adel M. Belal, Sameh Abu El-Soud, Mariam Farid

Abstract:

This study presents the effect of reinforcement inclusions (fibers-geogrids) on fine sand bearing capacity under strip footings. Experimental model tests were carried out using a rectangular plates [(10cm x 38 cm), (7.5 cm x 38 cm), and (12.5 cm x 38 cm)] with a geogrids and randomly reinforced fibers. The width and depth of the geogrid were varied to determine their effects on the engineering properties of treated poorly graded fine sand. Laboratory model test results for the ultimate stresses and the settlement of a rigid strip foundation supported by single and multi-layered fiber-geogrid-reinforced sand are presented. The number of layers of geogrid was varied between 1 to 4. The effect of the first geogrid reinforcement depth, the spacing between the reinforcement and its length on the bearing capacity is investigated by experimental program. Results show that the use of flexible random fibers with a content of 0.125% by weight of the treated sand dunes, with 3 geogrid reinforcement layers, u/B= 0.25 and L/B=7.5, has a significant increase in the bearing capacity of the proposed system.

Keywords: earth reinforcement, geogrid, random fiber, reinforced soil

Procedia PDF Downloads 310

Authors: Can Otuzbir

Abstract:

It is one of the most difficult areas of civil engineering to provide long-lasting structures with the rapid development of concrete and reinforced concrete structures. Concrete is a living material, and the structure where the concrete is located is constantly exposed to external influences. One of these effects is reinforcement corrosion. Reinforcement corrosion of reinforced concrete structures leads to a significant decrease in the carrying capacity of the structural elements, as well as reduced service life. It is undesirable that the service life should be completed sooner than expected. In recent years, advances in glass fiber technology and its use with concrete have developed rapidly. As a result of inability to protect steel reinforcements against corrosion, fiberglass reinforcements have started to be investigated as an alternative material to steel reinforcements, and researches and experimental studies are still continuing. Glass fiber reinforcements have become an alternative material to steel reinforcement because they are resistant to corrosion, lightweight and simple to install compared to steel reinforcement. Glass fiber reinforcements are not corroded and have higher tensile strength, longer life, lighter and insulating properties compared to steel reinforcement. In experimental studies, glass fiber reinforcements have been shown to show superior mechanical properties similar to beams produced with steel reinforcement. The performance of long-term use of glass fiber fibers continues with accelerated experimental studies.

Keywords: glass fiber polymer reinforcement, steel fiber concrete, ultra high performance concrete, bending, GFRP

Procedia PDF Downloads 124

Authors: Ziying Wu, Danfeng Yan

Abstract:

Multi-Access Edge Computing (MEC) is one of the key technologies of the future 5G network. By deploying edge computing centers at the edge of wireless access network, the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios. Meanwhile, with the development of IOV (Internet of Vehicles) technology, various delay-sensitive and compute-intensive in-vehicle applications continue to appear. Compared with traditional internet business, these computation tasks have higher processing priority and lower delay requirements. In this paper, we design a 5G-based Vehicle-Aware Multi-Access Edge Computing Network (VAMECN) and propose a joint optimization problem of minimizing total system cost. In view of the problem, a deep reinforcement learning-based joint computation offloading and task migration optimization (JCOTM) algorithm is proposed, considering the influences of multiple factors such as concurrent multiple computation tasks, system computing resources distribution, and network communication bandwidth. And, the mixed integer nonlinear programming problem is described as a Markov Decision Process. Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption, optimize computing offloading and resource allocation schemes, and improve system resource utilization, compared with other computing offloading policies.

Keywords: multi-access edge computing, computation offloading, 5th generation, vehicle-aware, deep reinforcement learning, deep q-network

Procedia PDF Downloads 112

Authors: D. Moser, D. Pinto, A. Cipriano

Abstract:

A Disaster Management System (DMS) for countries with different disasters is very important. In the world different disasters like earthquakes, tsunamis, volcanic eruption, fire or other natural or man-made disasters occurs and have an effect on the population. It is also possible that two or more disasters arisen at the same time, this means to handle multi-risk situations. To handle such a situation a Decision Support System (DSS) based on multiagents is a suitable architecture. The most known DMSs deal with one (in the case of an earthquake-tsunami combination with two) disaster and often with one particular disaster. Nevertheless, a DSS helps for a better realtime response. Analyze the existing systems in the literature and expand them for multi-risk disasters to construct a well-organized system is the proposal of our work. The here shown work is an approach of a multi-risk system, which needs an architecture, and well-defined aims. In this moment our study is a kind of case study to analyze the way we have to follow to create our proposed system in the future.

Keywords: decision support system, disaster management system, multi-risk, multiagent system

Procedia PDF Downloads 421

Authors: SKhan Park, Seok Jin Lee, Jong Sun Kim, Jun Ho Lee, Bong Chan Kim

Abstract:

Because of the large economic losses caused by traffic congestion in metropolitan areas, various studies on the underground network design and construction techniques has been performed various studies in the developed countries. In Korea, it has performed a study to develop a versatile double-deck of deep tunnel model. This paper is an introduction to develop a ground reinforcement method to enable the safe tunnel construction in the weakened pillar section like as junction of tunnel. Applicable ground reinforcement method in the weakened section is proposed and it is expected to verify the method by the field application tests.

Keywords: double-deck tunnel, ground reinforcement, tunnel construction, weakened pillar section

Procedia PDF Downloads 404

Authors: S. N. Moghaddas Tafreshi, S. M. Amin Ghotbi

Abstract:

This paper presents a numerical study for investigating the effectiveness of geocell reinforcement in reducing pressure and settlement over EPS geofoam blocks in road embankments. A 3-D FEM model of soil and geofoam was created in ABAQUS, and geocell was also modeled realistically using membrane elements. The accuracy of the model was tested by comparing its results with previous works. Sensitivity analyses showed that reinforcing the soil cover with geocell has a significant influence on the reduction of imposed stresses over geofoam and consequently decreasing its deformation.

Keywords: EPS geofoam, geocell, reinforcement, road embankments, lightweight fill

Procedia PDF Downloads 266

Authors: Gopika Rajagopal, Sudheesh Thiyyakkandi

Abstract:

Back-to-back reinforced earth (RE) walls are extensively used in these days as bridge abutments and highway ramps, owing to their cost efficiency and ease of construction. High quality select fill is the most suitable backfill material due to its excellent engineering properties and constructability. However, industries are compelled to use low quality, locally available soil because of its ample availability on site. However, several failure cases of such walls are reported, especially subsequent to rainfall events. The stiffness of reinforcement is one of the major factors affecting the performance of RE walls. The present study focused on analyzing the effect of reinforcement stiffness on the performance of complete select fill, complete marginal fill, and hybrid-fill (i.e., combination of select and marginal fills) back-to-back RE walls, immediately after construction and upon rainwater infiltration through finite element modelling. A constant width to height (W/H) ratio of 3 and height (H) of 6 m was considered for the numerical analysis and the stiffness of reinforcement layers was varied from 500 kN/m to 10000 kN/m. Results showed that reinforcement stiffness had a noticeable influence on the response of RE wall, subsequent to construction as well as rainwater infiltration. Facing displacement was found to decrease and maximum reinforcement tension and factor of safety were observed to increase with increasing the stiffness of reinforcement. However, beyond a stiffness of 5000 kN/m, no significant reduction in facing displacement was observed. The behavior of fully marginal fill wall considered in this study was found to be reasonable even after rainwater infiltration when the high stiffness reinforcement layers are used.

Keywords: back-to-back reinforced earth wall, finite element modelling, rainwater infiltration, reinforcement stiffness

Procedia PDF Downloads 151