Search results for: batch-constrained reinforcement learning

Authors: Migena Mana, Ahmed Khalid Syed, Abdul Malik, Nikhil Cherian

Abstract:

In recent years, advancements in deep learning enabled researchers to tackle the problem of self-driving cars. Car companies use huge datasets to train their deep learning models to make autonomous cars a reality. However, this approach has certain drawbacks in that the state space of possible actions for a car is so huge that there cannot be a dataset for every possible road scenario. To overcome this problem, the concept of reinforcement learning (RL) is being investigated in this research. Since the problem of autonomous driving can be modeled in a simulation, it lends itself naturally to the domain of reinforcement learning. The advantage of this approach is that we can model different and complex road scenarios in a simulation without having to deploy in the real world. The autonomous agent can learn to drive by finding the optimal policy. This learned model can then be easily deployed in a real-world setting. In this project, we focus on three RL algorithms: Q-learning, Deep Deterministic Policy Gradient (DDPG), and Proximal Policy Optimization (PPO). To model the environment, we have used TORCS (The Open Racing Car Simulator), which provides us with a strong foundation to test our model. The inputs to the algorithms are the sensor data provided by the simulator such as velocity, distance from side pavement, etc. The outcome of this research project is a comparative analysis of these algorithms. Based on the comparison, the PPO algorithm gives the best results. When using PPO algorithm, the reward is greater, and the acceleration, steering angle and braking are more stable compared to the other algorithms, which means that the agent learns to drive in a better and more efficient way in this case. Additionally, we have come up with a dataset taken from the training of the agent with DDPG and PPO algorithms. It contains all the steps of the agent during one full training in the form: (all input values, acceleration, steering angle, break, loss, reward). This study can serve as a base for further complex road scenarios. Furthermore, it can be enlarged in the field of computer vision, using the images to find the best policy.

Keywords: autonomous driving, DDPG (deep deterministic policy gradient), PPO (proximal policy optimization), reinforcement learning

Procedia PDF Downloads 147

Authors: C. H. S. Vidyasagar, D. B. Karunakar

Abstract:

In this present work, five different composite samples with AA2024 as matrix and varying amounts of yttrium (0.1-0.5 wt.%) as reinforcement are developed through cold compaction. The microstructures of the developed composite samples revealed that the yttrium reinforcement caused grain refinement up to 0.3 wt.% and beyond which the refinement is not effective. The microstructure revealed Al2Cu precipitation which strengthened the composite up to 0.3 wt.% yttrium reinforcement. Upon further increase in yttrium reinforcement, the intermetallics and the precipitation coarsen and their corresponding strengthening effect decreases. The mechanical characterization revealed that the composite sample reinforced with 0.3 wt.% yttrium showed highest mechanical properties like 82 HV of hardness, 276 MPa Ultimate Tensile Strength (UTS), 229 MPa Yield Strength (YS) and an elongation (EL) of 18.9% respectively. However, the relative density of the developed composites decreased with the increase in yttrium reinforcement.

Keywords: mechanical properties, AA 2024 matrix, yttrium reinforcement, cold compaction, precipitation

Procedia PDF Downloads 151

Authors: Zheni S. Stefanova, Kandethody M. Ramachandran

Abstract:

With the increasing dependency on our computer devices, we face the necessity of adequate, efficient and effective mechanisms, for protecting our network. There are two main problems that Intrusion Detection Systems (IDS) attempt to solve. 1) To detect the attack, by analyzing the incoming traffic and inspect the network (intrusion detection). 2) To produce a prompt response when the attack occurs (intrusion prevention). It is critical creating an Intrusion detection model that will detect a breach in the system on time and also challenging making it provide an automatic and with an acceptable delay response at every single stage of the monitoring process. We cannot afford to adopt security measures with a high exploiting computational power, and we are not able to accept a mechanism that will react with a delay. In this paper, we will propose an intrusion response mechanism that is based on artificial intelligence, and more precisely, reinforcement learning techniques (RLT). The RLT will help us to create a decision agent, who will control the process of interacting with the undetermined environment. The goal is to find an optimal policy, which will represent the intrusion response, therefore, to solve the Reinforcement learning problem, using a Q-learning approach. Our agent will produce an optimal immediate response, in the process of evaluating the network traffic.This Q-learning approach will establish the balance between exploration and exploitation and provide a unique, self-learning and strategic artificial intelligence response mechanism for IDS.

Keywords: cyber security, intrusion prevention, optimal policy, Q-learning

Procedia PDF Downloads 236

Authors: Lifar M. S., Tereshchenko A. A., Bulgakov A. N., Guda S. A., Guda A. A., Soldatov A. V.

Abstract:

Metal nanoparticles are widely used as heterogeneous catalysts to activate adsorbed molecules and reduce the energy barrier of the reaction. Reaction product yield depends on the interplay between elementary processes - adsorption, activation, reaction, and desorption. These processes, in turn, depend on the inlet feed concentrations, temperature, and pressure. At stationary conditions, the active surface sites may be poisoned by reaction byproducts or blocked by thermodynamically adsorbed gaseous reagents. Thus, the yield of reaction products can significantly drop. On the contrary, the dynamic control accounts for the changes in the surface properties and adjusts reaction parameters accordingly. Therefore dynamic control may be more efficient than stationary control. In this work, a reinforcement learning algorithm has been applied to control the simulation of CO oxidation on a catalyst. The policy gradient algorithm is learned to maximize the CO₂ production rate based on the CO and O₂ flows at a given time step. Nonstationary solutions were found for the regime with surface deactivation. The maximal product yield was achieved for periodic variations of the gas flows, ensuring a balance between available adsorption sites and the concentration of activated intermediates. This methodology opens a perspective for the optimization of catalytic reactions under nonstationary conditions.

Keywords: artificial intelligence, catalyst, co oxidation, reinforcement learning, dynamic control

Procedia PDF Downloads 129

Authors: Ahmed Negm, George Aggidis, Xiandong Ma

Abstract:

With the diverse nature of urban cities, customer demand patterns, landscape topologies or even seasonal weather trends; managing our water distribution networks (WDNs) has proved a complex task. These unpredictable circumstances manifest as pipe failures, intermittent supply and burst events thus adding to water loss, energy waste and increased carbon emissions. Whilst these events are unavoidable, advanced pressure management has proved an effective tool to control and mitigate them. Henceforth, water utilities have struggled with developing a real-time control method that is resilient when confronting the challenges of water distribution. In this paper we use deep reinforcement learning (DRL) algorithms as a novel pressure control strategy to minimise pressure violations and leakage under both burst and background leakage conditions. Agents based on asynchronous actor critic (A2C) and recurrent proximal policy optimisation (Recurrent PPO) were trained and compared to benchmarked optimisation algorithms (differential evolution, particle swarm optimisation. A2C manages to minimise leakage by 32.48% under burst conditions and 67.17% under background conditions which was the highest performance in the DRL algorithms. A2C and Recurrent PPO performed well in comparison to the benchmarks with higher processing speed and lower computational effort.

Keywords: deep reinforcement learning, pressure management, water distribution networks, leakage management

Procedia PDF Downloads 91

Authors: Rodrigo S. Piera, Jailson Sales Ara´ujo, Gabriela B. Lemos, Matthew B. Weiss, John B. DeBrota, Gabriel H. Aguilar, Jacques L. Pienaar

Abstract:

The Born rule was historically viewed as an independent axiom of quantum mechanics until Gleason derived it in 1957 by assuming the Hilbert space structure of quantum measurements [1]. In subsequent decades there have been diverse proposals to derive the Born rule starting from even more basic assumptions [2]. In this work, we demonstrate that a simple reinforcement-learning algorithm, having no pre-programmed assumptions about quantum theory, will nevertheless converge to a behaviour pattern that accords with the Born rule, when tasked with predicting the output of a quantum optical implementation of a symmetric informationally-complete measurement (SIC). Our findings support a hypothesis due to QBism (the subjective Bayesian approach to quantum theory), which states that the Born rule can be thought of as a normative rule for making decisions in a quantum world [3].

Keywords: quantum Bayesianism, quantum theory, quantum information, quantum measurement

Procedia PDF Downloads 108

Authors: Taylan Kabbani, Ekrem Duman

Abstract:

The design of adaptive systems that take advantage of financial markets while reducing the risk can bring more stagnant wealth into the global market. However, most efforts made to generate successful deals in trading financial assets rely on Supervised Learning (SL), which suffered from various limitations. Deep Reinforcement Learning (DRL) offers to solve these drawbacks of SL approaches by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with its environment to make optimal decisions through trial and error. In this paper, a continuous action space approach is adopted to give the trading agent the ability to gradually adjust the portfolio's positions with each time step (dynamically re-allocate investments), resulting in better agent-environment interaction and faster convergence of the learning process. In addition, the approach supports the managing of a portfolio with several assets instead of a single one. This work represents a novel DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate the trading problem, or what is referred to as The Agent Environment as Partially observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market, such as liquidity and transaction costs. More specifically, we design an environment that simulates the real-world trading process by augmenting the state representation with ten different technical indicators and sentiment analysis of news articles for each stock. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, which can learn policies in high-dimensional and continuous action spaces like those typically found in the stock market environment. From the point of view of stock market forecasting and the intelligent decision-making mechanism, this paper demonstrates the superiority of deep reinforcement learning in financial markets over other types of machine learning such as supervised learning and proves its credibility and advantages of strategic decision-making.

Keywords: the stock market, deep reinforcement learning, MDP, twin delayed deep deterministic policy gradient, sentiment analysis, technical indicators, autonomous agent

Procedia PDF Downloads 178

Authors: Siti Rossidatul Munawaroh, Siti Khusnul Khowatim

Abstract:

This study is determined to find how is the influence of family attention of students in provides guidance of the student learning. The increasing of student’s learning motivation can be increased made up in various ways, one of them are through students social guidance in their relation with the family. The family not only provides the matter and the learning time but also be supervise for the learning time and guide his children to overcome a learning disability. The character of physics subject in their science experiences at junior high schools has demanded that student’s ability is to think symbolically and understand something in a meaningful manner. Therefore, the reinforcement of the physics learning motivation is clearly necessary not only by the school are related, but the family environment and the society. As for the role of family which includes maintenance, parenting, coaching, and educating both of physically and spiritually, this way is expected to give spirit impulsion in studying physics subject in order to increase student learning achievements.

Keywords: physics subject, the influence of family attention, learning motivation, the Student care

Procedia PDF Downloads 430

Authors: Huan Ting Liao

Abstract:

In a typical manufacturing environment, computer numerical control (CNC) machining is essential for automating production through precise computer-controlled tool operations, significantly enhancing efficiency and ensuring consistent product quality. However, traditional CNC production lines often rely on manual loading and unloading, limiting operational efficiency and scalability. Although automated loading systems have been developed, they frequently lack sufficient intelligence and configuration efficiency, requiring extensive setup adjustments for different products and impacting overall productivity. This research addresses the job shop scheduling problem (JSSP) in CNC machining environments, aiming to minimize total completion time (makespan) and maximize CNC machine utilization. We propose a novel approach using reinforcement learning (RL), specifically the Q-learning algorithm, to optimize scheduling decisions. The study simulates the JSSP, incorporating robotic arm operations, machine processing times, and work order demand allocation to determine optimal processing sequences. The Q-learning algorithm enhances machine utilization by dynamically balancing workloads across CNC machines, adapting to varying job demands and machine states. This approach offers robust solutions for complex manufacturing environments by automating decision-making processes for job assignments. Additionally, we evaluate various layout configurations to identify the most efficient setup. By integrating RL-based scheduling optimization with layout analysis, this research aims to provide a comprehensive solution for improving manufacturing efficiency and productivity in CNC-based job shops. The proposed method's adaptability and automation potential promise significant advancements in tackling dynamic manufacturing challenges.

Keywords: job shop scheduling problem, reinforcement learning, operations sequence, layout optimization, q-learning

Procedia PDF Downloads 24

Authors: Hakan Ozturk, Aydin Demir, Kemal Edip, Marta Stojmanovska, Julijana Bojadjieva

Abstract:

There are many factors that affect the behavior of reinforced concrete beams. These can be listed as concrete compressive and reinforcement yield strength, amount of tension, compression and confinement bars, and strain hardening of reinforcement. In the study, support condition of short beams is selected statically indeterminate to first degree. Experimental and numerical analysis are carried for reinforcement concrete (RC) short beams. Dimensions of cross sections are selected as 250mm width and 500 mm height. The length of RC short beams is designed as 2250 mm and these values are constant in all beams. After verifying accurately finite element model, a numerical parametric study is performed with varied diameter of tension reinforcement. Effect of change in diameter is investigated on behavior of RC short beams. As a result of the study, ductility ratios and failure modes are determined, and load-displacement graphs are obtained in order to understand the behavior of short beams. It is deduced that diameter of tension reinforcement plays very important role on the behavior of RC short beams in terms of ductility and brittleness.

Keywords: short beam, reinforced concrete, finite element analysis, longitudinal reinforcement

Procedia PDF Downloads 210

Authors: Rahul Paul, Kedar Nath Das

Abstract:

The paradigm of Reinforcement Learning (RL) has become prominent in training intelligent agents to make decisions in environments that are both dynamic and uncertain. The primary objective of RL is to optimize the policy of an agent in order to maximize the cumulative reward it receives throughout a given period. Nevertheless, the process of optimization presents notable difficulties as a result of the inherent trade-off between exploration and exploitation, the presence of extensive state-action spaces, and the intricate nature of the dynamics involved. Evolutionary Optimization Methods (EOMs) have garnered considerable attention as a supplementary approach to tackle these challenges, providing distinct capabilities for optimizing RL policies and value functions. The ongoing advancement of research in both RL and EOMs presents an opportunity for significant advancements in autonomous decision-making systems. The convergence of these two fields has the potential to have a transformative impact on various domains of artificial intelligence (AI) applications. This article highlights the considerable influence of EOMs in enhancing the capabilities of RL. Taking advantage of evolutionary principles enables RL algorithms to effectively traverse extensive action spaces and discover optimal solutions within intricate environments. Moreover, this paper emphasizes the practical implementations of EOMs in the field of RL, specifically in areas such as robotic control, autonomous systems, inventory problems, and multi-agent scenarios. The article highlights the utilization of EOMs in facilitating RL agents to effectively adapt, evolve, and uncover proficient strategies for complex tasks that may pose challenges for conventional RL approaches.

Keywords: machine learning, reinforcement learning, loss function, optimization techniques, evolutionary optimization methods

Procedia PDF Downloads 80

Authors: A. Ardianto, M. A. Putera Agung, S. Pramusandi

Abstract:

Power waterway is one of dam structures and as an intake vertical tunnel or well function for hydroelectric power plants in Jatigede area, Sumedang, West Java. Gate shaft is also one of parts the power waterway system. The paper concerns some consideration in determining a critical state parameter on the back stability analysis of gate shaft or excavation wall stability during excavation. Study analysis was carried out using without and with reinforcement system. Results study showed that reinforcement shaft could reduce the total displacement and safety factor could increases significantly. Based on the back calculation results, it was recommended to install some reinforcement materials and drainage system to reduce pore water pressure.

Keywords: power waterway, reinforcement, displacement, safety

Procedia PDF Downloads 410

Authors: Faris A. Uriayer, Mehtab Alam

Abstract:

The concept of using a hybrid composite by combining two or more different materials to produce bilinear stress–strain behaviour has become a subject of interest. Having studied the mechanical properties of steel-CFRP specimens (CFRP Laminate Sandwiched between Mild Steel Strips), full size steel-CFRP composite reinforcement were fabricated and used as a new reinforcing material inside beams in lieu of traditional steel bars. Four beams, three beams reinforced with steel-CFRP composite reinforcement and one beam reinforced with traditional steel bars were cast, cured and tested under quasi-static loading. The flexural test results of the beams reinforced with this composite reinforcement showed that the beams with steel-CFRP composite reinforcement had comparable flexural strength and flexural ductility with beams reinforced with traditional steel bars.

Keywords: CFRP laminate, steel strip, flexural behaviour, modified model, concrete beam

Procedia PDF Downloads 689

Authors: Najmeh Abedzadeh, Matthew Jacobs

Abstract:

An intrusion detection system (IDS) is a software application that monitors malicious activities and generates alerts if any are detected. However, most network activities in IDS datasets are normal, and the relatively few numbers of attacks make the available data imbalanced. Consequently, cyber-attacks can hide inside a large number of normal activities, and machine learning algorithms have difficulty learning and classifying the data correctly. In this paper, a comprehensive literature review is conducted on different types of algorithms for both implementing the IDS and methods in correcting the imbalanced IDS dataset. The most famous algorithms are machine learning (ML), deep learning (DL), synthetic minority over-sampling technique (SMOTE), and reinforcement learning (RL). Most of the research use the CSE-CIC-IDS2017, CSE-CIC-IDS2018, and NSL-KDD datasets for evaluating their algorithms.

Keywords: IDS, imbalanced datasets, sampling algorithms, big data

Procedia PDF Downloads 327

Authors: S. K. Khosrowshahi, E. Güler

Abstract:

This study investigates the reinforcement function of geosynthetics on the shear strength and strain profile of sand. Conducting a series of simple shear tests, the shearing behavior of the samples under static and cyclic loads was evaluated. Three different types of geosynthetics including geotextile and geonets were used as the reinforcement materials. An image processing analysis based on the optical flow method was performed to measure the lateral displacements and estimate the shear strains. It is shown that besides improving the shear strength, the geosynthetic reinforcement leads a remarkable reduction on the shear strains. The improved layer reduces the required thickness of the soil layer to resist against shear stresses. Consequently, the geosynthetic reinforcement can be considered as a proper approach for the sustainable designs, especially in the projects with huge amount of geotechnical applications like subgrade of the pavements, roadways, and railways.

Keywords: image processing, soil reinforcement, geosynthetics, simple shear test, shear strain profile

Procedia PDF Downloads 219

Authors: Seyed-Ali Sadegh-Zadeh, Kaveh Kavianpour, Hamed Atashbar, Elham Heidari, Saeed Shiry Ghidary, Amir M. Hajiyavand

Abstract:

Evaluation metrics play a critical role in assessing the performance of machine learning models. In this review paper, we provide a comprehensive overview of performance measurement approaches for machine learning models. For each category, we discuss the most widely used metrics, including their mathematical formulations and interpretation. Additionally, we provide a comparative analysis of performance measurement approaches for metric combinations. Our review paper aims to provide researchers and practitioners with a better understanding of performance measurement approaches and to aid in the selection of appropriate evaluation metrics for their specific applications.

Keywords: evaluation metrics, performance measurement, supervised learning, unsupervised learning, reinforcement learning, model robustness and stability, comparative analysis

Procedia PDF Downloads 73

Authors: Xueming Xu, Jiahao Liu, Jichao Li, Kewei Yang, Minghao Li, Bingfeng Ge

Abstract:

A combat system-of-systems (CSoS) comprises various types of functional combat entities that interact to meet corresponding task requirements in the present and future. Enhancing the resilience of CSoS holds significant military value in optimizing the operational planning process, improving military survivability, and ensuring the successful completion of operational tasks. Accordingly, this research proposes an integrated framework called CSoS resilience enhancement (CSoSRE) to enhance the resilience of CSoS from a recovery perspective. Specifically, this research presents a two-phase resilience optimization model to define a resilience optimization objective for CSoS. This model considers not only task baseline, recovery cost, and recovery time limit but also the characteristics of emergency recovery and comprehensive recovery. Moreover, the research extends it from the deterministic case to the stochastic case to describe the uncertainty in the recovery process. Based on this, a resilience-oriented recovery optimization method based on deep reinforcement learning (RRODRL) is proposed to determine a set of entities requiring restoration and their recovery sequence, thereby enhancing the resilience of CSoS. This method improves the deep Q-learning algorithm by designing a discount factor that adapts to changes in CSoS state at different phases, simultaneously considering the network’s structural and functional characteristics within CSoS. Finally, extensive experiments are conducted to test the feasibility, effectiveness and superiority of the proposed framework. The obtained results offer useful insights for guiding operational recovery activity and designing a more resilient CSoS.

Keywords: combat system-of-systems, resilience optimization model, recovery optimization method, deep reinforcement learning, certainty and uncertainty

Procedia PDF Downloads 16

Authors: Guray Arslan, Riza Secer Orkun Keskin

Abstract:

Numerical investigations were conducted to study the influence of flexural reinforcement ratio on the diagonal cracking strength and ultimate shear strength of reinforced concrete (RC) beams without stirrups. Three-dimensional nonlinear finite element analyses (FEAs) of the beams with flexural reinforcement ratios ranging from 0.58% to 2.20% subjected to a mid-span concentrated load were carried out. It is observed that the load-deflection and load-strain curves obtained from the numerical analyses agree with those obtained from the experiments. It is concluded that flexural reinforcement ratio has a significant effect on the shear strength and deflection capacity of RC beams without stirrups. The predictions of the diagonal cracking strength and ultimate shear strength of beams obtained by using the equations defined by a number of codes and researchers are compared with each other and with the experimental values.

Keywords: finite element, flexural reinforcement, reinforced concrete beam, shear strength

Procedia PDF Downloads 329

Authors: Adel M. Belal, Sameh Abu El-Soud, Mariam Farid

Abstract:

This study presents the effect of reinforcement inclusions (fibers-geogrids) on fine sand bearing capacity under strip footings. Experimental model tests were carried out using a rectangular plates [(10cm x 38 cm), (7.5 cm x 38 cm), and (12.5 cm x 38 cm)] with a geogrids and randomly reinforced fibers. The width and depth of the geogrid were varied to determine their effects on the engineering properties of treated poorly graded fine sand. Laboratory model test results for the ultimate stresses and the settlement of a rigid strip foundation supported by single and multi-layered fiber-geogrid-reinforced sand are presented. The number of layers of geogrid was varied between 1 to 4. The effect of the first geogrid reinforcement depth, the spacing between the reinforcement and its length on the bearing capacity is investigated by experimental program. Results show that the use of flexible random fibers with a content of 0.125% by weight of the treated sand dunes, with 3 geogrid reinforcement layers, u/B= 0.25 and L/B=7.5, has a significant increase in the bearing capacity of the proposed system.

Keywords: earth reinforcement, geogrid, random fiber, reinforced soil

Procedia PDF Downloads 312

Authors: Can Otuzbir

Abstract:

It is one of the most difficult areas of civil engineering to provide long-lasting structures with the rapid development of concrete and reinforced concrete structures. Concrete is a living material, and the structure where the concrete is located is constantly exposed to external influences. One of these effects is reinforcement corrosion. Reinforcement corrosion of reinforced concrete structures leads to a significant decrease in the carrying capacity of the structural elements, as well as reduced service life. It is undesirable that the service life should be completed sooner than expected. In recent years, advances in glass fiber technology and its use with concrete have developed rapidly. As a result of inability to protect steel reinforcements against corrosion, fiberglass reinforcements have started to be investigated as an alternative material to steel reinforcements, and researches and experimental studies are still continuing. Glass fiber reinforcements have become an alternative material to steel reinforcement because they are resistant to corrosion, lightweight and simple to install compared to steel reinforcement. Glass fiber reinforcements are not corroded and have higher tensile strength, longer life, lighter and insulating properties compared to steel reinforcement. In experimental studies, glass fiber reinforcements have been shown to show superior mechanical properties similar to beams produced with steel reinforcement. The performance of long-term use of glass fiber fibers continues with accelerated experimental studies.

Keywords: glass fiber polymer reinforcement, steel fiber concrete, ultra high performance concrete, bending, GFRP

Procedia PDF Downloads 128

Authors: Ziying Wu, Danfeng Yan

Abstract:

Multi-Access Edge Computing (MEC) is one of the key technologies of the future 5G network. By deploying edge computing centers at the edge of wireless access network, the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios. Meanwhile, with the development of IOV (Internet of Vehicles) technology, various delay-sensitive and compute-intensive in-vehicle applications continue to appear. Compared with traditional internet business, these computation tasks have higher processing priority and lower delay requirements. In this paper, we design a 5G-based Vehicle-Aware Multi-Access Edge Computing Network (VAMECN) and propose a joint optimization problem of minimizing total system cost. In view of the problem, a deep reinforcement learning-based joint computation offloading and task migration optimization (JCOTM) algorithm is proposed, considering the influences of multiple factors such as concurrent multiple computation tasks, system computing resources distribution, and network communication bandwidth. And, the mixed integer nonlinear programming problem is described as a Markov Decision Process. Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption, optimize computing offloading and resource allocation schemes, and improve system resource utilization, compared with other computing offloading policies.

Keywords: multi-access edge computing, computation offloading, 5th generation, vehicle-aware, deep reinforcement learning, deep q-network

Procedia PDF Downloads 117

Authors: SKhan Park, Seok Jin Lee, Jong Sun Kim, Jun Ho Lee, Bong Chan Kim

Abstract:

Because of the large economic losses caused by traffic congestion in metropolitan areas, various studies on the underground network design and construction techniques has been performed various studies in the developed countries. In Korea, it has performed a study to develop a versatile double-deck of deep tunnel model. This paper is an introduction to develop a ground reinforcement method to enable the safe tunnel construction in the weakened pillar section like as junction of tunnel. Applicable ground reinforcement method in the weakened section is proposed and it is expected to verify the method by the field application tests.

Keywords: double-deck tunnel, ground reinforcement, tunnel construction, weakened pillar section

Procedia PDF Downloads 408

Authors: S. N. Moghaddas Tafreshi, S. M. Amin Ghotbi

Abstract:

This paper presents a numerical study for investigating the effectiveness of geocell reinforcement in reducing pressure and settlement over EPS geofoam blocks in road embankments. A 3-D FEM model of soil and geofoam was created in ABAQUS, and geocell was also modeled realistically using membrane elements. The accuracy of the model was tested by comparing its results with previous works. Sensitivity analyses showed that reinforcing the soil cover with geocell has a significant influence on the reduction of imposed stresses over geofoam and consequently decreasing its deformation.

Keywords: EPS geofoam, geocell, reinforcement, road embankments, lightweight fill

Procedia PDF Downloads 273

Authors: Gopika Rajagopal, Sudheesh Thiyyakkandi

Abstract:

Back-to-back reinforced earth (RE) walls are extensively used in these days as bridge abutments and highway ramps, owing to their cost efficiency and ease of construction. High quality select fill is the most suitable backfill material due to its excellent engineering properties and constructability. However, industries are compelled to use low quality, locally available soil because of its ample availability on site. However, several failure cases of such walls are reported, especially subsequent to rainfall events. The stiffness of reinforcement is one of the major factors affecting the performance of RE walls. The present study focused on analyzing the effect of reinforcement stiffness on the performance of complete select fill, complete marginal fill, and hybrid-fill (i.e., combination of select and marginal fills) back-to-back RE walls, immediately after construction and upon rainwater infiltration through finite element modelling. A constant width to height (W/H) ratio of 3 and height (H) of 6 m was considered for the numerical analysis and the stiffness of reinforcement layers was varied from 500 kN/m to 10000 kN/m. Results showed that reinforcement stiffness had a noticeable influence on the response of RE wall, subsequent to construction as well as rainwater infiltration. Facing displacement was found to decrease and maximum reinforcement tension and factor of safety were observed to increase with increasing the stiffness of reinforcement. However, beyond a stiffness of 5000 kN/m, no significant reduction in facing displacement was observed. The behavior of fully marginal fill wall considered in this study was found to be reasonable even after rainwater infiltration when the high stiffness reinforcement layers are used.

Keywords: back-to-back reinforced earth wall, finite element modelling, rainwater infiltration, reinforcement stiffness

Procedia PDF Downloads 155

Authors: Yassir M. Abbas, Mohammad Iqbal Khan, Galal Fare

Abstract:

In the present work, the structural responses of 12 ultrahigh-performance concrete (UHPC) beams to four-point loading conditions were experimentally and analytically studied. The inclusion of a fibrous system in the UHPC material increased its compressive and flexural strengths by 31.5% and 237.8%, respectively. Based on the analysis of the load-deflection curves of UHPC beams, it was found that UHPC beams with a low reinforcement ratio are prone to sudden brittle failure. This failure behavior was changed, however, to a ductile one in beams with medium to high ratios. The implication is that improving UHPC beam tensile reinforcement could result in a higher level of safety. More reinforcement bars also enabled the load-deflection behavior to be improved, particularly after yielding.

Keywords: ultrahigh-performance concrete, moment capacity, RC beams, hybrid fiber, ductility

Procedia PDF Downloads 69

Authors: Jingwei Du

Abstract:

The need of optimizing air conditioning systems for existing buildings calls for control methods designed with energy-efficiency as a primary goal. The majority of current control methods boil down to two categories: empirical and model-based. To be effective, the former heavily relies on engineering expertise and the latter requires extensive historical data. Reinforcement Learning (RL), on the other hand, is a model-free approach that explores the environment to obtain an optimal control strategy often referred to as “policy”. This research adopts Proximal Policy Optimization (PPO) to improve chiller plant control, and enable the RL agent to collaborate with experienced engineers. It exploits the fact that while the industry lacks historical data, abundant operational data is available and allows the agent to learn and evolve safely under human supervision. Thanks to the development of language models, renewed interest in RL has led to modern, online, policy-based RL algorithms such as the PPO. This research took inspiration from “alignment”, a process that utilizes human feedback to finetune the pretrained model in case of unsafe content. The methodology can be summarized into three steps. First, an initial policy model is generated based on minimal prior knowledge. Next, the prepared PPO agent is deployed so feedback from both critic model and human experts can be collected for future finetuning. Finally, the agent learns and adapts itself to the specific chiller plant, updates the policy model and is ready for the next iteration. Besides the proposed approach, this study also used traditional RL methods to optimize the same simulated chiller plants for comparison, and it turns out that the proposed method is safe and effective at the same time and needs less to no historical data to start up.

Keywords: chiller plant, control methods, energy efficiency, proximal policy optimization, reinforcement learning

Procedia PDF Downloads 28

Authors: Sh. Minapoor, S. Ajeli

Abstract:

Non-crimp three-dimensional (3D) orthogonal carbon fabrics are one of the useful textiles reinforcements in composites. In this paper, flexural and bending properties of a carbon non-crimp 3D orthogonal woven reinforcement are experimentally investigated. The present study is focused on the understanding and measurement of the main bending parameters including flexural stress, strain, and modulus. For this purpose, the three-point bending test method is used and the load-displacement curves are analyzed. The influence of some weave's parameters such as yarn type, geometry of structure, and fiber volume fraction on bending behavior of non-crimp 3D orthogonal carbon fabric is investigated. The obtained results also represent a dataset for the simulation of flexural behavior of non-crimp 3D orthogonal weave carbon composite reinforcement.

Keywords: non-crimp 3D orthogonal weave, carbon composite reinforcement, flexural behavior, three-point bending

Procedia PDF Downloads 297

Authors: Mizam Dogan, Hande Gökdemir

Abstract:

In reinforced concrete elements, reinforcement steel bars are placed in concrete both longitudinal and lateral directions. The lateral reinforcement (called as confinement) which is used for confining circular RC elements is in a spiral shape. If the cross section of RC element is rectangular, stirrups should be rectangular too. At very high compressive stresses concrete will reach its limit strain value and therefore concrete outside the lateral reinforcement, which is not confined, will crush and start to spell. At this stage, concrete core of the RC element tries to expand laterally as a reason of high Poisson’s ratio value of concrete. Such a deformation is prevented by the lateral reinforcement which applies lateral passive pressure on concrete. At very high compressive stresses, the strength of reinforced column member rises to four times σ 2. This increase in strength of member is related to the properties of rectangular stirrups. In this paper, effect of stirrup step spacing to column behavior is calculated and presented confined concrete model is proved by numerical solutions.

Keywords: confined concrete, concrete column, stress-strain, stirrup, solid, frame

Procedia PDF Downloads 451

Authors: Yifan Fan, Xudong Luo, Pingping Lin

Abstract:

Nowadays, dialogue systems increasingly become the way for humans to access many computer systems. So, humans can interact with computers in natural language. A dialogue system consists of three parts: understanding what humans say in natural language, managing dialogue, and generating responses in natural language. In this paper, we survey deep learning based methods for dialogue management, response generation and dialogue evaluation. Specifically, these methods are based on neural network, long short-term memory network, deep reinforcement learning, pre-training and generative adversarial network. We compare these methods and point out the further research directions.

Keywords: dialogue management, response generation, deep learning, evaluation

Procedia PDF Downloads 167

Authors: Ahmed M. Gamal, Adel M. Belal, S. A. Elsoud

Abstract:

This article aims to study the effect of reinforcement inclusions (geogrids) on the sand dunes bearing capacity under strip footings. In this research experimental physical model was carried out to study the effect of the first geogrid reinforcement depth (u/B), the spacing between the reinforcement (h/B) and its extension relative to the footing length (L/B) on the mobilized bearing capacity. This paper presents the numerical modeling using the commercial finite element package (PLAXIS version 8.2) to simulate the laboratory physical model, studying the same parameters previously handled in the experimental work (u/B, L/B & h/B) for the purpose of validation. In this study the soil, the geogrid, the interface element and the boundary condition are discussed with a set of finite element results and the validation. Then the validated FEM used for studying real material and dimensions of strip foundation. Based on the experimental and numerical investigation results, a significant increase in the bearing capacity of footings has occurred due to an appropriate location of the inclusions in sand. The optimum embedment depth of the first reinforcement layer (u/B) is equal to 0.25. The optimum spacing between each successive reinforcement layer (h/B) is equal to 0.75 B. The optimum Length of the reinforcement layer (L/B) is equal to 7.5 B. The optimum number of reinforcement is equal to 4 layers. The study showed a directly proportional relation between the number of reinforcement layer and the Bearing Capacity Ratio BCR, and an inversely proportional relation between the footing width and the BCR.

Keywords: reinforced soil, geogrid, sand dunes, bearing capacity

Procedia PDF Downloads 417