Search results for: multi-agent reinforcement learning

Authors: Francisco José Plácido da Cunha, Tassio Ferenzini Martins Sirqueira, Marx Leles Viana, Carlos José Pereira de Lucena

Abstract:

Open Multiagent Systems (MASs) are societies in which heterogeneous and independently designed entities (agents) work towards similar, or different ends. Software agents are autonomous and the diversity of interests among different members living in the same society is a fact. In order to deal with this autonomy, these open systems use mechanisms of social control (norms) to ensure a desirable social order. This paper considers the following types of norms: (i) obligation — agents must accomplish a specific outcome; (ii) permission — agents may act in a particular way, and (iii) prohibition — agents must not act in a specific way. All of these characteristics mean to encourage the fulfillment of norms through rewards and to discourage norm violation by pointing out the punishments. Once the software agent decides that its priority is the satisfaction of its own desires and goals, each agent must evaluate the effects associated to the fulfillment of one or more norms before choosing which one should be fulfilled. The same applies when agents decide to violate a norm. This paper also introduces a framework for the development of MASs that provide support mechanisms to the agent’s decision-making, using norm-based reasoning. The applicability and validation of this approach is demonstrated applying a traffic intersection scenario.

Keywords: BDI agent, BDI4JADE framework, multiagent systems, normative agents

Procedia PDF Downloads 208

Authors: Ahmed Negm, George Aggidis, Xiandong Ma

Abstract:

With the diverse nature of urban cities, customer demand patterns, landscape topologies or even seasonal weather trends; managing our water distribution networks (WDNs) has proved a complex task. These unpredictable circumstances manifest as pipe failures, intermittent supply and burst events thus adding to water loss, energy waste and increased carbon emissions. Whilst these events are unavoidable, advanced pressure management has proved an effective tool to control and mitigate them. Henceforth, water utilities have struggled with developing a real-time control method that is resilient when confronting the challenges of water distribution. In this paper we use deep reinforcement learning (DRL) algorithms as a novel pressure control strategy to minimise pressure violations and leakage under both burst and background leakage conditions. Agents based on asynchronous actor critic (A2C) and recurrent proximal policy optimisation (Recurrent PPO) were trained and compared to benchmarked optimisation algorithms (differential evolution, particle swarm optimisation. A2C manages to minimise leakage by 32.48% under burst conditions and 67.17% under background conditions which was the highest performance in the DRL algorithms. A2C and Recurrent PPO performed well in comparison to the benchmarks with higher processing speed and lower computational effort.

Keywords: deep reinforcement learning, pressure management, water distribution networks, leakage management

Procedia PDF Downloads 47

Authors: Rodrigo S. Piera, Jailson Sales Ara´ujo, Gabriela B. Lemos, Matthew B. Weiss, John B. DeBrota, Gabriel H. Aguilar, Jacques L. Pienaar

Abstract:

The Born rule was historically viewed as an independent axiom of quantum mechanics until Gleason derived it in 1957 by assuming the Hilbert space structure of quantum measurements [1]. In subsequent decades there have been diverse proposals to derive the Born rule starting from even more basic assumptions [2]. In this work, we demonstrate that a simple reinforcement-learning algorithm, having no pre-programmed assumptions about quantum theory, will nevertheless converge to a behaviour pattern that accords with the Born rule, when tasked with predicting the output of a quantum optical implementation of a symmetric informationally-complete measurement (SIC). Our findings support a hypothesis due to QBism (the subjective Bayesian approach to quantum theory), which states that the Born rule can be thought of as a normative rule for making decisions in a quantum world [3].

Keywords: quantum Bayesianism, quantum theory, quantum information, quantum measurement

Procedia PDF Downloads 62

Authors: Taylan Kabbani, Ekrem Duman

Abstract:

The design of adaptive systems that take advantage of financial markets while reducing the risk can bring more stagnant wealth into the global market. However, most efforts made to generate successful deals in trading financial assets rely on Supervised Learning (SL), which suffered from various limitations. Deep Reinforcement Learning (DRL) offers to solve these drawbacks of SL approaches by combining the financial assets price "prediction" step and the "allocation" step of the portfolio in one unified process to produce fully autonomous systems capable of interacting with its environment to make optimal decisions through trial and error. In this paper, a continuous action space approach is adopted to give the trading agent the ability to gradually adjust the portfolio's positions with each time step (dynamically re-allocate investments), resulting in better agent-environment interaction and faster convergence of the learning process. In addition, the approach supports the managing of a portfolio with several assets instead of a single one. This work represents a novel DRL model to generate profitable trades in the stock market, effectively overcoming the limitations of supervised learning approaches. We formulate the trading problem, or what is referred to as The Agent Environment as Partially observed Markov Decision Process (POMDP) model, considering the constraints imposed by the stock market, such as liquidity and transaction costs. More specifically, we design an environment that simulates the real-world trading process by augmenting the state representation with ten different technical indicators and sentiment analysis of news articles for each stock. We then solve the formulated POMDP problem using the Twin Delayed Deep Deterministic Policy Gradient (TD3) algorithm, which can learn policies in high-dimensional and continuous action spaces like those typically found in the stock market environment. From the point of view of stock market forecasting and the intelligent decision-making mechanism, this paper demonstrates the superiority of deep reinforcement learning in financial markets over other types of machine learning such as supervised learning and proves its credibility and advantages of strategic decision-making.

Keywords: the stock market, deep reinforcement learning, MDP, twin delayed deep deterministic policy gradient, sentiment analysis, technical indicators, autonomous agent

Procedia PDF Downloads 154

Authors: Siti Rossidatul Munawaroh, Siti Khusnul Khowatim

Abstract:

This study is determined to find how is the influence of family attention of students in provides guidance of the student learning. The increasing of student’s learning motivation can be increased made up in various ways, one of them are through students social guidance in their relation with the family. The family not only provides the matter and the learning time but also be supervise for the learning time and guide his children to overcome a learning disability. The character of physics subject in their science experiences at junior high schools has demanded that student’s ability is to think symbolically and understand something in a meaningful manner. Therefore, the reinforcement of the physics learning motivation is clearly necessary not only by the school are related, but the family environment and the society. As for the role of family which includes maintenance, parenting, coaching, and educating both of physically and spiritually, this way is expected to give spirit impulsion in studying physics subject in order to increase student learning achievements.

Keywords: physics subject, the influence of family attention, learning motivation, the Student care

Procedia PDF Downloads 403

Authors: Rahul Paul, Kedar Nath Das

Abstract:

The paradigm of Reinforcement Learning (RL) has become prominent in training intelligent agents to make decisions in environments that are both dynamic and uncertain. The primary objective of RL is to optimize the policy of an agent in order to maximize the cumulative reward it receives throughout a given period. Nevertheless, the process of optimization presents notable difficulties as a result of the inherent trade-off between exploration and exploitation, the presence of extensive state-action spaces, and the intricate nature of the dynamics involved. Evolutionary Optimization Methods (EOMs) have garnered considerable attention as a supplementary approach to tackle these challenges, providing distinct capabilities for optimizing RL policies and value functions. The ongoing advancement of research in both RL and EOMs presents an opportunity for significant advancements in autonomous decision-making systems. The convergence of these two fields has the potential to have a transformative impact on various domains of artificial intelligence (AI) applications. This article highlights the considerable influence of EOMs in enhancing the capabilities of RL. Taking advantage of evolutionary principles enables RL algorithms to effectively traverse extensive action spaces and discover optimal solutions within intricate environments. Moreover, this paper emphasizes the practical implementations of EOMs in the field of RL, specifically in areas such as robotic control, autonomous systems, inventory problems, and multi-agent scenarios. The article highlights the utilization of EOMs in facilitating RL agents to effectively adapt, evolve, and uncover proficient strategies for complex tasks that may pose challenges for conventional RL approaches.

Keywords: machine learning, reinforcement learning, loss function, optimization techniques, evolutionary optimization methods

Procedia PDF Downloads 54

Authors: Hakan Ozturk, Aydin Demir, Kemal Edip, Marta Stojmanovska, Julijana Bojadjieva

Abstract:

There are many factors that affect the behavior of reinforced concrete beams. These can be listed as concrete compressive and reinforcement yield strength, amount of tension, compression and confinement bars, and strain hardening of reinforcement. In the study, support condition of short beams is selected statically indeterminate to first degree. Experimental and numerical analysis are carried for reinforcement concrete (RC) short beams. Dimensions of cross sections are selected as 250mm width and 500 mm height. The length of RC short beams is designed as 2250 mm and these values are constant in all beams. After verifying accurately finite element model, a numerical parametric study is performed with varied diameter of tension reinforcement. Effect of change in diameter is investigated on behavior of RC short beams. As a result of the study, ductility ratios and failure modes are determined, and load-displacement graphs are obtained in order to understand the behavior of short beams. It is deduced that diameter of tension reinforcement plays very important role on the behavior of RC short beams in terms of ductility and brittleness.

Keywords: short beam, reinforced concrete, finite element analysis, longitudinal reinforcement

Procedia PDF Downloads 186

Authors: A. Ardianto, M. A. Putera Agung, S. Pramusandi

Abstract:

Power waterway is one of dam structures and as an intake vertical tunnel or well function for hydroelectric power plants in Jatigede area, Sumedang, West Java. Gate shaft is also one of parts the power waterway system. The paper concerns some consideration in determining a critical state parameter on the back stability analysis of gate shaft or excavation wall stability during excavation. Study analysis was carried out using without and with reinforcement system. Results study showed that reinforcement shaft could reduce the total displacement and safety factor could increases significantly. Based on the back calculation results, it was recommended to install some reinforcement materials and drainage system to reduce pore water pressure.

Keywords: power waterway, reinforcement, displacement, safety

Procedia PDF Downloads 380

Authors: Najmeh Abedzadeh, Matthew Jacobs

Abstract:

An intrusion detection system (IDS) is a software application that monitors malicious activities and generates alerts if any are detected. However, most network activities in IDS datasets are normal, and the relatively few numbers of attacks make the available data imbalanced. Consequently, cyber-attacks can hide inside a large number of normal activities, and machine learning algorithms have difficulty learning and classifying the data correctly. In this paper, a comprehensive literature review is conducted on different types of algorithms for both implementing the IDS and methods in correcting the imbalanced IDS dataset. The most famous algorithms are machine learning (ML), deep learning (DL), synthetic minority over-sampling technique (SMOTE), and reinforcement learning (RL). Most of the research use the CSE-CIC-IDS2017, CSE-CIC-IDS2018, and NSL-KDD datasets for evaluating their algorithms.

Keywords: IDS, imbalanced datasets, sampling algorithms, big data

Procedia PDF Downloads 283

Authors: Faris A. Uriayer, Mehtab Alam

Abstract:

The concept of using a hybrid composite by combining two or more different materials to produce bilinear stress–strain behaviour has become a subject of interest. Having studied the mechanical properties of steel-CFRP specimens (CFRP Laminate Sandwiched between Mild Steel Strips), full size steel-CFRP composite reinforcement were fabricated and used as a new reinforcing material inside beams in lieu of traditional steel bars. Four beams, three beams reinforced with steel-CFRP composite reinforcement and one beam reinforced with traditional steel bars were cast, cured and tested under quasi-static loading. The flexural test results of the beams reinforced with this composite reinforcement showed that the beams with steel-CFRP composite reinforcement had comparable flexural strength and flexural ductility with beams reinforced with traditional steel bars.

Keywords: CFRP laminate, steel strip, flexural behaviour, modified model, concrete beam

Procedia PDF Downloads 659

Authors: S. K. Khosrowshahi, E. Güler

Abstract:

This study investigates the reinforcement function of geosynthetics on the shear strength and strain profile of sand. Conducting a series of simple shear tests, the shearing behavior of the samples under static and cyclic loads was evaluated. Three different types of geosynthetics including geotextile and geonets were used as the reinforcement materials. An image processing analysis based on the optical flow method was performed to measure the lateral displacements and estimate the shear strains. It is shown that besides improving the shear strength, the geosynthetic reinforcement leads a remarkable reduction on the shear strains. The improved layer reduces the required thickness of the soil layer to resist against shear stresses. Consequently, the geosynthetic reinforcement can be considered as a proper approach for the sustainable designs, especially in the projects with huge amount of geotechnical applications like subgrade of the pavements, roadways, and railways.

Keywords: image processing, soil reinforcement, geosynthetics, simple shear test, shear strain profile

Procedia PDF Downloads 198

Authors: Seyed-Ali Sadegh-Zadeh, Kaveh Kavianpour, Hamed Atashbar, Elham Heidari, Saeed Shiry Ghidary, Amir M. Hajiyavand

Abstract:

Evaluation metrics play a critical role in assessing the performance of machine learning models. In this review paper, we provide a comprehensive overview of performance measurement approaches for machine learning models. For each category, we discuss the most widely used metrics, including their mathematical formulations and interpretation. Additionally, we provide a comparative analysis of performance measurement approaches for metric combinations. Our review paper aims to provide researchers and practitioners with a better understanding of performance measurement approaches and to aid in the selection of appropriate evaluation metrics for their specific applications.

Keywords: evaluation metrics, performance measurement, supervised learning, unsupervised learning, reinforcement learning, model robustness and stability, comparative analysis

Procedia PDF Downloads 26

Authors: Guray Arslan, Riza Secer Orkun Keskin

Abstract:

Numerical investigations were conducted to study the influence of flexural reinforcement ratio on the diagonal cracking strength and ultimate shear strength of reinforced concrete (RC) beams without stirrups. Three-dimensional nonlinear finite element analyses (FEAs) of the beams with flexural reinforcement ratios ranging from 0.58% to 2.20% subjected to a mid-span concentrated load were carried out. It is observed that the load-deflection and load-strain curves obtained from the numerical analyses agree with those obtained from the experiments. It is concluded that flexural reinforcement ratio has a significant effect on the shear strength and deflection capacity of RC beams without stirrups. The predictions of the diagonal cracking strength and ultimate shear strength of beams obtained by using the equations defined by a number of codes and researchers are compared with each other and with the experimental values.

Keywords: finite element, flexural reinforcement, reinforced concrete beam, shear strength

Procedia PDF Downloads 304

Authors: Ziying Wu, Danfeng Yan

Abstract:

Multi-Access Edge Computing (MEC) is one of the key technologies of the future 5G network. By deploying edge computing centers at the edge of wireless access network, the computation tasks can be offloaded to edge servers rather than the remote cloud server to meet the requirements of 5G low-latency and high-reliability application scenarios. Meanwhile, with the development of IOV (Internet of Vehicles) technology, various delay-sensitive and compute-intensive in-vehicle applications continue to appear. Compared with traditional internet business, these computation tasks have higher processing priority and lower delay requirements. In this paper, we design a 5G-based Vehicle-Aware Multi-Access Edge Computing Network (VAMECN) and propose a joint optimization problem of minimizing total system cost. In view of the problem, a deep reinforcement learning-based joint computation offloading and task migration optimization (JCOTM) algorithm is proposed, considering the influences of multiple factors such as concurrent multiple computation tasks, system computing resources distribution, and network communication bandwidth. And, the mixed integer nonlinear programming problem is described as a Markov Decision Process. Experiments show that our proposed algorithm can effectively reduce task processing delay and equipment energy consumption, optimize computing offloading and resource allocation schemes, and improve system resource utilization, compared with other computing offloading policies.

Keywords: multi-access edge computing, computation offloading, 5th generation, vehicle-aware, deep reinforcement learning, deep q-network

Procedia PDF Downloads 81

Authors: Adel M. Belal, Sameh Abu El-Soud, Mariam Farid

Abstract:

This study presents the effect of reinforcement inclusions (fibers-geogrids) on fine sand bearing capacity under strip footings. Experimental model tests were carried out using a rectangular plates [(10cm x 38 cm), (7.5 cm x 38 cm), and (12.5 cm x 38 cm)] with a geogrids and randomly reinforced fibers. The width and depth of the geogrid were varied to determine their effects on the engineering properties of treated poorly graded fine sand. Laboratory model test results for the ultimate stresses and the settlement of a rigid strip foundation supported by single and multi-layered fiber-geogrid-reinforced sand are presented. The number of layers of geogrid was varied between 1 to 4. The effect of the first geogrid reinforcement depth, the spacing between the reinforcement and its length on the bearing capacity is investigated by experimental program. Results show that the use of flexible random fibers with a content of 0.125% by weight of the treated sand dunes, with 3 geogrid reinforcement layers, u/B= 0.25 and L/B=7.5, has a significant increase in the bearing capacity of the proposed system.

Keywords: earth reinforcement, geogrid, random fiber, reinforced soil

Procedia PDF Downloads 285

Authors: Can Otuzbir

Abstract:

It is one of the most difficult areas of civil engineering to provide long-lasting structures with the rapid development of concrete and reinforced concrete structures. Concrete is a living material, and the structure where the concrete is located is constantly exposed to external influences. One of these effects is reinforcement corrosion. Reinforcement corrosion of reinforced concrete structures leads to a significant decrease in the carrying capacity of the structural elements, as well as reduced service life. It is undesirable that the service life should be completed sooner than expected. In recent years, advances in glass fiber technology and its use with concrete have developed rapidly. As a result of inability to protect steel reinforcements against corrosion, fiberglass reinforcements have started to be investigated as an alternative material to steel reinforcements, and researches and experimental studies are still continuing. Glass fiber reinforcements have become an alternative material to steel reinforcement because they are resistant to corrosion, lightweight and simple to install compared to steel reinforcement. Glass fiber reinforcements are not corroded and have higher tensile strength, longer life, lighter and insulating properties compared to steel reinforcement. In experimental studies, glass fiber reinforcements have been shown to show superior mechanical properties similar to beams produced with steel reinforcement. The performance of long-term use of glass fiber fibers continues with accelerated experimental studies.

Keywords: glass fiber polymer reinforcement, steel fiber concrete, ultra high performance concrete, bending, GFRP

Procedia PDF Downloads 103

Authors: SKhan Park, Seok Jin Lee, Jong Sun Kim, Jun Ho Lee, Bong Chan Kim

Abstract:

Because of the large economic losses caused by traffic congestion in metropolitan areas, various studies on the underground network design and construction techniques has been performed various studies in the developed countries. In Korea, it has performed a study to develop a versatile double-deck of deep tunnel model. This paper is an introduction to develop a ground reinforcement method to enable the safe tunnel construction in the weakened pillar section like as junction of tunnel. Applicable ground reinforcement method in the weakened section is proposed and it is expected to verify the method by the field application tests.

Keywords: double-deck tunnel, ground reinforcement, tunnel construction, weakened pillar section

Procedia PDF Downloads 373

Authors: S. N. Moghaddas Tafreshi, S. M. Amin Ghotbi

Abstract:

This paper presents a numerical study for investigating the effectiveness of geocell reinforcement in reducing pressure and settlement over EPS geofoam blocks in road embankments. A 3-D FEM model of soil and geofoam was created in ABAQUS, and geocell was also modeled realistically using membrane elements. The accuracy of the model was tested by comparing its results with previous works. Sensitivity analyses showed that reinforcing the soil cover with geocell has a significant influence on the reduction of imposed stresses over geofoam and consequently decreasing its deformation.

Keywords: EPS geofoam, geocell, reinforcement, road embankments, lightweight fill

Procedia PDF Downloads 241

Authors: Yifan Fan, Xudong Luo, Pingping Lin

Abstract:

Nowadays, dialogue systems increasingly become the way for humans to access many computer systems. So, humans can interact with computers in natural language. A dialogue system consists of three parts: understanding what humans say in natural language, managing dialogue, and generating responses in natural language. In this paper, we survey deep learning based methods for dialogue management, response generation and dialogue evaluation. Specifically, these methods are based on neural network, long short-term memory network, deep reinforcement learning, pre-training and generative adversarial network. We compare these methods and point out the further research directions.

Keywords: dialogue management, response generation, deep learning, evaluation

Procedia PDF Downloads 135

Authors: Gopika Rajagopal, Sudheesh Thiyyakkandi

Abstract:

Back-to-back reinforced earth (RE) walls are extensively used in these days as bridge abutments and highway ramps, owing to their cost efficiency and ease of construction. High quality select fill is the most suitable backfill material due to its excellent engineering properties and constructability. However, industries are compelled to use low quality, locally available soil because of its ample availability on site. However, several failure cases of such walls are reported, especially subsequent to rainfall events. The stiffness of reinforcement is one of the major factors affecting the performance of RE walls. The present study focused on analyzing the effect of reinforcement stiffness on the performance of complete select fill, complete marginal fill, and hybrid-fill (i.e., combination of select and marginal fills) back-to-back RE walls, immediately after construction and upon rainwater infiltration through finite element modelling. A constant width to height (W/H) ratio of 3 and height (H) of 6 m was considered for the numerical analysis and the stiffness of reinforcement layers was varied from 500 kN/m to 10000 kN/m. Results showed that reinforcement stiffness had a noticeable influence on the response of RE wall, subsequent to construction as well as rainwater infiltration. Facing displacement was found to decrease and maximum reinforcement tension and factor of safety were observed to increase with increasing the stiffness of reinforcement. However, beyond a stiffness of 5000 kN/m, no significant reduction in facing displacement was observed. The behavior of fully marginal fill wall considered in this study was found to be reasonable even after rainwater infiltration when the high stiffness reinforcement layers are used.

Keywords: back-to-back reinforced earth wall, finite element modelling, rainwater infiltration, reinforcement stiffness

Procedia PDF Downloads 126

Authors: D. Moser, D. Pinto, A. Cipriano

Abstract:

A Disaster Management System (DMS) for countries with different disasters is very important. In the world different disasters like earthquakes, tsunamis, volcanic eruption, fire or other natural or man-made disasters occurs and have an effect on the population. It is also possible that two or more disasters arisen at the same time, this means to handle multi-risk situations. To handle such a situation a Decision Support System (DSS) based on multiagents is a suitable architecture. The most known DMSs deal with one (in the case of an earthquake-tsunami combination with two) disaster and often with one particular disaster. Nevertheless, a DSS helps for a better realtime response. Analyze the existing systems in the literature and expand them for multi-risk disasters to construct a well-organized system is the proposal of our work. The here shown work is an approach of a multi-risk system, which needs an architecture, and well-defined aims. In this moment our study is a kind of case study to analyze the way we have to follow to create our proposed system in the future.

Keywords: decision support system, disaster management system, multi-risk, multiagent system

Procedia PDF Downloads 393

Authors: Yassir M. Abbas, Mohammad Iqbal Khan, Galal Fare

Abstract:

In the present work, the structural responses of 12 ultrahigh-performance concrete (UHPC) beams to four-point loading conditions were experimentally and analytically studied. The inclusion of a fibrous system in the UHPC material increased its compressive and flexural strengths by 31.5% and 237.8%, respectively. Based on the analysis of the load-deflection curves of UHPC beams, it was found that UHPC beams with a low reinforcement ratio are prone to sudden brittle failure. This failure behavior was changed, however, to a ductile one in beams with medium to high ratios. The implication is that improving UHPC beam tensile reinforcement could result in a higher level of safety. More reinforcement bars also enabled the load-deflection behavior to be improved, particularly after yielding.

Keywords: ultrahigh-performance concrete, moment capacity, RC beams, hybrid fiber, ductility

Procedia PDF Downloads 47

Authors: Sh. Minapoor, S. Ajeli

Abstract:

Non-crimp three-dimensional (3D) orthogonal carbon fabrics are one of the useful textiles reinforcements in composites. In this paper, flexural and bending properties of a carbon non-crimp 3D orthogonal woven reinforcement are experimentally investigated. The present study is focused on the understanding and measurement of the main bending parameters including flexural stress, strain, and modulus. For this purpose, the three-point bending test method is used and the load-displacement curves are analyzed. The influence of some weave's parameters such as yarn type, geometry of structure, and fiber volume fraction on bending behavior of non-crimp 3D orthogonal carbon fabric is investigated. The obtained results also represent a dataset for the simulation of flexural behavior of non-crimp 3D orthogonal weave carbon composite reinforcement.

Keywords: non-crimp 3D orthogonal weave, carbon composite reinforcement, flexural behavior, three-point bending

Procedia PDF Downloads 279

Authors: Jared Beard, Ali Baheri

Abstract:

As autonomous systems become more prominent in society, ensuring their safe application becomes increasingly important. This is clearly demonstrated with autonomous cars traveling through a crowded city or robots traversing a warehouse with heavy equipment. Human environments can be complex, having high dimensional state and action spaces. This gives rise to two problems. One being that analytic solutions may not be possible. The other is that in simulation based approaches, searching the entirety of the problem space could be computationally intractable, ruling out formal methods. To overcome this, approximate solutions may seek to find failures or estimate their likelihood of occurrence. One such approach is adaptive stress testing (AST) which uses reinforcement learning to induce failures in the system. The premise of which is that a learned model can be used to help find new failure scenarios, making better use of simulations. In spite of these failures AST fails to find particularly sparse failures and can be inclined to find similar solutions to those found previously. To help overcome this, multi-fidelity learning can be used to alleviate this overuse of information. That is, information in lower fidelity can simulations can be used to build up samples less expensively, and more effectively cover the solution space to find a broader set of failures. Recent work in multi-fidelity learning has passed information bidirectionally using “knows what it knows” (KWIK) reinforcement learners to minimize the number of samples in high fidelity simulators (thereby reducing computation time and load). The contribution of this work, then, is development of the bidirectional multi-fidelity AST framework. Such an algorithm, uses multi-fidelity KWIK learners in an adversarial context to find failure modes. Thus far, a KWIK learner has been used to train an adversary in a grid world to prevent an agent from reaching its goal; thus demonstrating the utility of KWIK learners in an AST framework. The next step is implementation of the bidirectional multi-fidelity AST framework described. Testing will be conducted in a grid world containing an agent attempting to reach a goal position and adversary tasked with intercepting the agent as demonstrated previously. Fidelities will be modified by adjusting the size of a time-step, with higher-fidelity effectively allowing for more responsive closed loop feedback. Results will compare the single KWIK AST learner with the multi-fidelity algorithm with respect to number of samples, distinct failure modes found, and relative effect of learning after a number of trials.

Keywords: multi-fidelity reinforcement learning, multi-fidelity simulation, safety validation, falsification

Procedia PDF Downloads 130

Authors: Mizam Dogan, Hande Gökdemir

Abstract:

In reinforced concrete elements, reinforcement steel bars are placed in concrete both longitudinal and lateral directions. The lateral reinforcement (called as confinement) which is used for confining circular RC elements is in a spiral shape. If the cross section of RC element is rectangular, stirrups should be rectangular too. At very high compressive stresses concrete will reach its limit strain value and therefore concrete outside the lateral reinforcement, which is not confined, will crush and start to spell. At this stage, concrete core of the RC element tries to expand laterally as a reason of high Poisson’s ratio value of concrete. Such a deformation is prevented by the lateral reinforcement which applies lateral passive pressure on concrete. At very high compressive stresses, the strength of reinforced column member rises to four times σ 2. This increase in strength of member is related to the properties of rectangular stirrups. In this paper, effect of stirrup step spacing to column behavior is calculated and presented confined concrete model is proved by numerical solutions.

Keywords: confined concrete, concrete column, stress-strain, stirrup, solid, frame

Procedia PDF Downloads 424

Authors: Chia-Yi Liu, Bo-Bin Xiao, Wen-Bin Lin, Hsiang-Ning Wu, Liang-Hsun Huang

Abstract:

In order to reduce the risk of radiation damage that personnel may suffer during operations in the radiation environment, the use of automated guided vehicles to assist or replace on-site personnel in the radiation environment has become a key technology and has become an important trend. In this paper, we demonstrate our proof of concept for autonomous self-learning radiation source searcher in an unknown environment without a map. The research uses GPU version of Asynchronous Advantage Actor-Critic network (GA3C) of deep reinforcement learning to search for radiation sources. The searcher network, based on GA3C architecture, has self-directed learned and improved how search the anomalous radiation source by training 1 million episodes under three simulation environments. In each episode of training, the radiation source position, the radiation source intensity, starting position, are all set randomly in one simulation environment. The input for searcher network is the fused data from a 2D laser scanner and a RGB-D camera as well as the value of the radiation detector. The output actions are the linear and angular velocities. The searcher network is trained in a simulation environment to accelerate the learning process. The well-performance searcher network is deployed to the real unmanned vehicle, Dashgo E2, which mounts LIDAR of YDLIDAR G4, RGB-D camera of Intel D455, and radiation detector made by Institute of Nuclear Energy Research. In the field experiment, the unmanned vehicle is enable to search out the radiation source of the 18.5MBq Na-22 by itself and avoid obstacles simultaneously without human interference.

Keywords: deep reinforcement learning, GA3C, source searching, source detection

Procedia PDF Downloads 87

Authors: Ahmed M. Gamal, Adel M. Belal, S. A. Elsoud

Abstract:

This article aims to study the effect of reinforcement inclusions (geogrids) on the sand dunes bearing capacity under strip footings. In this research experimental physical model was carried out to study the effect of the first geogrid reinforcement depth (u/B), the spacing between the reinforcement (h/B) and its extension relative to the footing length (L/B) on the mobilized bearing capacity. This paper presents the numerical modeling using the commercial finite element package (PLAXIS version 8.2) to simulate the laboratory physical model, studying the same parameters previously handled in the experimental work (u/B, L/B & h/B) for the purpose of validation. In this study the soil, the geogrid, the interface element and the boundary condition are discussed with a set of finite element results and the validation. Then the validated FEM used for studying real material and dimensions of strip foundation. Based on the experimental and numerical investigation results, a significant increase in the bearing capacity of footings has occurred due to an appropriate location of the inclusions in sand. The optimum embedment depth of the first reinforcement layer (u/B) is equal to 0.25. The optimum spacing between each successive reinforcement layer (h/B) is equal to 0.75 B. The optimum Length of the reinforcement layer (L/B) is equal to 7.5 B. The optimum number of reinforcement is equal to 4 layers. The study showed a directly proportional relation between the number of reinforcement layer and the Bearing Capacity Ratio BCR, and an inversely proportional relation between the footing width and the BCR.

Keywords: reinforced soil, geogrid, sand dunes, bearing capacity

Procedia PDF Downloads 391

Authors: Magdi M. E. Zumrawi, Nehla Mansour

Abstract:

This paper aims to assess the efficiency of using geogrid reinforcement for subgrade stabilization. The literature of applying geogrid reinforcement technique for pavements built on soft subgrades and the previous experiences were reviewed. Laboratory tests were conducted on soil reinforced with geogrids in one or several layers. The soil specimens were compacted in four layers with or without geogrid sheets. The California Bearing Ratio (CBR) test, in soaking condition, was performed on natural soil and soil-geogrid specimens. The test results revealed that the CBR value is much affected by the geogrid sheet location and the number of sheets used in the soil specimen. When a geogrid sheet was placed at the 1^st layer of the soil, there was an increment of 26% in the CBR value. Moreover, the CBR value was significantly increased by 62% when geogrid sheets were placed at all four layers. The high CBR value is attributed to interface friction and interlock involved in the geogrid/ soil interactions. It could be concluded that geogrid reinforcement is successful and more economical technique.

Keywords: geogrid, reinforcement, stabilization, subgrade

Procedia PDF Downloads 291

Authors: Zeineb Ben Salem, Wissem Frikha, Mounir Bouassida

Abstract:

The aim of this paper is to assess the effectiveness of stone columns as a liquefaction countermeasure focusing on shear reinforcementbenefit. In fact, stone columns which have high shear modulus relative to the surrounding soils potentially can carry higher shear stress levels. Thus, stone columns provide shear reinforcement and decrease the Cyclic Shear Stress Ratio CSR to which the treated soils would be subjected during an earthquake. In order to quantify the level of shear stress reduction in reinforced soil, several approaches have been developed. Nevertheless, the available approaches do not take into account the improvement of the soil parameters, mainly the shear modulusdue to stone columns installation. Indeed, in situ control tests carried out before and after the installation of stone columns based upon the results of collected data derived from 24 case histories have given evidence of the improvement of the existing soil properties.In this paper, the assessment of shear reinforcement of stone columns that accounts such improvement of the soil parameters due to stone column installation is investigated. Comparative results indicate that considering the improvement effects considerably affect the assessment of shear reinforcement for liquefaction analysis of reinforced soil by stone columns.

Keywords: stone column, liquefaction, shear reinforcement, CSR, soil improvement

Procedia PDF Downloads 122

Authors: Dhyaalddin Bahaalddin Noori Zangana

Abstract:

This study involves the behavior of reinforced sand under a square footing. A series of bearing capacity tests were performed on a small-scale laboratory model, which filled with a poorly-graded homogenous bed of sand, which was placed in a medium dense state using sand raining technique. The sand was reinforced with 40 mm wide household aluminum foil strips. The main studied parameters was to consider the effect of reinforcing strip length, with various linear density of reinforcement, number of reinforcement layers and depth of top layer of reinforcement below the footing, on load-settlement behavior, bearing capacity ratio and settlement reduction factor. The relation of load-settlement generally showed similar trend in all the tests. Failure was defined as settlement equal to 10% of the footing width. The recommended optimum reinforcing strip length, linear density of reinforcement, number of reinforcement layers and depth of top layer of reinforcing strips that give the maximum bearing capacity improvement and minimum settlement reduction factor were presented and discussed. Different bearing capacity ration versus length of the reinforcing strips and settlement reduction factor versus length of the reinforcing strips relations at failure were showed improvement of bearing capacity ratio by a factor of 3.82 and reduction of settlement reduction factor by a factor of 0.813. The optimum length of reinforcement was found to be 7.5 times the footing width.

Keywords: square footing, relative density, linear density of reinforcement, bearing capacity ratio, load-settlement behaviour

Procedia PDF Downloads 75