Search results for: yttrium reinforcement

Authors: Albert Bou, Sebastian Dittert, Gianni de Fabritiis

Abstract:

Advancing reinforcement learning (RL) requires tools that are flexible enough to easily prototype new methods while avoiding impractically slow experimental turnaround times. To match the first requirement, the most popular RL libraries advocate for highly modular agent composability, which facilitates experimentation and development. To solve challenging environments within reasonable time frames, scaling RL to large sampling and computing resources has proved a successful strategy. However, this capability has been so far difficult to combine with modularity. In this work, we explore design choices to allow agent composability both at a local and distributed level of execution. We propose a versatile approach that allows the definition of RL agents at different scales through independent, reusable components. We demonstrate experimentally that our design choices allow us to reproduce classical benchmarks, explore multiple distributed architectures, and solve novel and complex environments while giving full control to the user in the agent definition and training scheme definition. We believe this work can provide useful insights to the next generation of RL libraries.

Keywords: deep reinforcement learning, Python, PyTorch, distributed training, modularity, library

Procedia PDF Downloads 54

Authors: Dhval Chauhan, Mahesh Chudasama

Abstract:

This study has been undertaken to give a review of Polyethylene Terephthalate Glycol (PETG) material used in Fused Deposition Modelling (FDM). This paper offers a review of the existing literature on polyethylene terephthalate glycol (PETG) material, the objective of the paper is to providing guidance on different process parameters that can be used to improve the strength of the part by performing various testing like tensile, compressive, flexural, etc. This work is target to find new paths that can be used for further development of the use of fiber reinforcement in PETG material.

Keywords: PETG, FDM, tensile strength, flexural strength, fiber reinforcement

Procedia PDF Downloads 155

Authors: Sirapop Nuannimnoi, Hojjat Baghban, Ching-Yao Huang

Abstract:

Over the past decade, with the fast development of blockchain technology since the birth of Bitcoin, there has been a massive increase in the usage of Cryptocurrencies. Cryptocurrencies are not seen as an investment opportunity due to the market’s erratic behavior and high price volatility. With the recent success of deep reinforcement learning (DRL), portfolio management can be modeled and automated. In this paper, we propose a novel DRL-based multi-agent system to automatically make proper trading decisions on multiple cryptocurrencies and gain profits in the highly volatile cryptocurrency market. We also extend this multi-agent system with horizontal federated transfer learning for better adapting to the inclusion of new cryptocurrencies in our portfolio; therefore, we can, through the concept of diversification, maximize our profits and minimize the trading risks. Experimental results through multiple simulation scenarios reveal that this proposed algorithmic trading system can offer three promising key advantages over other systems, including maximized profits, minimized risks, and adaptability.

Keywords: cryptocurrency portfolio management, algorithmic trading, federated learning, multi-agent reinforcement learning

Procedia PDF Downloads 86

Authors: Akshay Paranjape, Nils Plettenberg, Robert Schmitt

Abstract:

Producing faulty products can be costly for manufacturing companies and wastes resources. To reduce scrap rates in manufacturing, process parameters can be optimized using machine learning. Thus far, research mainly focused on optimizing specific processes using traditional algorithms. To develop a framework that enables real-time optimization based on a predictive model for an arbitrary production process, this study explores the application of reinforcement learning (RL) in this field. Based on a thorough review of literature about RL and process parameter optimization, a model based on maximum a posteriori policy optimization that can handle both numerical and categorical parameters is proposed. A case study compares the model to state–of–the–art traditional algorithms and shows that RL can find optima of similar quality while requiring significantly less time. These results are confirmed in a large-scale validation study on data sets from both production and other fields. Finally, multiple ways to improve the model are discussed.

Keywords: reinforcement learning, production process optimization, evolutionary algorithms, policy optimization, actor critic approach

Procedia PDF Downloads 63

Authors: C. P. Providakis, G. M. Angeli, M. J. Favvata, N. A. Papadopoulos, C. E. Chalioris, C. G. Karayannis

Abstract:

An effort for the detection of damages in the reinforcement bars of reinforced concrete members using PZTs is presented. The damage can be the result of excessive elongation of the steel bar due to steel yielding or due to local steel corrosion. In both cases the damage is simulated by considering reduced diameter of the rebar along the damaged part of its length. An integration approach based on both electromechanical admittance methodology and guided wave propagation technique is used to evaluate the artificial damage on the examined longitudinal steel bar. Two actuator PZTs and a sensor PZT are considered to be bonded on the examined steel bar. The admittance of the Sensor PZT is calculated using COMSOL 3.4a. Fast Furrier Transformation for a better evaluation of the results is employed. An effort for the quantification of the damage detection using the root mean square deviation (RMSD) between the healthy condition and damage state of the sensor PZT is attempted. The numerical value of the RSMD yields a level for the difference between the healthy and the damaged admittance computation indicating this way the presence of damage in the structure. Experimental measurements are also presented.

Keywords: concrete reinforcement, damage detection, electromechanical admittance, experimental measurements, finite element method, guided waves, PZT

Procedia PDF Downloads 229

Authors: C. P. Providakis, G. M. Angeli, M. J. Favvata, N. A. Papadopoulos, C. E. Chalioris, C. G. Karayannis

Abstract:

An effort for the detection of damages in the reinforcement bars of reinforced concrete members using PZTs is presented. The damage can be the result of excessive elongation of the steel bar due to steel yielding or due to local steel corrosion. In both cases the damage is simulated by considering reduced diameter of the rebar along the damaged part of its length. An integration approach based on both electro-mechanical admittance methodology and guided wave propagation technique is used to evaluate the artificial damage on the examined longitudinal steel bar. Two actuator PZTs and a sensor PZT are considered to be bonded on the examined steel bar. The admittance of the Sensor PZT is calculated using COMSOL 3.4a. Fast Furrier Transformation for a better evaluation of the results is employed. An effort for the quantification of the damage detection using the root mean square deviation (RMSD) between the healthy condition and damage state of the sensor PZT is attempted. The numerical value of the RSMD yields a level for the difference between the healthy and the damaged admittance computation indicating this way the presence of damage in the structure. Experimental measurements are also presented.

Keywords: concrete reinforcement, damage detection, electromechanical admittance, experimental measurements, finite element method, guided waves, PZT

Procedia PDF Downloads 263

Authors: Zaidour Mohamed, Ghalem Ali Jr., Achit Henni Mohamed

Abstract:

This project is destined to study pole junctions of reinforced concrete beams subjected to seismic loads. A literature review was made to clarify the work done by researchers in the last three decades and especially the results of the last two years that were studied for the determination of the method of calculating the transverse reinforcement in the different nodes of a structure. For implementation efforts in the columns and beams of a building R + 4 in zone 3 were calculated using the finite element method through software. These results are the basis of our work which led to the calculation of the transverse reinforcement of the nodes of the structure in question.

Keywords: beam–column joints, cyclic loading, shearing force, damaged joint

Procedia PDF Downloads 523

Authors: H. Tanyildizi, M. Abuqebitah, I. Cavdar, M. Demir, L. Kabasakal

Abstract:

Aim: Whole liver radiation has the modest benefit in the treatment of unresectable hepatic metastases but the radiation doses must keep in control. Otherwise, RILD complications may arise. In this study, we aimed to calculate amount of maximum permissible activity (MPA) and critical organ absorbed doses with MIRD methodology, to evaluate tumour doses for treatment response and whole liver doses for RILD and to find optimal liver function test additionally. Materials and Methods: This study includes 29 patients who attended our nuclear medicine department suffering from Y-90 microspheres treatment. 10 mCi Tc-99m MAA was applied to the patients for dosimetry via IV. After the injection, whole body SPECT/CT images were taken in one hour. The minimum therapeutic tumour dose is on the point of being 120 Gy1, the amount of activities were calculated with MIRD methodology considering volumetric tumour/liver rate. A sub-working group was created with 11 patients randomly and liver clearance rate with Tc-99m-Mebrofenin was calculated according to Ekman formalism. Results: The volumetric tumour/liver rates were found between 33-66% (Maksimum Tolarable Dose (MTD) 48-52Gy3) for 4 patients, were found less than 33% (MTD 72Gy3) for 25 patients. According to these results the average amount of activity, mean liver dose and mean tumour dose were found 1793.9±1.46 MBq, 32.86±0.19 Gy, and 138.26±0.40 Gy. RILD was not observed in any patient. In sub-working group, the relationship between Bilirubin, Albumin, INR (which show presence of liver disease and its degree), liver clearance with Tc-99m-Mebrofenin and calculated activity amounts were found r=0.49, r=0.27, r=0.43, r=0.57, respectively. Discussions: The minimum tumour dose was found 120 Gy for positive dose-response relation. If volumetric tumour/liver rate was > 66%, dose 30 Gy; if volumetric tumour/liver rate 33-66%, dose escalation 48 Gy; if volumetric tumour/liver rate < 33%, dose 72 Gy. These dose limitations did not create RILD. Clearance measurement with Mebrofenin was concluded that the best method to determine the liver function. Therefore, liver clearance rate with Tc-99m-Mebrofenin should be considered in calculation of yttrium-90 microspheres dosimetry.

Keywords: clearance, dosimetry, liver, RILD

Procedia PDF Downloads 407

Authors: Romisaa Ali

Abstract:

This paper presents a comparison between twin-delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC) reinforcement learning algorithms in the context of training robust navigation policies for Jackal robots. By leveraging an open-source framework and custom motion control environments, the study evaluates the performance, robustness, and transferability of the trained policies across a range of scenarios. The primary focus of the experiments is to assess the training process, the adaptability of the algorithms, and the robot’s ability to navigate in previously unseen environments. Moreover, the paper examines the influence of varying environmental complexities on the learning process and the generalization capabilities of the resulting policies. The results of this study aim to inform and guide the development of more efficient and practical reinforcement learning-based navigation policies for Jackal robots in real-world scenarios.

Keywords: Jackal robot environments, reinforcement learning, TD3, SAC, robust navigation, transferability, custom environment

Procedia PDF Downloads 51

Authors: Till Fehlauer, Blanche Collin, Bernard Angeletti, Andrea Somogyi, Claire Lallemand, Perrine Chaurand, Cédric Dentant, Clement Levard, Jerome Rose

Abstract:

Over the last decades, yttrium (Y) has gained importance in high-tech applications. It is an essential part of alloys and compounds used for lasers, displays, or cell phones, for example. Due to its chemical similarities with the lanthanides, Y is often considered a rare earth element (REE). Despite their increased usage, the environmental behavior of REEs remains poorly understood. Especially regarding their interactions with plants, many uncertainties exist. On the one hand, Y is known to have a negative effect on root development and germination, but on the other hand, it appears to promote plant growth at low concentrations. In order to understand these phenomena, a precise knowledge is necessary about how Y is absorbed by the plant and how it is handled once inside the organism. Contradictory studies exist, stating that due to a similar ionic radius, Y and the other REEs might be absorbed through Ca²⁺-channels, while others suspect that Y has a shared pathway with Al³⁺. In this study, laser ablation coupled ICP-MS, and synchrotron-based micro-X-ray fluorescence (µXRF, beamline Nanoscopium, SOLEIL, France) have been used in order to localize Y within the plant tissue and identify associated elements. The plant used in this study is Saxifraga paniculata, a rugged alpine plant that has shown an affinity for Y in previous studies (in prep.). Furthermore, Saxifraga paniculata performs guttation, which means that it possesses phloem sap secreting openings on the leaf surface that serve to regulate root pressure. These so-called hydathodes could provide special insights in elemental transport in plants. The plants have been grown on Y doped soil (500mg/kg DW) for four months. The results showed that Y was mainly concentrated in the roots of Saxifraga paniculata (260 ± 85mg/kg), and only a small amount was translocated to the leaves (10 ± 7.8mg/kg). µXRF analysis indicated that within the root transects, the majority of Y remained in the epidermis and hardly penetrated the stele. Laser ablation coupled ICP-MS confirmed this finding and showed a positive correlation in the roots between Y, Fe, Al, and to a lesser extent Ca. In the stem transect, Y was mainly detected in a hotspot of approximately 40µm in diameter situated in the endodermis area. Within the stem and especially in the hotspot, Y was highly colocalized with Al and Fe. Similar-sized Y hotspots have been detected in/on the leaves. All of them were strongly colocalized with Al and Fe, except for those situated within the hydathodes, which showed no colocalization with any of the measured elements. Accordingly, a relation between Y and Ca during root uptake remains possible, whereas a correlation to Fe and Al appears to be dominant in the aerial parts, suggesting common storage compartments, the formation of complexes, or a shared pathway during translocation.

Keywords: laser ablation inductively coupled plasma mass spectrometry (LA-ICP-MS), Phytoaccumulation, Rare earth elements, Saxifraga paniculata, Synchrotron-based micro-X-ray fluorescence, Yttrium

Procedia PDF Downloads 124

Authors: S. Bahadır Yüksel, Alptuğ Ünal

Abstract:

The composite shear walls (CSW) with steel encased profiles can be used as lateral-load resisting systems for buildings that require considerable large lateral-load capacity. The aim of this work is to propose the experimental work conducted on CSW having L section folded plate (L shape steel made-up sections) as longitudinal reinforcement in boundary regions. The study in this paper present the experimental test conducted on CSW having L section folded plate as longitudinal reinforcement in boundary regions. The tested 1/3 geometric scaled CSW has aspect ratio of 3.2. L-shape structural steel materials with 2L-19x57x7mm dimensions were placed in shear wall boundary zones. The seismic behavior of CSW test specimen was investigated by evaluating and interpreting the hysteresis curves, envelope curves, rigidity and consumed energy graphs of this tested element. In addition to this, the experimental results, deformation and cracking patterns were evaluated, interpreted and suggestions of the design recommendations were proposed.

Keywords: shear wall, composite shear wall, boundary reinforcement, earthquake resistant structural design, L section

Procedia PDF Downloads 298

Authors: S. A. Naeini, H. R. Rahmani, M. Hossein Zade

Abstract:

Many classical bearing capacity theories assume that the natural soil's layers are homogenous for determining the bearing capacity of the soil. But, in many practical projects, we encounter multi-layer soils. Geosynthetic as reinforcement materials have been extensively used in the construction of various structures. In this paper, numerical analysis of the Plate Load Test (PLT) using of ABAQUS software in double-layered soils with different thicknesses of sandy and gravelly layers reinforced with geogrid was considered. The PLT is one of the common filed methods to calculate parameters such as soil bearing capacity, the evaluation of the compressibility and the determination of the Subgrade Reaction module. In fact, the influence of the geogrid layers on the bearing capacity of the layered soils is investigated. Finally, the most appropriate mode for the distance and number of reinforcement layers is determined. Results show that using three layers of geogrid with a distance of 0.3 times the width of the loading plate has the highest efficiency in bearing capacity of double-layer (sand and gravel) soils. Also, the significant increase in bearing capacity between unreinforced and reinforced soil with three layers of geogrid is caused by the condition that the upper layer (gravel) thickness is equal to the loading plate width.

Keywords: bearing capacity, reinforcement, geogrid, plate load test, layered soils

Procedia PDF Downloads 143

Authors: Bindhu Lal, Karnika Kochal

Abstract:

Clayey soil contains clay minerals with traces of metal oxides and organic matter, which exhibits properties like low drainage, high plasticity, and shrinkage. To overcome these issues, various soil reinforcement techniques are used to elevate the stiffness, water tightness, and bearing capacity of the soil. Such techniques include cementation, bituminization, freezing, fiber inclusion, geo-synthetics, nailing, etc. Reinforcement of soil with fibers has been a cost-effective solution to soil improvement problems. An experimental study was undertaken involving the inclusion of cotton waste fibers in clayey soil as reinforcement with different fiber contents (1%, 1.5%, 2%, and 2.5% by weight) and analyzing its effects on the unconfined compressive strength of the soil. Two categories of soil were taken, comprising of natural clay and clay mixed with 5% sodium bentonite by weight. The soil specimens were subjected to proctor compaction and unconfined compression tests. The validated outcome shows that fiber inclusion has a strikingly positive impact on the compressive strength and axial strain at failure of the soil. Based on the commendatory results procured, compressive strength was found to be directly proportional to the fiber content, with the effect being more pronounced at lower water content.

Keywords: bentonite clay, clay, cotton fibers, unconfined compressive strength

Procedia PDF Downloads 148

Authors: Yuling Cui, Danhao Deng, Chaowei Wang, Weidong Wang

Abstract:

In recent years, unmanned aerial vehicles (UAVs) have been widely used in wireless communication, attracting more and more attention from researchers. UAVs can not only serve as a relay for auxiliary communication but also serve as an aerial base station for ground users (GUs). However, limited energy means that they cannot work all the time and cover a limited range of services. In this paper, we investigate 2D UAV trajectory design and power allocation in order to maximize the UAV's service time and downlink throughput. Based on deep reinforcement learning, we propose a depth deterministic strategy gradient algorithm for trajectory design and power distribution (TDPA-DDPG) to solve the energy-efficient and communication service quality problem. The simulation results show that TDPA-DDPG can extend the service time of UAV as much as possible, improve the communication service quality, and realize the maximization of downlink throughput, which is significantly improved compared with existing methods.

Keywords: UAV trajectory design, power allocation, energy efficient, downlink throughput, deep reinforcement learning, DDPG

Procedia PDF Downloads 107

Authors: Mohamed Haneef, Shanawaz Patil, Syed Zameer, Mohammed Mohsin Ali

Abstract:

The emerging technologies and trends of present generation requires downsizing the unwieldy structures to light weight structures on one hand and integration of varied properties on other hand to meet the application demands. In the present investigation an attempt is made to familiarize and best possibilities of reinforcing agent in aluminum 7075 matrix with naturally occurring beryl (Be) and graphene (Gr) to develop a new hybrid composite material. A stir casting process was used to fabricate with fixed volume fraction of 6wt% weight beryl and various volume fractions of 0.5wt%, 1wt%, 1.5wt% and 2wt% of graphene. The properties such as tensile strength, hardness and dry sliding wear behavior of hybrid composites were examined. The crystallite size and morphology of the graphene and beryl particles were analyzed with X-ray diffraction (XRD) and scanning electron microscopy (SEM) respectively. It was observed that ultimate tensile strength and hardness of the hybrid composite increased with increasing reinforcement volume fraction as compared to specimen without reinforcement additions. The dry sliding wear behavior of the hybrid composites decreases as compared to Al7075 alloy without reinforcement.

Keywords: Al7075, beryl, graphene, TEM, wear

Procedia PDF Downloads 131

Authors: Akshay Paranjape, Nils Plettenberg, Robert Schmitt

Abstract:

Traditional optimization methods like evolutionary algorithms are widely used in production processes to find an optimal or near-optimal solution of control parameters based on the simulated environment space of a process. These algorithms are computationally intensive and therefore do not provide the opportunity for real-time optimization. This paper utilizes the Deep Reinforcement Learning (DRL) framework to find an optimal or near-optimal solution for control parameters. A model based on maximum a posteriori policy optimization (Hybrid-MPO) that can handle both numerical and categorical parameters is used as a benchmark for comparison. A comparative study shows that DRL can find optimal solutions of similar quality as compared to evolutionary algorithms while requiring significantly less time making them preferable for real-time optimization. The results are confirmed in a large-scale validation study on datasets from production and other fields. A trained XGBoost model is used as a surrogate for process simulation. Finally, multiple ways to improve the model are discussed.

Keywords: reinforcement learning, evolutionary algorithms, production process optimization, real-time optimization, hybrid-MPO

Procedia PDF Downloads 77

Authors: O. Vlcek

Abstract:

The paper deals with current issues in the research of advanced methods to increase the reliability of traditional timber structural elements. It analyses the issue of strengthening of bent timber beams, such as ceiling beams in old (historical) buildings with the additional concrete slab in combination with externally bonded fibre-reinforced polymer. The study evaluates deflection of a selected group of timber beams with concrete slab and additional CFRP reinforcement using different calculating methods and observes differences in results from different calculating methods. An elastic calculation method and evaluation with FEM analysis software were used.

Keywords: timber-concrete composite, strengthening, fibre-reinforced polymer, theoretical analysis

Procedia PDF Downloads 289

Authors: Tong He, Long Chen, Irag Mantegh, Wen-Fang Xie

Abstract:

This paper proposes an image-based obstacle avoidance and tracking target identification strategy in GPS-degraded or GPS-denied environment for an Unmanned Aerial Vehicle (UAV). The traditional force algorithm for obstacle avoidance could produce local minima area, in which UAV cannot get away obstacle effectively. In order to eliminate it, an artificial potential approach based on harmonic potential is proposed to guide the UAV to avoid the obstacle by using the vision system. And image-based visual servoing scheme (IBVS) has been adopted to implement the proposed obstacle avoidance approach. In IBVS, the pixel accuracy is a key factor to realize the obstacle avoidance. In this paper, the deep reinforcement learning framework has been applied by reducing pixel errors through constant interaction between the environment and the agent. In addition, the combination of OpenTLD and Tensorflow based on neural network is used to identify the type of tracking target. Numerical simulation in Matlab and ROS GAZEBO show the satisfactory result in target identification and obstacle avoidance.

Keywords: image-based visual servoing, obstacle avoidance, tracking target identification, deep reinforcement learning, artificial potential approach, neural network

Procedia PDF Downloads 114

Authors: Soofiyan Atar, Adil Shaikh, Sahil Rajpurkar, Pragnesh Bhalala, Aniket Desai, Irfan Siddavatam

Abstract:

Deep reinforcement learning (deep RL) algorithms leverage the symbolic power of complex controllers by automating it by mapping sensory inputs to low-level actions. Deep RL eliminates the complex robot dynamics with minimal engineering. Deep RL provides high-risk involvement by directly implementing it in real-world scenarios and also high sensitivity towards hyperparameters. Tuning of hyperparameters on a pneumatic quadruped robot becomes very expensive through trial-and-error learning. This paper presents an automated learning control for a pneumatic quadruped robot using sample efficient deep Q learning, enabling minimal tuning and very few trials to learn the neural network. Long training hours may degrade the pneumatic cylinder due to jerk actions originated through stochastic weights. We applied this method to the pneumatic quadruped robot, which resulted in a hopping gait. In our process, we eliminated the use of a simulator and acquired a stable gait. This approach evolves so that the resultant gait matures more sturdy towards any stochastic changes in the environment. We further show that our algorithm performed very well as compared to programmed gait using robot dynamics.

Keywords: model-based reinforcement learning, gait stability, supervised learning, pneumatic quadruped

Procedia PDF Downloads 282

Authors: N. Attari, S. Amziane, M. Chemrouk

Abstract:

A large number of old buildings have been identified as having potentially critical detailing to resist earthquakes. The main reinforcement of lap-spliced columns just above the joint region, discontinuous bottom beam reinforcement, and little or no joint transverse reinforcement are the most critical details of interior beam column joints in such buildings. This structural type constitutes a large share of the building stock, both in developed and developing countries, and hence it represents a substantial exposure. Direct observation of damaged structures, following the Algiers 2003 earthquake, has shown that damage occurs usually at the beam-column joints, with failure in bending or shear, depending on geometry and reinforcement distribution and type. While substantial literature exists for the design of concrete frame joints to withstand this type of failure, after the earthquake many structures were classified as slightly damaged and, being uneconomic to replace them, at least in the short term, suitable means of repairs of the beam column joint area are being studied. Furthermore; there exists a large number of buildings that need retrofitting of the joints before the next earthquake. The paper reports the results of the experimental programme, constituted of three beam-column reinforced concrete joints at a scale of one to three (1/3) tested under the effect of a pre-stressing axial load acting over the column. The beams were subjected at their ends to an alternate cyclic loading under displacement control to simulate a seismic action. Strain and cracking fields were monitored with the help a digital recording camera. Following the analysis of the results, a comparison can be made between the performances in terms of ductility, strength and mode of failure of the different strengthening solution considered.

Keywords: fibre reinforced polymers, joints, reinforced concrete, beam columns

Procedia PDF Downloads 384

Authors: Ahmed Shaban Abdel Hay Gabr

Abstract:

Water hyacinth (W-H) has an adverse effect on Nile river in Egypt, it absorbs high quantities of water, it needs to serve these quantities especially at this time, so by burning W-H, it can be used in concrete mix to reduce the permeability of concrete and increase both the compressive and splitting strength. The effect of W-H on non-structural concrete properties was studied, but there is a lack of studies about the behavior of structural concrete containing W-H. Therefore, in the present study, the behavior of 15 RC beams with 100 x 150 mm cross section, 1250 mm span, different reinforcement ratios and different W-H ratios were studied by testing the beams under two-point bending test. The test results showed that Water Hyacinth is compatible with RC which yields promising results.

Keywords: beams, reinforcement ratio, reinforced concrete, water hyacinth

Procedia PDF Downloads 423

Authors: Xiaohua Zou, Yongxin Su

Abstract:

The reasonable operation of heating, ventilation, and air conditioning (HVAC) is of great significance in improving the security, stability, and economy of power system operation. However, the uncertainty of the operating environment, thermal comfort varies by users and rapid decision-making pose challenges for HVAC demand response optimization. In this regard, this paper proposes a reinforcement learning-based method for HVAC demand response optimization considering few-shot personalized thermal comfort (PTC). First, an HVAC DR optimization framework based on few-shot PTC model and DRL is designed, in which the output of few-shot PTC model is regarded as the input of DRL. Then, a few-shot PTC model that distinguishes between awake and asleep states is established, which has excellent engineering usability. Next, based on soft actor criticism, an HVAC DR optimization algorithm considering the user’s PTC is designed to deal with uncertainty and make decisions rapidly. Experiment results show that the proposed method can efficiently obtain use’s PTC temperature, reduce energy cost while ensuring user’s PTC, and achieve rapid decision-making under uncertainty.

Keywords: HVAC, few-shot personalized thermal comfort, deep reinforcement learning, demand response

Procedia PDF Downloads 40

Authors: Wiem Zemzem, Moncef Tagina

Abstract:

In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new action selection strategy, called ε-greedy-MPA (the ε-greedy policy favoring the most promising actions) is proposed. Unlike existing exploration/exploitation policies (EEPs) such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. Finally, various simulations with ball-catching problem have been conducted to evaluate the ε-greedy-MPA policy. The results of simulated experiments show that combining this policy with the Qlearning method is more effective and efficient compared with the ε-greedy policy in stationary environments and the utility-based reinforcement learning approach in non stationary environments.

Keywords: autonomous mobile robot, exploration/ exploitation policy, large, dynamic environment, reinforcement learning

Procedia PDF Downloads 386

Authors: Mohd Ritzman Abdul Karim, Zhaohui Huang

Abstract:

A current focus in seismic engineering practice is the development of seismic design approach that focuses on the performance-based design. Performance-based design aims to design the structures to achieve specified performance based on the damage limit states. This damage limit is more restrictive limit than life safety and needs to be carefully estimated to avoid damage in piers due to failure in transverse reinforcement. In this paper, a different perspective of damage limit states has been explored by integrating two damage control material limit state, concrete and reinforcement by introduced parameters such as expected yield stress of transverse reinforcement where peak tension strain prior to bar buckling is introduced in a recent study. The different perspective of damage limit states with modified yield displacement and the modified plastic-hinge length is used in order to predict damage-control target displacement for reinforced concreate (RC) bridge pier. Three-dimensional (3D) finite element (FE) model has been developed for estimating damage target displacement to validate proposed damage limit states. The result from 3D FE analysis was validated with experimental study found in the literature. The validated model then was applied to predict the damage target displacement for RC bridge pier and to validate the proposed study. The tensile strain on reinforcement and compression on concrete were used to determine the predicted damage target displacement and compared with the proposed study. The result shows that the proposed damage limit states were efficient in predicting damage-control target displacement consistent with FE simulations.

Keywords: damage-control target displacement, damage limit states, reinforced concrete bridge pier, yield displacement

Procedia PDF Downloads 124

Authors: K. S. Sivakumaran, Bo Chen

Abstract:

The floor beams of steel buildings, cold-formed steel floor joists, in particular, often require large web openings, which may affect their shear capacities. A cost effective way to mitigate the detrimental effects of such openings is to weld/fasten reinforcements. A difficulty associated with an experimental investigation to establish suitable reinforcement schemes for openings in shear zone is that moment always coexists with the shear, and thus, it is impossible to create pure shear state in experiments, resulting in moment influenced results. However, finite element analysis can be conveniently used to investigate the pure shear behaviour of webs including webs with reinforced opening. This paper presents that the details associated with the finite element analysis of thick/thin-plates (representing the web of hot-rolled steel beam, and the web of a cold-formed steel member) having a large reinforced openings. The study considered thin simply supported rectangular plates subjected to inplane shear loadings until failure (including post-buckling behaviour). The plate was modelled using geometrically non-linear quadrilateral shell elements, and non-linear stress-strain relationship based on experiments. Total Lagrangian (TL) with large displacement/small strain formulation was used for such analysis. The model also considered the initial geometric imperfections. This study considered three reinforcement schemes, namely, flat, lip, and angle reinforcements. This paper discusses the modelling considerations and presents the results associated with the various reinforcement schemes under consideration. The paper briefly compares the analysis results with the experimental results.

Keywords: cold-formed steel, finite element analysis, opening, reinforcement, shear resistance

Procedia PDF Downloads 255

Authors: H. Cano, P. Ferrer, N. Garcia, M. Popovic, J. Zapata

Abstract:

Positive Reinforcement Training is a well-known methodology which has been reported frequently to be used in captive non-human primates. As a matter of fact, it is an invaluable tool for different purposes related with animal welfare, such as primate husbandry and environmental enrichment. It is also essential to perform some cognitive experiments. The main propose of this pilot study was to establish an efficient protocol to train captive olive baboons (Papio anubis). This protocol seems to be vital in the context of a larger research program in which it will be necessary to train a complete population of around 40 baboons. Baboons were studied at the Veterinary Research Farm of the University of Murcia. Temporally isolated animals were trained to perform three basic tasks. Firstly, they were required to take food prices directly from the researchers’ hands. Then a clicker sound or bridge stimulus was added each time the animal acceded to the reinforcement. Finally, they were trained to touch a target, consisted of a whip with a red ball in its end, with their hands or their nose. When the subject completed correctly this task, it was also exposed to the bridge stimulus and awarded with a food price, such as a portion of banana, orange, apple, peach or a raisin. Two protocols were tested during this experiment. In both of them, there were 6 series of 2min training periods each day. However, in the first protocol, the series consisted in 3 trials, whereas in the second one, in each series there were 5 trials. A reliable performance was obtained with only 6 days of training in the case of the 5-trials protocol. However, with the 3-trials one, 26 days of training were needed. As a result, the 5-trials protocol seems to be more effective than the 3-trials one, in order to teach these three basic tasks to olive baboons. In consequence, it will be used to train the rest of the colony.

Keywords: captive primates, olive baboon, positive reinforcement training, Papio anubis, training

Procedia PDF Downloads 94

Authors: Qier Wu, Issam Takla, Thomas Rougelot, Nicolas Burlion

Abstract:

In the framework of French underground disposal of intermediate level radioactive wastes, concrete is widely used as a construction material for containers and tunnels. Drying shrinkage is one of the most disadvantageous phenomena of concrete structures. Cracks generated by differential shrinkage could impair the mechanical behavior, increase the permeability of concrete and act as a preferential path for aggressive species, hence leading to an overall decrease in durability and serviceability. It is of great interest to understand the drying shrinkage phenomenon in order to predict and even to control the strains of concrete. The question is whether the results obtained from laboratory samples are in accordance with the measurements on a real structure. Another question concerns the influence of reinforcement on drying shrinkage of concrete. As part of a global project with Andra (French National Radioactive Waste Management Agency), the present study aims to experimentally investigate the scale effect as well as the influence of reinforcement on the development of drying shrinkage of two high performance concretes (based on CEM I and CEM V cements, according to European standards). Various sizes of samples are chosen, from ordinary laboratory specimens up to real-scale specimens: prismatic specimens with different volume-to-surface (V/S) ratios, thin slices (thickness of 2 mm), cylinders with different sizes (37 and 160 mm in diameter), hollow cylinders, cylindrical columns (height of 1000 mm) and square columns (320×320×1000 mm). The square columns have been manufactured with different reinforcement rates and can be considered as mini-structures, to approximate the behavior of a real voussoir from the waste disposal facility. All the samples are kept, in a first stage, at 20°C and 50% of relative humidity (initial conditions in the tunnel) in a specific climatic chamber developed by the Laboratory of Mechanics of Lille. The mass evolution and the drying shrinkage are monitored regularly. The obtained results show that the specimen size has a great impact on water loss and drying shrinkage of concrete. The specimens with a smaller V/S ratio and a smaller size have a bigger drying shrinkage. The correlation between mass variation and drying shrinkage follows the same tendency for all specimens in spite of the size difference. However, the influence of reinforcement rate on drying shrinkage is not clear based on the present results. The second stage of conservation (50°C and 30% of relative humidity) could give additional results on these influences.

Keywords: concrete, drying shrinkage, mass evolution, reinforcement, scale effect

Procedia PDF Downloads 147

Authors: Abdelhak Soudani

Abstract:

Using the limit equilibrium method in geotechnical field is very important for large projects. This work contributes to the understanding and analysis of the building unstable slopes by the technique of soil nailed with the used of software called GEO-SLOPE calculation based on limit equilibrium method. To achieve our objective, we began a review of the literature on landslides, and techniques of slope stability. Then, we presented a real case slope likely to slip through the realization of the EastWest Highway (M5 stretch between Khemis Miliana and Hoceinia). We also process the application of reinforcement technique nailed soil. The analysis is followed by a parametric study, which shows the impact of parameters given or chosen on various outcomes. Another method of reinforcement (use of micro-piles) has been suggested for improving the stability of the slope

Keywords: slope stability, strengthening, slip, soil nail, GEO-SLOPE

Procedia PDF Downloads 434

Authors: Faycal Benyahia, Abdelmohsen Albedah, Bel Abbes Bachir Bouiadjra

Abstract:

In this paper, the finite element analysis is applied to study the performance of the bonded composite reinforcement or repair for reducing stress concentration at a semi-circular lateral notch and repairing cracks emanating from this kind of notch. The effects of the adhesive properties on the variation of the stress intensity factor at the crack tip were highlighted. The obtained results show that the stress concentration factor at the notch tip is reduced about 30% and the maximal reduction of the stress intensity factor is about 80%. The adhesive properties must be optimized in order to increase the performance of the patch repair or reinforcement.

Keywords: bonded repair, notch, crack, adhesive, composite

Procedia PDF Downloads 360

Authors: J. Novak, A. Kohoutkova, V. Kristek, J. Vodicka, M. Sramek

Abstract:

While the use of cast-in-place concrete for an airfield and highway pavement overlay is very common, the application of precast concrete elements is very limited today. The main reasons consist of high production costs and complex structural behavior. Despite that, several precast concrete systems have been developed and tested with the aim to provide a system with rapid construction. The contribution deals with the reinforcement design of a hexagonal element developed for a proposed airfield pavement system. The sub-base course of the system is composed of compacted recycled concrete aggregates and fiber reinforced concrete with recycled aggregates place on top of it. The selected element belongs to a group of precast concrete elements which are being considered for the construction of a surface course. Both high costs of full-scale experiments and the need to investigate various elements force to simulate their behavior in a numerical analysis software by using finite element method instead of performing expensive experiments. The simulation of the selected element was conducted on a nonlinear model in order to obtain such results which could fully compensate results from experiments. The main objective was to design reinforcement of the precast concrete element subject to quasi-static loading from airplanes with respect to geometrical imperfections, manufacturing imperfections, tensile stress in reinforcement, compressive stress in concrete and crack width. The obtained findings demonstrate that the position and the presence of imperfection in a pavement highly affect the stress distribution in the precast concrete element. The precast concrete element should be heavily reinforced to fulfill all the demands. Using under-reinforced concrete elements would lead to the formation of wide cracks and cracks permanently open.

Keywords: imperfection, numerical simulation, pavement, precast concrete element, reinforcement design, stress analysis

Procedia PDF Downloads 139