Search results for: Reinforcement leaning schedule

Authors: Romisaa Ali

Abstract:

This paper presents a comparison between twin-delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC) reinforcement learning algorithms in the context of training robust navigation policies for Jackal robots. By leveraging an open-source framework and custom motion control environments, the study evaluates the performance, robustness, and transferability of the trained policies across a range of scenarios. The primary focus of the experiments is to assess the training process, the adaptability of the algorithms, and the robot’s ability to navigate in previously unseen environments. Moreover, the paper examines the influence of varying environmental complexities on the learning process and the generalization capabilities of the resulting policies. The results of this study aim to inform and guide the development of more efficient and practical reinforcement learning-based navigation policies for Jackal robots in real-world scenarios.

Keywords: Jackal robot environments, reinforcement learning, TD3, SAC, robust navigation, transferability, custom environment

Procedia PDF Downloads 75

Authors: S. M. T. Mustafa, D. Raes, M. Huysmans

Abstract:

Proper utilization of water resource is very important in agro-based Bangladesh. Irrigation schedule based on local environmental conditions, soil type and water availability will allow a sustainable use of water resources in agriculture. In this study, the FAO crop water model (AquaCrop) was used to simulate the different water and fertilizer management strategies in different location of Bangladesh to obtain a management guideline for the farmer. Model was calibrated and validated for wheat (Triticum aestivum L.). The statistical indices between the observed and simulated grain yields obtained were very good with R2, RMSE, and EF values of 0.92, 0.33, and 0.83, respectively for model calibration and 0.92, 0.68 and 0.77, respectively for model validations. Stem elongation (jointing) to booting and flowering stage were identified as most water sensitive for wheat. Deficit irrigation on water sensitive stage could increase the grain yield for increasing soil fertility levels both for loamy and sandy type soils. Deficit irrigation strategies provides higher water productivity than full irrigation strategies and increase the yield stability (reduce the standard deviation). The practical deficit irrigation schedule for wheat for four different stations and two different soils were designed. Farmer can produce more crops by using deficit irrigation schedule under water stress condition. Practical application and validation of proposed strategies will make them more credible.

Keywords: crop-water model, deficit irrigation, irrigation scheduling, wheat

Procedia PDF Downloads 415

Authors: Hyeon-Seong Kim, Sang-Mi Park, Seul-Gi Kim, Seon-Ju Han, Leen-Seok Kang

Abstract:

The use of BIM, including 4D CAD system, in a construction project is gradually increasing. Since the building construction works repeatedly in the vertical space, it is relatively easy to confirm the interference effect when applying the BIM, but the interference effect for the civil engineering project is relatively small because the civil works perform non-repetitive processes in the horizontal space. For this reason, it is desirable to apply BIM to the construction phase when applying BIM to the civil engineering project, and the most active BIM tool applied to the construction phase is the 4D CAD function for the schedule management. This paper proposes the application procedure of BIM by the construction phase of civil engineering project and a linear 4D CAD construction methodology suitable for the civil engineering project in which linear work is performed.

Keywords: BIM, 4D CAD, linear 4D simulation, VR

Procedia PDF Downloads 385

Authors: Usharani Rathinam

Abstract:

Although policies directing inclusion of marginalized were in effect, majority of chronically impoverished in India belonged to schedule caste and schedule tribes. Also, taking into account that poverty is gendered; destitute women belonged to lower social order whose need is not largely highlighted at policy level. This paper discusses on social relations poverty which highlights on how social order that existed structurally in the society can perpetuate chronic poverty, followed by a critical review on social inclusion policies of India, its merits and demerits in addressing chronic poverty. Multiple case study design is utilized to address this concern in four districts of India; Jhansi, Tikamgarh, Cuddalore and Anantapur. These four districts were selected by purposive sampling based on the criteria; the district should either be categorized as a backward district or should have a history of high poverty rate. Qualitative methods including eighty in-depth interviews, six focus group discussions, six social mapping procedures and three key informant interviews were conducted in 2011, at each of the locations. Analysis of the data revealed that irrespective of gender, schedule castes and schedule tribe participants were found to be chronically poor in all districts. Caste based discrimination is exhibited at both micro and macro levels; village and institutional levels. At village level, lower caste respondents had lesser access to public resources. Also, within institutional settings, due to confiscation, unequal access to resources is noticed, especially in fund distribution. This study found that half of the budget intended for schedule caste and schedule tribes were confiscated by upper caste administrative staffs. This implies that power based on social hierarchy marginalize lower caste participants from accessing better economic, social, and political benefits, that had led them to suffer long term poverty. This study also explored the traditional ties between caste, social structure and bonded labour as a cause of long-term poverty. Though equal access is being emphasized in constitutional rights, issues at micro level have not been reflected in formulation of these rights. Therefore, it is significant for a policy to consider the structural complexity and then focus on issues such as equal distribution of assets and infrastructural facilities that will reduce exclusion and foster long-term security in areas such as employment, markets and public distribution.

Keywords: caste, inclusion policies, India, social order

Procedia PDF Downloads 192

Authors: Tommaso Adamo, Gianpaolo Ghiani, Antonio Domenico Grieco, Emanuela Guerriero

Abstract:

Batch production plants provide a wide range of scheduling problems. In pharmaceutical industries a batch process is usually described by a recipe, consisting of an ordering of tasks to produce the desired product. In this research work we focused on pharmaceutical production processes requiring the culture of a microorganism population (i.e. bacteria, yeasts or antibiotics). Several sources of uncertainty may influence the yield of the culture processes, including (i) low performance and quality of the cultured microorganism population or (ii) microbial contamination. For these reasons, robustness is a valuable property for the considered application context. In particular, a robust schedule will not collapse immediately when a cell of microorganisms has to be thrown away due to a microbial contamination. Indeed, a robust schedule should change locally in small proportions and the overall performance measure (i.e. makespan, lateness) should change a little if at all. In this research work we formulated a constraint programming optimization (COP) model for the robust planning of antibiotics production. We developed a discrete-time model with a multi-criteria objective, ordering the different criteria and performing a lexicographic optimization. A feasible solution of the proposed COP model is a schedule of a given set of tasks onto available resources. The schedule has to satisfy tasks precedence constraints, resource capacity constraints and time constraints. In particular time constraints model tasks duedates and resource availability time windows constraints. To improve the schedule robustness, we modeled the concept of (a, b) super-solutions, where (a, b) are input parameters of the COP model. An (a, b) super-solution is one in which if a variables (i.e. the completion times of a culture tasks) lose their values (i.e. cultures are contaminated), the solution can be repaired by assigning these variables values with a new values (i.e. the completion times of a backup culture tasks) and at most b other variables (i.e. delaying the completion of at most b other tasks). The efficiency and applicability of the proposed model is demonstrated by solving instances taken from Sanofi Aventis, a French pharmaceutical company. Computational results showed that the determined super-solutions are near-optimal.

Keywords: constraint programming, super-solutions, robust scheduling, batch process, pharmaceutical industries

Procedia PDF Downloads 599

Authors: S. Bahadır Yüksel, Alptuğ Ünal

Abstract:

The composite shear walls (CSW) with steel encased profiles can be used as lateral-load resisting systems for buildings that require considerable large lateral-load capacity. The aim of this work is to propose the experimental work conducted on CSW having L section folded plate (L shape steel made-up sections) as longitudinal reinforcement in boundary regions. The study in this paper present the experimental test conducted on CSW having L section folded plate as longitudinal reinforcement in boundary regions. The tested 1/3 geometric scaled CSW has aspect ratio of 3.2. L-shape structural steel materials with 2L-19x57x7mm dimensions were placed in shear wall boundary zones. The seismic behavior of CSW test specimen was investigated by evaluating and interpreting the hysteresis curves, envelope curves, rigidity and consumed energy graphs of this tested element. In addition to this, the experimental results, deformation and cracking patterns were evaluated, interpreted and suggestions of the design recommendations were proposed.

Keywords: shear wall, composite shear wall, boundary reinforcement, earthquake resistant structural design, L section

Procedia PDF Downloads 312

Authors: Jong Woo Kim, Go Bong Choi, Sang Hwan Son, Dae Shik Kim, Jung Chul Suh, Jong Min Lee

Abstract:

The Markov Decision Process (MDP) based methodology is implemented in order to establish the optimal schedule which minimizes the cost. Formulation of MDP problem is presented using the information about the current state of pipe, improvement cost, failure cost and pipe deterioration model. The objective function and detailed algorithm of dynamic programming (DP) are modified due to the difficulty of implementing the conventional DP approaches. The optimal schedule derived from suggested model is compared to several policies via Monte Carlo simulation. Validity of the solution and improvement in computational time are proved.

Keywords: Markov decision processes, dynamic programming, Monte Carlo simulation, periodic replacement, Weibull distribution

Procedia PDF Downloads 404

Authors: S. A. Naeini, H. R. Rahmani, M. Hossein Zade

Abstract:

Many classical bearing capacity theories assume that the natural soil's layers are homogenous for determining the bearing capacity of the soil. But, in many practical projects, we encounter multi-layer soils. Geosynthetic as reinforcement materials have been extensively used in the construction of various structures. In this paper, numerical analysis of the Plate Load Test (PLT) using of ABAQUS software in double-layered soils with different thicknesses of sandy and gravelly layers reinforced with geogrid was considered. The PLT is one of the common filed methods to calculate parameters such as soil bearing capacity, the evaluation of the compressibility and the determination of the Subgrade Reaction module. In fact, the influence of the geogrid layers on the bearing capacity of the layered soils is investigated. Finally, the most appropriate mode for the distance and number of reinforcement layers is determined. Results show that using three layers of geogrid with a distance of 0.3 times the width of the loading plate has the highest efficiency in bearing capacity of double-layer (sand and gravel) soils. Also, the significant increase in bearing capacity between unreinforced and reinforced soil with three layers of geogrid is caused by the condition that the upper layer (gravel) thickness is equal to the loading plate width.

Keywords: bearing capacity, reinforcement, geogrid, plate load test, layered soils

Procedia PDF Downloads 158

Authors: Bindhu Lal, Karnika Kochal

Abstract:

Clayey soil contains clay minerals with traces of metal oxides and organic matter, which exhibits properties like low drainage, high plasticity, and shrinkage. To overcome these issues, various soil reinforcement techniques are used to elevate the stiffness, water tightness, and bearing capacity of the soil. Such techniques include cementation, bituminization, freezing, fiber inclusion, geo-synthetics, nailing, etc. Reinforcement of soil with fibers has been a cost-effective solution to soil improvement problems. An experimental study was undertaken involving the inclusion of cotton waste fibers in clayey soil as reinforcement with different fiber contents (1%, 1.5%, 2%, and 2.5% by weight) and analyzing its effects on the unconfined compressive strength of the soil. Two categories of soil were taken, comprising of natural clay and clay mixed with 5% sodium bentonite by weight. The soil specimens were subjected to proctor compaction and unconfined compression tests. The validated outcome shows that fiber inclusion has a strikingly positive impact on the compressive strength and axial strain at failure of the soil. Based on the commendatory results procured, compressive strength was found to be directly proportional to the fiber content, with the effect being more pronounced at lower water content.

Keywords: bentonite clay, clay, cotton fibers, unconfined compressive strength

Procedia PDF Downloads 162

Authors: Yuling Cui, Danhao Deng, Chaowei Wang, Weidong Wang

Abstract:

In recent years, unmanned aerial vehicles (UAVs) have been widely used in wireless communication, attracting more and more attention from researchers. UAVs can not only serve as a relay for auxiliary communication but also serve as an aerial base station for ground users (GUs). However, limited energy means that they cannot work all the time and cover a limited range of services. In this paper, we investigate 2D UAV trajectory design and power allocation in order to maximize the UAV's service time and downlink throughput. Based on deep reinforcement learning, we propose a depth deterministic strategy gradient algorithm for trajectory design and power distribution (TDPA-DDPG) to solve the energy-efficient and communication service quality problem. The simulation results show that TDPA-DDPG can extend the service time of UAV as much as possible, improve the communication service quality, and realize the maximization of downlink throughput, which is significantly improved compared with existing methods.

Keywords: UAV trajectory design, power allocation, energy efficient, downlink throughput, deep reinforcement learning, DDPG

Procedia PDF Downloads 129

Authors: Li-hsing Shih, Tzu-hsun Yen

Abstract:

Because of the long-term commitment to net zero carbon emission, utility companies include more renewable energy supply, which generates electricity with time and weather restrictions. This leads to time-of-use electricity pricing to reflect the actual cost of energy supply. From an end-user point of view, better residential energy management is needed to incorporate the time-of-use prices and assist end users in scheduling their daily use of electricity. This study uses bi-criteria optimization methods to schedule daily energy consumption by minimizing the electricity cost and maximizing the comfort of end users. Different from most previous research, this study schedules users’ activities rather than household appliances to have better measures of users’ comfort/satisfaction. The relation between each activity and the use of different appliances could be defined by users. The comfort level is at the highest when the time and duration of an activity completely meet the user’s expectation, and the comfort level decreases when the time and duration do not meet expectations. A questionnaire survey was conducted to collect data for establishing regression models that describe users’ comfort levels when the execution time and duration of activities are different from user expectations. Six regression models representing the comfort levels for six types of activities were established using the responses to the questionnaire survey. A computer program is developed to evaluate electricity cost and the comfort level for each feasible schedule and then find the non-dominated schedules. The Epsilon constraint method is used to find the optimal schedule out of the non-dominated schedules. A hypothetical case is presented to demonstrate the effectiveness of the proposed approach and the computer program. Using the program, users can obtain the optimal schedule of daily energy consumption by inputting the intended time and duration of activities and the given time-of-use electricity prices.

Keywords: bi-criteria optimization, energy consumption, time-of-use price, scheduling

Procedia PDF Downloads 43

Authors: Mohamed Haneef, Shanawaz Patil, Syed Zameer, Mohammed Mohsin Ali

Abstract:

The emerging technologies and trends of present generation requires downsizing the unwieldy structures to light weight structures on one hand and integration of varied properties on other hand to meet the application demands. In the present investigation an attempt is made to familiarize and best possibilities of reinforcing agent in aluminum 7075 matrix with naturally occurring beryl (Be) and graphene (Gr) to develop a new hybrid composite material. A stir casting process was used to fabricate with fixed volume fraction of 6wt% weight beryl and various volume fractions of 0.5wt%, 1wt%, 1.5wt% and 2wt% of graphene. The properties such as tensile strength, hardness and dry sliding wear behavior of hybrid composites were examined. The crystallite size and morphology of the graphene and beryl particles were analyzed with X-ray diffraction (XRD) and scanning electron microscopy (SEM) respectively. It was observed that ultimate tensile strength and hardness of the hybrid composite increased with increasing reinforcement volume fraction as compared to specimen without reinforcement additions. The dry sliding wear behavior of the hybrid composites decreases as compared to Al7075 alloy without reinforcement.

Keywords: Al7075, beryl, graphene, TEM, wear

Procedia PDF Downloads 137

Authors: Xinyu Wang

Abstract:

In the context of the rapid development of Internet technology and the increasing prevalence of online social media, this study investigates the impact of digital communication on social learning. Through three behavioral experiments, we explore both affective and cognitive social learning in online environments. Experiment 1 manipulates the content of experimental materials and two forms of feedback, emotional valence, sociability, and repetition, to verify whether individuals can achieve online emotional social learning through reinforcement using two social learning strategies. Results reveal that both social learning strategies can assist individuals in affective, social learning through reinforcement, with feedback-based learning strategies outperforming frequency-dependent strategies. Experiment 2 similarly manipulates the content of experimental materials and two forms of feedback to verify whether individuals can achieve online knowledge social learning through reinforcement using two social learning strategies. Results show that similar to online affective social learning, individuals adopt both social learning strategies to achieve cognitive social learning through reinforcement, with feedback-based learning strategies outperforming frequency-dependent strategies. Experiment 3 simultaneously observes online affective and cognitive social learning by manipulating the content of experimental materials and feedback at different levels of social pressure. Results indicate that online affective social learning exhibits different learning effects under different levels of social pressure, whereas online cognitive social learning remains unaffected by social pressure, demonstrating more stable learning effects. Additionally, to explore the sustained effects of online social learning and differences in duration among different types of online social learning, all three experiments incorporate two test time points. Results reveal significant differences in pre-post-test scores for online social learning in Experiments 2 and 3, whereas differences are less apparent in Experiment 1. To accurately measure the sustained effects of online social learning, the researchers conducted a mini-meta-analysis of all effect sizes of online social learning duration. Results indicate that although the overall effect size is small, the effect of online social learning weakens over time.

Keywords: online social learning, affective social learning, cognitive social learning, social learning strategies, social reinforcement, social pressure, duration

Procedia PDF Downloads 22

Authors: Akshay Paranjape, Nils Plettenberg, Robert Schmitt

Abstract:

Traditional optimization methods like evolutionary algorithms are widely used in production processes to find an optimal or near-optimal solution of control parameters based on the simulated environment space of a process. These algorithms are computationally intensive and therefore do not provide the opportunity for real-time optimization. This paper utilizes the Deep Reinforcement Learning (DRL) framework to find an optimal or near-optimal solution for control parameters. A model based on maximum a posteriori policy optimization (Hybrid-MPO) that can handle both numerical and categorical parameters is used as a benchmark for comparison. A comparative study shows that DRL can find optimal solutions of similar quality as compared to evolutionary algorithms while requiring significantly less time making them preferable for real-time optimization. The results are confirmed in a large-scale validation study on datasets from production and other fields. A trained XGBoost model is used as a surrogate for process simulation. Finally, multiple ways to improve the model are discussed.

Keywords: reinforcement learning, evolutionary algorithms, production process optimization, real-time optimization, hybrid-MPO

Procedia PDF Downloads 96

Authors: O. Vlcek

Abstract:

The paper deals with current issues in the research of advanced methods to increase the reliability of traditional timber structural elements. It analyses the issue of strengthening of bent timber beams, such as ceiling beams in old (historical) buildings with the additional concrete slab in combination with externally bonded fibre-reinforced polymer. The study evaluates deflection of a selected group of timber beams with concrete slab and additional CFRP reinforcement using different calculating methods and observes differences in results from different calculating methods. An elastic calculation method and evaluation with FEM analysis software were used.

Keywords: timber-concrete composite, strengthening, fibre-reinforced polymer, theoretical analysis

Procedia PDF Downloads 301

Authors: Tong He, Long Chen, Irag Mantegh, Wen-Fang Xie

Abstract:

This paper proposes an image-based obstacle avoidance and tracking target identification strategy in GPS-degraded or GPS-denied environment for an Unmanned Aerial Vehicle (UAV). The traditional force algorithm for obstacle avoidance could produce local minima area, in which UAV cannot get away obstacle effectively. In order to eliminate it, an artificial potential approach based on harmonic potential is proposed to guide the UAV to avoid the obstacle by using the vision system. And image-based visual servoing scheme (IBVS) has been adopted to implement the proposed obstacle avoidance approach. In IBVS, the pixel accuracy is a key factor to realize the obstacle avoidance. In this paper, the deep reinforcement learning framework has been applied by reducing pixel errors through constant interaction between the environment and the agent. In addition, the combination of OpenTLD and Tensorflow based on neural network is used to identify the type of tracking target. Numerical simulation in Matlab and ROS GAZEBO show the satisfactory result in target identification and obstacle avoidance.

Keywords: image-based visual servoing, obstacle avoidance, tracking target identification, deep reinforcement learning, artificial potential approach, neural network

Procedia PDF Downloads 128

Authors: Soofiyan Atar, Adil Shaikh, Sahil Rajpurkar, Pragnesh Bhalala, Aniket Desai, Irfan Siddavatam

Abstract:

Deep reinforcement learning (deep RL) algorithms leverage the symbolic power of complex controllers by automating it by mapping sensory inputs to low-level actions. Deep RL eliminates the complex robot dynamics with minimal engineering. Deep RL provides high-risk involvement by directly implementing it in real-world scenarios and also high sensitivity towards hyperparameters. Tuning of hyperparameters on a pneumatic quadruped robot becomes very expensive through trial-and-error learning. This paper presents an automated learning control for a pneumatic quadruped robot using sample efficient deep Q learning, enabling minimal tuning and very few trials to learn the neural network. Long training hours may degrade the pneumatic cylinder due to jerk actions originated through stochastic weights. We applied this method to the pneumatic quadruped robot, which resulted in a hopping gait. In our process, we eliminated the use of a simulator and acquired a stable gait. This approach evolves so that the resultant gait matures more sturdy towards any stochastic changes in the environment. We further show that our algorithm performed very well as compared to programmed gait using robot dynamics.

Keywords: model-based reinforcement learning, gait stability, supervised learning, pneumatic quadruped

Procedia PDF Downloads 295

Authors: N. Attari, S. Amziane, M. Chemrouk

Abstract:

A large number of old buildings have been identified as having potentially critical detailing to resist earthquakes. The main reinforcement of lap-spliced columns just above the joint region, discontinuous bottom beam reinforcement, and little or no joint transverse reinforcement are the most critical details of interior beam column joints in such buildings. This structural type constitutes a large share of the building stock, both in developed and developing countries, and hence it represents a substantial exposure. Direct observation of damaged structures, following the Algiers 2003 earthquake, has shown that damage occurs usually at the beam-column joints, with failure in bending or shear, depending on geometry and reinforcement distribution and type. While substantial literature exists for the design of concrete frame joints to withstand this type of failure, after the earthquake many structures were classified as slightly damaged and, being uneconomic to replace them, at least in the short term, suitable means of repairs of the beam column joint area are being studied. Furthermore; there exists a large number of buildings that need retrofitting of the joints before the next earthquake. The paper reports the results of the experimental programme, constituted of three beam-column reinforced concrete joints at a scale of one to three (1/3) tested under the effect of a pre-stressing axial load acting over the column. The beams were subjected at their ends to an alternate cyclic loading under displacement control to simulate a seismic action. Strain and cracking fields were monitored with the help a digital recording camera. Following the analysis of the results, a comparison can be made between the performances in terms of ductility, strength and mode of failure of the different strengthening solution considered.

Keywords: fibre reinforced polymers, joints, reinforced concrete, beam columns

Procedia PDF Downloads 398

Authors: Ahmed Shaban Abdel Hay Gabr

Abstract:

Water hyacinth (W-H) has an adverse effect on Nile river in Egypt, it absorbs high quantities of water, it needs to serve these quantities especially at this time, so by burning W-H, it can be used in concrete mix to reduce the permeability of concrete and increase both the compressive and splitting strength. The effect of W-H on non-structural concrete properties was studied, but there is a lack of studies about the behavior of structural concrete containing W-H. Therefore, in the present study, the behavior of 15 RC beams with 100 x 150 mm cross section, 1250 mm span, different reinforcement ratios and different W-H ratios were studied by testing the beams under two-point bending test. The test results showed that Water Hyacinth is compatible with RC which yields promising results.

Keywords: beams, reinforcement ratio, reinforced concrete, water hyacinth

Procedia PDF Downloads 435

Authors: Xiaohua Zou, Yongxin Su

Abstract:

The reasonable operation of heating, ventilation, and air conditioning (HVAC) is of great significance in improving the security, stability, and economy of power system operation. However, the uncertainty of the operating environment, thermal comfort varies by users and rapid decision-making pose challenges for HVAC demand response optimization. In this regard, this paper proposes a reinforcement learning-based method for HVAC demand response optimization considering few-shot personalized thermal comfort (PTC). First, an HVAC DR optimization framework based on few-shot PTC model and DRL is designed, in which the output of few-shot PTC model is regarded as the input of DRL. Then, a few-shot PTC model that distinguishes between awake and asleep states is established, which has excellent engineering usability. Next, based on soft actor criticism, an HVAC DR optimization algorithm considering the user’s PTC is designed to deal with uncertainty and make decisions rapidly. Experiment results show that the proposed method can efficiently obtain use’s PTC temperature, reduce energy cost while ensuring user’s PTC, and achieve rapid decision-making under uncertainty.

Keywords: HVAC, few-shot personalized thermal comfort, deep reinforcement learning, demand response

Procedia PDF Downloads 58

Authors: Wiem Zemzem, Moncef Tagina

Abstract:

In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new action selection strategy, called ε-greedy-MPA (the ε-greedy policy favoring the most promising actions) is proposed. Unlike existing exploration/exploitation policies (EEPs) such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. Finally, various simulations with ball-catching problem have been conducted to evaluate the ε-greedy-MPA policy. The results of simulated experiments show that combining this policy with the Qlearning method is more effective and efficient compared with the ε-greedy policy in stationary environments and the utility-based reinforcement learning approach in non stationary environments.

Keywords: autonomous mobile robot, exploration/ exploitation policy, large, dynamic environment, reinforcement learning

Procedia PDF Downloads 402

Authors: Mohd Ritzman Abdul Karim, Zhaohui Huang

Abstract:

A current focus in seismic engineering practice is the development of seismic design approach that focuses on the performance-based design. Performance-based design aims to design the structures to achieve specified performance based on the damage limit states. This damage limit is more restrictive limit than life safety and needs to be carefully estimated to avoid damage in piers due to failure in transverse reinforcement. In this paper, a different perspective of damage limit states has been explored by integrating two damage control material limit state, concrete and reinforcement by introduced parameters such as expected yield stress of transverse reinforcement where peak tension strain prior to bar buckling is introduced in a recent study. The different perspective of damage limit states with modified yield displacement and the modified plastic-hinge length is used in order to predict damage-control target displacement for reinforced concreate (RC) bridge pier. Three-dimensional (3D) finite element (FE) model has been developed for estimating damage target displacement to validate proposed damage limit states. The result from 3D FE analysis was validated with experimental study found in the literature. The validated model then was applied to predict the damage target displacement for RC bridge pier and to validate the proposed study. The tensile strain on reinforcement and compression on concrete were used to determine the predicted damage target displacement and compared with the proposed study. The result shows that the proposed damage limit states were efficient in predicting damage-control target displacement consistent with FE simulations.

Keywords: damage-control target displacement, damage limit states, reinforced concrete bridge pier, yield displacement

Procedia PDF Downloads 141

Authors: K. S. Sivakumaran, Bo Chen

Abstract:

The floor beams of steel buildings, cold-formed steel floor joists, in particular, often require large web openings, which may affect their shear capacities. A cost effective way to mitigate the detrimental effects of such openings is to weld/fasten reinforcements. A difficulty associated with an experimental investigation to establish suitable reinforcement schemes for openings in shear zone is that moment always coexists with the shear, and thus, it is impossible to create pure shear state in experiments, resulting in moment influenced results. However, finite element analysis can be conveniently used to investigate the pure shear behaviour of webs including webs with reinforced opening. This paper presents that the details associated with the finite element analysis of thick/thin-plates (representing the web of hot-rolled steel beam, and the web of a cold-formed steel member) having a large reinforced openings. The study considered thin simply supported rectangular plates subjected to inplane shear loadings until failure (including post-buckling behaviour). The plate was modelled using geometrically non-linear quadrilateral shell elements, and non-linear stress-strain relationship based on experiments. Total Lagrangian (TL) with large displacement/small strain formulation was used for such analysis. The model also considered the initial geometric imperfections. This study considered three reinforcement schemes, namely, flat, lip, and angle reinforcements. This paper discusses the modelling considerations and presents the results associated with the various reinforcement schemes under consideration. The paper briefly compares the analysis results with the experimental results.

Keywords: cold-formed steel, finite element analysis, opening, reinforcement, shear resistance

Procedia PDF Downloads 272

Authors: H. Cano, P. Ferrer, N. Garcia, M. Popovic, J. Zapata

Abstract:

Positive Reinforcement Training is a well-known methodology which has been reported frequently to be used in captive non-human primates. As a matter of fact, it is an invaluable tool for different purposes related with animal welfare, such as primate husbandry and environmental enrichment. It is also essential to perform some cognitive experiments. The main propose of this pilot study was to establish an efficient protocol to train captive olive baboons (Papio anubis). This protocol seems to be vital in the context of a larger research program in which it will be necessary to train a complete population of around 40 baboons. Baboons were studied at the Veterinary Research Farm of the University of Murcia. Temporally isolated animals were trained to perform three basic tasks. Firstly, they were required to take food prices directly from the researchers’ hands. Then a clicker sound or bridge stimulus was added each time the animal acceded to the reinforcement. Finally, they were trained to touch a target, consisted of a whip with a red ball in its end, with their hands or their nose. When the subject completed correctly this task, it was also exposed to the bridge stimulus and awarded with a food price, such as a portion of banana, orange, apple, peach or a raisin. Two protocols were tested during this experiment. In both of them, there were 6 series of 2min training periods each day. However, in the first protocol, the series consisted in 3 trials, whereas in the second one, in each series there were 5 trials. A reliable performance was obtained with only 6 days of training in the case of the 5-trials protocol. However, with the 3-trials one, 26 days of training were needed. As a result, the 5-trials protocol seems to be more effective than the 3-trials one, in order to teach these three basic tasks to olive baboons. In consequence, it will be used to train the rest of the colony.

Keywords: captive primates, olive baboon, positive reinforcement training, Papio anubis, training

Procedia PDF Downloads 105

Authors: Qier Wu, Issam Takla, Thomas Rougelot, Nicolas Burlion

Abstract:

In the framework of French underground disposal of intermediate level radioactive wastes, concrete is widely used as a construction material for containers and tunnels. Drying shrinkage is one of the most disadvantageous phenomena of concrete structures. Cracks generated by differential shrinkage could impair the mechanical behavior, increase the permeability of concrete and act as a preferential path for aggressive species, hence leading to an overall decrease in durability and serviceability. It is of great interest to understand the drying shrinkage phenomenon in order to predict and even to control the strains of concrete. The question is whether the results obtained from laboratory samples are in accordance with the measurements on a real structure. Another question concerns the influence of reinforcement on drying shrinkage of concrete. As part of a global project with Andra (French National Radioactive Waste Management Agency), the present study aims to experimentally investigate the scale effect as well as the influence of reinforcement on the development of drying shrinkage of two high performance concretes (based on CEM I and CEM V cements, according to European standards). Various sizes of samples are chosen, from ordinary laboratory specimens up to real-scale specimens: prismatic specimens with different volume-to-surface (V/S) ratios, thin slices (thickness of 2 mm), cylinders with different sizes (37 and 160 mm in diameter), hollow cylinders, cylindrical columns (height of 1000 mm) and square columns (320×320×1000 mm). The square columns have been manufactured with different reinforcement rates and can be considered as mini-structures, to approximate the behavior of a real voussoir from the waste disposal facility. All the samples are kept, in a first stage, at 20°C and 50% of relative humidity (initial conditions in the tunnel) in a specific climatic chamber developed by the Laboratory of Mechanics of Lille. The mass evolution and the drying shrinkage are monitored regularly. The obtained results show that the specimen size has a great impact on water loss and drying shrinkage of concrete. The specimens with a smaller V/S ratio and a smaller size have a bigger drying shrinkage. The correlation between mass variation and drying shrinkage follows the same tendency for all specimens in spite of the size difference. However, the influence of reinforcement rate on drying shrinkage is not clear based on the present results. The second stage of conservation (50°C and 30% of relative humidity) could give additional results on these influences.

Keywords: concrete, drying shrinkage, mass evolution, reinforcement, scale effect

Procedia PDF Downloads 159

Authors: Abdelhak Soudani

Abstract:

Using the limit equilibrium method in geotechnical field is very important for large projects. This work contributes to the understanding and analysis of the building unstable slopes by the technique of soil nailed with the used of software called GEO-SLOPE calculation based on limit equilibrium method. To achieve our objective, we began a review of the literature on landslides, and techniques of slope stability. Then, we presented a real case slope likely to slip through the realization of the EastWest Highway (M5 stretch between Khemis Miliana and Hoceinia). We also process the application of reinforcement technique nailed soil. The analysis is followed by a parametric study, which shows the impact of parameters given or chosen on various outcomes. Another method of reinforcement (use of micro-piles) has been suggested for improving the stability of the slope

Keywords: slope stability, strengthening, slip, soil nail, GEO-SLOPE

Procedia PDF Downloads 447

Authors: Faycal Benyahia, Abdelmohsen Albedah, Bel Abbes Bachir Bouiadjra

Abstract:

In this paper, the finite element analysis is applied to study the performance of the bonded composite reinforcement or repair for reducing stress concentration at a semi-circular lateral notch and repairing cracks emanating from this kind of notch. The effects of the adhesive properties on the variation of the stress intensity factor at the crack tip were highlighted. The obtained results show that the stress concentration factor at the notch tip is reduced about 30% and the maximal reduction of the stress intensity factor is about 80%. The adhesive properties must be optimized in order to increase the performance of the patch repair or reinforcement.

Keywords: bonded repair, notch, crack, adhesive, composite

Procedia PDF Downloads 376

Authors: J. Novak, A. Kohoutkova, V. Kristek, J. Vodicka, M. Sramek

Abstract:

While the use of cast-in-place concrete for an airfield and highway pavement overlay is very common, the application of precast concrete elements is very limited today. The main reasons consist of high production costs and complex structural behavior. Despite that, several precast concrete systems have been developed and tested with the aim to provide a system with rapid construction. The contribution deals with the reinforcement design of a hexagonal element developed for a proposed airfield pavement system. The sub-base course of the system is composed of compacted recycled concrete aggregates and fiber reinforced concrete with recycled aggregates place on top of it. The selected element belongs to a group of precast concrete elements which are being considered for the construction of a surface course. Both high costs of full-scale experiments and the need to investigate various elements force to simulate their behavior in a numerical analysis software by using finite element method instead of performing expensive experiments. The simulation of the selected element was conducted on a nonlinear model in order to obtain such results which could fully compensate results from experiments. The main objective was to design reinforcement of the precast concrete element subject to quasi-static loading from airplanes with respect to geometrical imperfections, manufacturing imperfections, tensile stress in reinforcement, compressive stress in concrete and crack width. The obtained findings demonstrate that the position and the presence of imperfection in a pavement highly affect the stress distribution in the precast concrete element. The precast concrete element should be heavily reinforced to fulfill all the demands. Using under-reinforced concrete elements would lead to the formation of wide cracks and cracks permanently open.

Keywords: imperfection, numerical simulation, pavement, precast concrete element, reinforcement design, stress analysis

Procedia PDF Downloads 151

Authors: Rahul Paul, Kedar Nath Das

Abstract:

The field of machine learning (ML) is experiencing rapid development, resulting in a multitude of theoretical advancements and extensive practical implementations across various disciplines. The objective of ML is to facilitate the ability of machines to perform cognitive tasks by leveraging knowledge gained from prior experiences and effectively addressing complex problems, even in situations that deviate from previously encountered instances. Reinforcement Learning (RL) has emerged as a prominent subfield within ML and has gained considerable attention in recent times from researchers. This surge in interest can be attributed to the practical applications of RL, the increasing availability of data, and the rapid advancements in computing power. At the same time, optimization algorithms play a pivotal role in the field of ML and have attracted considerable interest from researchers. A multitude of proposals have been put forth to address optimization problems or improve optimization techniques within the domain of ML. The necessity of a thorough examination and implementation of optimization algorithms within the context of ML is of utmost importance in order to provide guidance for the advancement of research in both optimization and ML. This article provides a comprehensive overview of the application of metaheuristic evolutionary optimization algorithms in conjunction with RL to address a diverse range of scientific challenges. Furthermore, this article delves into the various challenges and unresolved issues pertaining to the optimization of RL models.

Keywords: machine learning, reinforcement learning, loss function, evolutionary optimization techniques

Procedia PDF Downloads 58

Authors: Joanne Hardman

Abstract:

South Africa’s crisis in mathematics attainment is well documented. To meet the need to develop students’ mathematical performance in schools the government has launched various initiatives using computers to impact on mathematical attainment. While it is clear that computers can change pedagogical practices, there is a dearth of qualitative studies indicating exactly how pedagogy is transformed with Information Communication Technologies (ICTs) in a teaching activity. Consequently, this paper addresses the following question: how, along which dimensions in an activity, does pedagogy alter with the use of computer drill and practice software in four disadvantaged grade 6 mathematics classrooms in the Western Cape province of South Africa? The paper draws on Cultural Historical Activity Theory (CHAT) to develop a view of pedagogy as socially situated. Four ideal pedagogical types are identified: Reinforcement pedagogy, which has the reinforcement of specialised knowledge as its object; Collaborative pedagogy, which has the development of metacognitive engagement with specialised knowledge as its object; Directive pedagogy, which has the development of technical task skills as its object, and finally, Defensive pedagogy, which has student regulation as its object. Face-to-face lessons were characterised as predominantly Reinforcement and Collaborative pedagogy and most computer lessons were characterised as mainly either Defensive or Directive.

Keywords: computers, cultural historical activity theory, mathematics, pedagogy

Procedia PDF Downloads 265