Search results for: robot learning

Authors: Nihal Murali, Kunal Gupta, Surekha Bhanot

Abstract:

Training of artificial neural networks (ANNs) using reinforcement learning (RL) techniques is being widely discussed in the robot learning literature. The high model complexity of ANNs along with the model-free nature of RL algorithms provides a desirable combination for many robotics applications. There is a huge need for algorithms that generalize using raw sensory inputs, such as vision, without any hand-engineered features or domain heuristics. In this paper, the standard control problem of line following robot was used as a test-bed, and an ANN controller for the robot was trained on images from a live video feed using Q-learning. A virtual agent was first trained in simulation environment and then deployed onto a robot’s hardware. The robot successfully learns to traverse a wide range of curves and displays excellent generalization ability. Qualitative analysis of the evolution of policies, performance and weights of the network provide insights into the nature and convergence of the learning algorithm.

Keywords: artificial neural networks, q-learning, reinforcement learning, robot learning

Procedia PDF Downloads 352

Authors: Delowar Hossain, Genci Capi

Abstract:

This paper concerns with the problem of deep learning parameters tuning using a genetic algorithm (GA) in order to improve the performance of deep learning (DL) method. We present a GA based DL method for robot object recognition and grasping. GA is used to optimize the DL parameters in learning procedure in term of the fitness function that is good enough. After finishing the evolution process, we receive the optimal number of DL parameters. To evaluate the performance of our method, we consider the object recognition and robot grasping tasks. Experimental results show that our method is efficient for robot object recognition and grasping.

Keywords: deep learning, genetic algorithm, object recognition, robot grasping

Procedia PDF Downloads 329

Authors: Syamimi Shamsuddin, Hanafiah Yussof, Fazah Akhtar Hanapiah, Salina Mohamed, Nur Farah Farhan Jamil, Farhana Wan Yunus

Abstract:

Robot-based intervention for children with autism is an evolving research niche in human-robot interaction (HRI). Recent studies in this area mostly covered the role of robots in the clinical and experimental setting. Our previous work had shown that interaction with a robot pose no adverse effects on the children. Also, the presence of the robot, together with specific modules of interaction was associated with less autistic behavior. Extending this impact on school-going children, interactions that are in-tune with special education lessons are needed. This methodological paper focuses on how a robot can be incorporated in a current learning environment for autistic children. Six interaction scenarios had been designed based on the existing syllabus to teach communication skills, using the Applied Behavior Analysis (ABA) technique as the framework. Development of the robotic experience in class also covers the required set-up involving participation from teachers. The actual research conduct involving autistic children, teachers and robot shall take place in the next phase.

Keywords: autism spectrum disorder, ASD, humanoid robot, communication skills, robot-assisted learning

Procedia PDF Downloads 342

Authors: Soofiyan Atar, Adil Shaikh, Sahil Rajpurkar, Pragnesh Bhalala, Aniket Desai, Irfan Siddavatam

Abstract:

Deep reinforcement learning (deep RL) algorithms leverage the symbolic power of complex controllers by automating it by mapping sensory inputs to low-level actions. Deep RL eliminates the complex robot dynamics with minimal engineering. Deep RL provides high-risk involvement by directly implementing it in real-world scenarios and also high sensitivity towards hyperparameters. Tuning of hyperparameters on a pneumatic quadruped robot becomes very expensive through trial-and-error learning. This paper presents an automated learning control for a pneumatic quadruped robot using sample efficient deep Q learning, enabling minimal tuning and very few trials to learn the neural network. Long training hours may degrade the pneumatic cylinder due to jerk actions originated through stochastic weights. We applied this method to the pneumatic quadruped robot, which resulted in a hopping gait. In our process, we eliminated the use of a simulator and acquired a stable gait. This approach evolves so that the resultant gait matures more sturdy towards any stochastic changes in the environment. We further show that our algorithm performed very well as compared to programmed gait using robot dynamics.

Keywords: model-based reinforcement learning, gait stability, supervised learning, pneumatic quadruped

Procedia PDF Downloads 288

Authors: Jingwen Shan

Abstract:

In order to explore the key factors affecting the robot program learning intention of school students, this study takes the technology acceptance model as the theoretical basis and invites 167 students from Jiading District of Shanghai as the research subjects. In the robot course, the model of school students on their learning behavior is constructed. By verifying the causal path relationship between variables, it is concluded that teachers can enhance students’ perceptual usefulness to robotics courses by enhancing subjective norms, entertainment perception, and reducing technical anxiety, such as focusing on the gradual progress of programming and analyzing learner characteristics. Students can improve perceived ease of use by enhancing self-efficacy. At the same time, robot hardware designers can optimize in terms of entertainment and interactivity, which will directly or indirectly increase the learning intention of the robot course. By changing these factors, the learning behavior of primary and secondary school students can be more sustainable.

Keywords: TAM, learning behavior intentions, robot courses, primary and secondary school students

Procedia PDF Downloads 129

Authors: Hussein Altartouri

Abstract:

In this paper the kinematic and kinetic models of an omnidirectional holonomic mobile robot is presented. The kinematic and kinetic models form the OmniDrive model. Therefore, a mathematical model for the robot equipped with three- omnidirectional wheels is derived. This model which takes into consideration the kinematics and kinetics of the robot, is developed to state space representation. Relative analysis of the velocities and displacements is used for the kinematics of the robot. Lagrange’s approach is considered in this study for deriving the equation of motion. The drive train and the mechanical assembly only of the Festo Robotino® is considered in this model. Mainly the model is developed for motion control. Furthermore, the model can be used for simulation purposes in different virtual environments not only Robotino® View. Further use of the model is in the mechatronics research fields with the aim of teaching and learning the advanced control theories.

Keywords: mobile robot, omni-direction wheel, mathematical model, holonomic mobile robot

Procedia PDF Downloads 567

Authors: Ahmed Amin Mousa, Tamer M. Ismail, M. Abd El Salam

Abstract:

This work presents a robot called Conceptual Robotic Cube, CR-Cube. The robot can be used as an educational tool for children from the age of three. It has a cube shape attached with a camera colours sensor. In addition, it contains four wheels to move smoothly. The researchers prepared a questionnaire to measure the efficiency of the robot. The design and the questionnaire was presented to 11 experts who agreed that the robot is appropriate for learning numbering and colours for preschool children.

Keywords: CR-Cube, robotic cube, conceptual robot, conceptual cube, colour concept, early childhood education

Procedia PDF Downloads 389

Authors: Ruth Manzanares, Victor Honores, Hugo Zapata, Javier Cansaya, Deivid Yavar, Junior Meza

Abstract:

LogiSum is a robot focused on education like a solution to the ecological crisis. This robot allows reducing the pollution on the beaches by stimulating environmental awareness of not contaminating through the collection of waste. Through the use of the methodology of design thinking, it is intended to reinforce values in adults and with a greater focus on children, so as not to contaminate the beaches. The goal is to encourage the use of the container of the robot LogiSum to put the garbage, with visual interaction and simulation of dialogue with the function of the robot. The results obtained of the testings of the interaction of children with the robot showed an encouraging behavior. With the robot, children left the waste in the right places and not bury it in the sand or in the floor.

Keywords: interaction human-robot, pollution reduction, social robot, robot container, beach pollution

Procedia PDF Downloads 243

Authors: Sahil Anand, John Luetke, Nikhil Venkatesh, Dorothy Wong

Abstract:

Trust plays an important role in building and sustaining long-term relationships between people. In this paper, we present a robot that communicates using nonverbal behaviors such as facial expressions and body movements. Our study reports on an experiment in which participants were asked to team up with the robot to perform specific tasks. We varied the expressivity of the robot and measured the effects on trust, quality of interactions as well as on the praising and punishing behavior of the participant towards the robot. We found that participants developed a stronger affinity towards the expressive robot, but did not show any significant differences in the level of trust. When the same robot made mistakes, participants unconsciously punished it with lesser intensity compared to the neutral robot. The results emphasize the role of expressive behaviors on participant’s perception of the robot and also on the quality of interactions between humans and robots.

Keywords: human-robot interaction, nonverbal communication, relationships, social robot, trust

Procedia PDF Downloads 340

Authors: Sultan Muhammed Fatih Apaydin

Abstract:

In this study, two types of robot were examined as being pioneer robot and follower robot for improving of the capabilities of tracking robots. Robots continue to tracking each other and measurement of the follow-up distance between them is very important for improvements to be applied. It was achieved that the follower robot follows the pioneer robot in line with intended goals. The tests were applied to the robots in various grounds and environments in point of performance and necessary improvements were implemented by measuring the results of these tests.

Keywords: mobile robot, remote and autonomous control, infra-red sensors, arduino

Procedia PDF Downloads 547

Authors: Murtaza Hassan Sheikh, Ahmed A. A. AlSaleh, Naser H. N. Jasem

Abstract:

Kuwait has not yet made its mark in the world of technology and research. Therefore, advancements have been made to fill in this gap. Since Robotics covers a wide variety of fields and helps innovation, efforts have been made to promote its education. Despite of the efforts made in Kuwait, robotics education is still on hold. The paper discusses the issues and obstacles in the implementation of robotics education in Kuwait and how a robotics kit “E-Robot” is making an impact in the Kuwait’s future education and innovation. Problems such as robotics competitions rather than education, complexity of robot programming and lack of organized open source platform are being addressed by the introduction of the E-Robot Kit in Kuwait. Due to its success since 2012 a total of 15 schools have accepted the Kit as a core subject, with 200 teaching it as an extracurricular activity.

Keywords: robotics education, Kuwait's education, e-robot kit, research and development, innovation and creativity

Procedia PDF Downloads 390

Authors: Fang Tang, Jacob Longazo

Abstract:

This paper describes a sliding autonomy approach for coordinating a team of robots to assist the human operator to accomplish tasks while adapting to new or unexpected situations by requesting help from the human operator. While sliding autonomy has been well studied in the context of controlling a single robot. Much work needs to be done to apply sliding autonomy to a multi-robot team, especially human-robot team. Our approach aims at a hierarchical sliding control structure, with components that support human-robot collaboration. We validated our approach in the USARSim simulation and demonstrated that the human-robot team's overall performance can be improved under the sliding autonomy control.

Keywords: sliding autonomy, multi-robot team, human-robot collaboration, USARSim

Procedia PDF Downloads 519

Authors: Sandra Baßler, Peter Dünow, Mathias Marquardt

Abstract:

A Fourier series based learning control (FSBLC) algorithm for tracking trajectories of mechanical systems with unknown nonlinearities is presented. Two processes are introduced to which the FSBLC with PD controller is applied. One is a simplified service robot capable of climbing stairs due to special wheels and the other is a propeller driven pendulum with nearly the same requirements on control. Additionally to the investigation of learning the feed forward for the desired trajectories some considerations on the implementation of such an algorithm on low cost microcontroller hardware are made. Simulations of the service robot as well as practical experiments on the pendulum show the capability of the used FSBLC algorithm to perform the task of improving control behavior for repetitive task of such mechanical systems.

Keywords: climbing stairs, FSBLC, ILC, service robot

Procedia PDF Downloads 287

Authors: Luis F. Recalde, Daniela A. Bastidas, Dayana E. Gallegos, Patricia N. Constante, Victor H. Andaluz

Abstract:

This article presents a non-immersive virtual lab-oratory to emulate the behavior of the Mitsubishi Melfa RV 2SDB robotic arm, allowing students and users to acquire skills and experience related to real robots, augmenting the access and learning of robotics in Universidad de las Fuerzas Armadas (ESPE). It was developed using the mathematical model of the robotic arm, thus defining the parameters for virtual recreation. The environment, interaction, and behavior of the robotic arm were developed in a graphic engine (Unity3D) to emulate learning tasks such as in a robotics laboratory. In the virtual system, four inputs were developed for the movement of the robot arm; further, to program the robot, a user interface was created where the user selects the trajectory such as point to point, line, arc, or circle. Finally, the hypothesis of the industrial robotic learning process is validated through the level of knowledge acquired after using the system.

Keywords: virtual learning, robot arm, non-immersive reality, mathematical model

Procedia PDF Downloads 74

Authors: Masud Karim, Md. Solaiman Mia, Saifuddin Md. Tareeq, Md. Hasanuzzaman

Abstract:

Autism Spectrum Disorder (ASD) is a neurodevelopment disorder. Such disorder is found in childhood life. Children with ASD have less capabilities in communication and social skills. Therapies are used to develop communication and social skills. Recently researchers have been trying to use robots in such therapies. In this paper, we have presented social skill learning test cases for children with ASD. Autism conditions are measured in 30 children in a special school. Among them, twelve children are selected who have equal ASD conditions. Then six children participated in training with humans, and another six children participated in training with robots. The learning session continued for one week and three hours each day. We have taken an assessment test before the learning sessions. After completing the learning sessions, we have taken another assessment test. We have found better performances from children who have participated in robotic sessions rather than the children who have participated in human sessions.

Keywords: children with ASD, NAO robot, human-robot interaction, social skills

Procedia PDF Downloads 61

Authors: Hazim Abdulsada

Abstract:

The inspection of underneath vehicle system has been given significant attention by governments after the threat of terrorism become more prevalent. New technologies such as mobile robots and computer vision are led to have more secure environment. This paper proposed that a mobile robot like Aria robot can be used to search and inspect the bombs under parking a lot vehicle. This robot is using fuzzy logic and subsumption algorithms to control the robot that movies underneath the vehicle. An OpenCV library and laser Hokuyo are added to Aria robot to complete the experiment for under vehicle inspection. This experiment was conducted at the indoor environment to demonstrate the efficiency of our methods to search objects and control the robot movements under vehicle. We got excellent results not only by controlling the robot movement but also inspecting object by the robot camera at same time. This success allowed us to know the requirement to construct a new cost effective robot with more functionality.

Keywords: fuzzy logic, mobile robots, Opencv, subsumption, under vehicle inspection

Procedia PDF Downloads 450

Authors: Jhu-Wei Ji, Fa-Shian Chang, Lih-Tyng Hwang, Chih-Feng Liu, Jeng-Nan Lee, Shun-Min Wang, Kai-Yi Cho

Abstract:

This paper presents a tracked robot with blade arms powered to assist movement in difficult environments. As a result, the tracked robot is able to pass a ramp or climb stairs. The main feature is a pair of blade arms on both sides of the vehicle body working in collaboration with previously validated transformable track system. When the robot encounters an obstacle in a terrain, it enlists the blade arms with power to overcome the obstacle. In disaster areas, there usually will be terrains that are full of broken and complicated slopes, broken walls, rubbles, and ditches. Thereupon, a robot, which is instructed to pass through such disaster areas, needs to have a good off-road capability for such complicated terrains. The robot with crawling-assisting blade arms would overcome the obstacles along the terrains, and possibly become to be a rescue robot. A prototype has been developed and built; experiments were carried out to validate the enhanced crawling capability of the robot.

Keywords: tracked robot, rescue robot, blade arm, crawling ability, control system

Procedia PDF Downloads 390

Authors: Rahul Arora, S. S. Dhami

Abstract:

This paper implements the design structure of industrial robot along with the different transmission components like gear assembly and analysis of complete industrial robot. In this paper, it gives the overview on the most efficient types of modeling and different analysis results that can be obtained for an industrial robot. The investigation is executed in regards to two classifications i.e. the deformation and the stress tests. SolidWorks is utilized to design and review the 3D drawing plan while ANSYS Workbench is utilized to execute the FEA on an industrial robot and the designed component. The CAD evaluation was conducted on a disentangled model of an industrial robot. The study includes design and drafting its transmission system. In CAE study static, modal and dynamic analysis are presented. Every one of the outcomes is divided in regard with the impact of the static and dynamic analysis on the situating exactness of the robot. It gives critical data with respect to parts of the industrial robot that are inclined to harm under higher high force applications. Therefore, the mechanical structure under different operating conditions can help in optimizing the manipulator geometry and in selecting the right material for the same. The FEA analysis is conducted for four different materials on the same industrial robot and gear assembly.

Keywords: CAD, CAE, FEA, robot, static, dynamic, modal, gear assembly

Procedia PDF Downloads 347

Authors: M. Aguilar, D. L. Araujo, A. L. Avendaño, D. C. Flores, I. Lascurain, R. A. Molina, M. Romero

Abstract:

This work presents an application of social robotics, specifically the use of a NAO Robot as a tool for therapists in the treatment of Autism Spectrum Disorder (ASD). According to this, therapies approved by specialist psychologists have been developed and implemented, focusing on creating a triangulation between the robot, the child, and the therapist, aiming to improve their social and language skills, as well as communication skills and joint attention. In addition, quantitative and qualitative analysis tools have been developed and applied to prove the acceptance and the impact of the robot in the treatment of ASD.

Keywords: autism spectrum disorder, NAO robot, social and language skills, therapy

Procedia PDF Downloads 91

Authors: G. Gurguze, I. Turkoglu

Abstract:

Today, the developing features of technological tools with limited energy resources have made it necessary to use energy efficiently. Energy management techniques have emerged for this purpose. As with every field, energy management is vital for robots that are being used in many areas from industry to daily life and that are thought to take up more spaces in the future. Particularly, effective power management in autonomous and multi robots, which are getting more complicated and increasing day by day, will improve the performance and success. In this study, robot management algorithms, usage of renewable and hybrid energy sources, robot motion patterns, robot designs, sharing strategies of workloads in multiple robots, road and mission planning algorithms are discussed for efficient use of energy resources by mobile robots. These techniques have been evaluated in terms of efficient use of existing energy resources and energy management in robots.

Keywords: energy management, mobile robot, robot administration, robot management, robot planning

Procedia PDF Downloads 245

Authors: Roya Khajepour, Alireza B. Novinzadeh

Abstract:

In this paper, the spherical robot is modeled using the Band-Graph approach. This breed of robots is typically employed in expedition missions to unknown territories. Its motion mechanism is based on convection of a fluid in a set of three donut vessels, arranged orthogonally in space. This robot is a non-linear, non-holonomic system. This paper utilizes the Band-Graph technique to derive the torque generation mechanism in a spherical robot. Eventually, this paper describes the motion of a sphere due to the exerted torque components.

Keywords: spherical robot, Band-Graph, modeling, torque

Procedia PDF Downloads 317

Authors: Tianyi Wang, Hieyong Jeong, An Guo, Yuko Ohno

Abstract:

Healthcare robot for assisting sit-to-stand (STS) performance had aroused numerous research interests. To author’s best knowledge, knowledge about how evaluating healthcare robot is still unknown. Robot should be labeled as fault if users feel demanding during STS when they are assisted by robot. In this research, we aim to propose a method to evaluate sit-to-stand assist robot through center of pressure (CoP), then classify different STS performance. Experiments were executed five times with ten healthy subjects under four conditions: two self-performed STSs with chair heights of 62 cm and 43 cm, and two robot-assisted STSs with chair heights of 43 cm and robot end-effect speed of 2 s and 5 s. CoP was measured using a Wii Balance Board (WBB). Bayesian classification was utilized to classify STS performance. The results showed that faults occurred when decreased the chair height and slowed robot assist speed. Proposed method for fault classification showed high probability of classifying fault classes form others. It was concluded that faults for STS assist robot could be detected by inspecting center of pressure and be classified through proposed classification algorithm.

Keywords: center of pressure, fault classification, healthcare robot, sit-to-stand movement

Procedia PDF Downloads 175

Authors: B. Arend, P. Sunnen, P. Caire

Abstract:

In a single case study, we show how a conversation analysis (CA) approach can shed light onto the sequential unfolding of human-robot interaction. Relying on video data, we are able to show that CA allows us to investigate the respective turn-taking systems of humans and a NAO robot in their dialogical dynamics, thus pointing out relevant differences. Our fine grained video analysis points out occurring breakdowns and their overcoming, when humans and a NAO-robot engage in a multimodally uttered multi-party communication during a sports guessing game. Our findings suggest that interdisciplinary work opens up the opportunity to gain new insights into the challenging issues of human robot communication in order to provide resources for developing mechanisms that enable complex human-robot interaction (HRI).

Keywords: human robot interaction, conversation analysis, dialogism, breakdown, museum

Procedia PDF Downloads 283

Authors: Kai-Yi Cho, Fa-Shian Chang, Lih-Tyng Hwang, Chih-Feng Liu, Jeng-Nan Lee, Shun-Min Wang, Jhu-Wei Ji

Abstract:

This paper presents a ground robot which has the tracked transformative structures of the motion mechanism. The robot has a good ability to adapt to the terrain, due to the front end of the track can be deformed, it can more easily pass the more complex area, such as to climb stairs and ramp areas. Usually in the disaster area, where the terrain is generally broken and complicated, there will be many slopes, broken walls, rubble, and obstacles, then if you want the robot through this area, you need to have a good off-road performance for possible complex terrain, this robot with the transformative tracked mechanism has a strong adaptability, it can overcome the limitation of the terrains to be a good rescue robot. Also, the robot has a good flexibility in the shape of contact with the ground; that can adapt the varied terrain by the deformable track, thus able to pass the different terrains, that was verified through the experiments on a test-platform and a field test. The prototype of the robot system has been developed, and experiments are carried out to verify the validity of the proposed design.

Keywords: tracked robot, rescue robot, transformation mechanism, deformable track, hull design

Procedia PDF Downloads 308

Authors: Roya Khajepour, Alireza B. Novinzadeh

Abstract:

A major issue with spherical robot is it surface shape, which is not always predictable. This means that given only the dynamic model of the robot, it is not possible to control the robot. Due to the fact that in certain conditions it is not possible to measure surface friction, control methods must be prepared for these conditions. Moreover, although spherical robot never becomes unstable or topples thanks to its special shape, since it moves by rolling it has a non-holonomic constraint at point of contact and therefore it is considered a non-holonomic system. Existence of such a point leads to complexity and non-linearity of robot's kinematic equations and makes the control problem difficult. Due to the non-linear dynamics and presence of uncertainty, the sliding-mode control is employed. The proposed method is based on Lyapunov Theory and guarantees system stability. This controller is insusceptible to external disturbances and un-modeled dynamics.

Keywords: sliding mode, spherical robot, non-holomonic constraint, system stability

Procedia PDF Downloads 359

Authors: Safiullah Abdullahi

Abstract:

This paper describes the design of a digital control system which controls the speed and direction of a robot. The robot is expected to follow a black thick line with the highest possible speed and lowest error around the line. The control system of the robot will correct for the angle error that is made between the frame axis of the robot and the line. The cause for error is the difference in speed of the two driving wheels of the robot which are driven by two separate DC motors, whereas the speed difference in wheels is due to the un-modeled fraction that is available in the wheels with different magnitudes in each. The control scheme is that a number of photo sensors are mounted in the front of the robot and report their position in reference to the black line to the digital controller. The controller then, evaluates the position error and generates the needed duty cycle for the related wheel motor to drive it faster or slower.

Keywords: digital control, robot, controller, control system

Procedia PDF Downloads 527

Authors: Seyed Reza Mirdehghan, Mohammad Reza Haeri Yazdi

Abstract:

In this paper, the stability and the performance of a two-degree-of-freedom robot under an interaction with a unknown environment has been investigated. The time when the robot returns to its initial position after an interaction and the primary resistance of the robot against the impact must be reduced. Thus, the applied torque on the motor will be reduced. The impedance control is an appropriate method for robot control in these conditions. The stability of the robot at interaction moment was transformed to be a robust stability problem. The dynamic of the unknown environment was modeled as a weight function and the stability of the robot under an interaction with the environment has been investigated using the robust control concept. To improve the performance of the system, a force controller has been designed which the normalized impedance after interaction has been reduced. The resistance of the robot has been considered as a normalized cost function and its value was 0.593. The results has showed reduction of resistance of the robot against impact and the reduction of convergence time by lower than one second.

Keywords: impedance control, control system, robots, interaction

Procedia PDF Downloads 404

Authors: Zhang Zijian

Abstract:

Nowadays, spherical robots have played an important role in many fields, but the insufficient ability of obstacle surmounting limits their wider application fields. To solve this problem, a jumping system of a spherical robot is designed based on Archimedes helix. The jumping system of the robot utilizes the characteristics of Archimedes helix and isovelocity helix to achieve constant speed and stable contraction, which ensures the stability of the system. Also, the jumping action of the robot is realized by instantaneous release of elastic potential energy. In order to verify the effectiveness of the jumping system, we designed a spherical robot and its jumping system. The experimental results show that the jumping system has the advantages of light weight, small size, high energy conversion efficiency, and can realize the spherical jumping function.

Keywords: hopping mechanism, Archimedes' Helix, hopping robot, spherical robot

Procedia PDF Downloads 116

Authors: Wiem Zemzem, Moncef Tagina

Abstract:

In this work, we are addressing the problem of an autonomous mobile robot navigating in a large, unknown and dynamic environment using reinforcement learning abilities. This problem is principally related to the exploration/exploitation dilemma, especially the need to find a solution letting the robot detect the environmental change and also learn in order to adapt to the new environmental form without ignoring knowledge already acquired. Firstly, a new action selection strategy, called ε-greedy-MPA (the ε-greedy policy favoring the most promising actions) is proposed. Unlike existing exploration/exploitation policies (EEPs) such as ε-greedy and Boltzmann, the new EEP doesn’t only rely on the information of the actual state but also uses those of the eventual next states. Secondly, as the environment is large, an exploration favoring least recently visited states is added to the proposed EEP in order to accelerate learning. Finally, various simulations with ball-catching problem have been conducted to evaluate the ε-greedy-MPA policy. The results of simulated experiments show that combining this policy with the Qlearning method is more effective and efficient compared with the ε-greedy policy in stationary environments and the utility-based reinforcement learning approach in non stationary environments.

Keywords: autonomous mobile robot, exploration/ exploitation policy, large, dynamic environment, reinforcement learning

Procedia PDF Downloads 395

Authors: Romisaa Ali

Abstract:

This paper presents a comparison between twin-delayed Deep Deterministic Policy Gradient (TD3) and Soft Actor-Critic (SAC) reinforcement learning algorithms in the context of training robust navigation policies for Jackal robots. By leveraging an open-source framework and custom motion control environments, the study evaluates the performance, robustness, and transferability of the trained policies across a range of scenarios. The primary focus of the experiments is to assess the training process, the adaptability of the algorithms, and the robot’s ability to navigate in previously unseen environments. Moreover, the paper examines the influence of varying environmental complexities on the learning process and the generalization capabilities of the resulting policies. The results of this study aim to inform and guide the development of more efficient and practical reinforcement learning-based navigation policies for Jackal robots in real-world scenarios.

Keywords: Jackal robot environments, reinforcement learning, TD3, SAC, robust navigation, transferability, custom environment

Procedia PDF Downloads 70