Learning Robot Arm Controls Using Augmented Random Search in a Simulated Environment

EasyChair Preprint 8873

Abstract

We investigate the learning of continuous action policy for controlling a six-axes robot arm. Traditional tabular Q-Learning can handle discrete actions well but less so for continuous actions since the tabular approach is constrained by the size of the state-value table. Recent advances in deep Reinforcement Learning (deep RL) and Policy Gradient (PG) learning abstract the look-up table using function approximators such as artificial neural networks (ANNs). ANNs abstract loop-up policy tables as policy networks that can predict discrete actions as well as continuous actions. However, deep RL and PG learning were criticized for their complexity. It was reported in recent works that Augmented Random Search (ARS) has a better sample efficiency and a simpler hyper-parameter tuning. This motivates us to apply the technique to our robot-arm reaching tasks. We constructed a custom simulated robot arm environment using the Unity Machine Learning Agents game engine, then designed three robot-arm reaching tasks. Twelve models were trained using ARS techniques. Another four models were trained using the state-of-the-art PG learning technique i.e., proximal policy optimization (PPO). Results from models trained using PPO provide a baseline from the PG technique. Empirical results of models trained using ARS and PPO were analyzed and discussed.

Keyphrases: Augmented Random Search, Deep Reinforcement Learning, Robot arm controls

@booklet{EasyChair:8873,
  author    = {Somnuk Phon-Amnuaisuk and Peter David Shannon and Saiful Omar},
  title     = {Learning Robot Arm Controls Using Augmented Random Search in a Simulated Environment},
  howpublished = {EasyChair Preprint 8873},
  year      = {EasyChair, 2022}}