MuJoCo Environments

Overview

MuJoCo (Multi-Joint dynamics with Contact) is a proprietary physics engine for detailed, efficient rigid body simulations with contacts. MuJoCo can be used to create environments with continuous control tasks such as walking or running. Thus, many policy gradient methods (TRPO, PPO) have been tested on various MuJoCo environments.

Environments

We list various reinforcement learning algorithms that were tested with MuJoCo. These results are from RL Database. If this page was helpful, please consider giving a star!

Star

Environment Result Algorithm Source
reacher -1.5 ACKTR Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
hopper 3915.9 ACKTR Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
inverted-double-pendulum 9356.1 A2C Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
swimmer 297.0 Trust-PCL Trust-PCL: An Off-Policy Trust Region Method for Continuous Control
walker2d 6874.1 TRPO Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
inverted-pendulum 1000.0 ACKTR Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation
half-cheetah 9636.95 TD3 Addressing Function Approximation Error in Actor-Critic Methods
ant 6104.2 Trust-PCL Trust-PCL: An Off-Policy Trust Region Method for Continuous Control

There are many papers that have experimented with the MuJoCo continuous control environment, but most papers decided not include exact scores and instead used performance curves. If you know other papers that report results on the MuJoCo environment, please email me!

Installation

Prerequisites

To install the MuJoCo environment, you need the OpenAI Gym toolkit. Read this page to learn how to install OpenAI Gym.

You also need to purchase MuJoCo license. MuJoCo offers a 30-day trial license for everyone, and a free license for students using MuJoCo for personal projects only. Visit their license page for more information.

Install MuJoCo binary

  1. Download the MuJoCo version 1.50 binaries for Linux, OSX, or Windows.
  2. Unzip the downloaded mjpro150 directory into ~/.mujoco/mjpro150, and place your license key (the mjkey.txt file from your email) at ~/.mujoco/mjkey.txt.

Install MuJoCo package

If you did a full install of OpenAI Gym, the MuJoCo package should already be installed. Otherwise, you can install the MuJoCo environments with a single pip command:

1

pip3 install gym[mujoco]

Test Installation

You can try rendering the Humanoid-v2 environment to make sure the MuJoCo environment was correctly installed.

1
2
3
4

import gym
env = gym.make('Humanoid-v2')
env.reset()
env.render()