The environnement consist of a 3D square world where a character can navigate and collect bananas.
A discrete (sparse) reward signal is provided by the environnement. A reward of +1 is provided in case of collecting a yellow banana, and a reward of -1 for blue bananas.
The state space (continuous) has 37 dimensions and contains the agent's velocity, along with ray-based perception of objects around the agent's forward direction.
The actions space is discrete and contains 4 actions labeled as
The task is episodic, and in order to solve the environment, we consider that the agent must get an average score of +13 over 100 consecutive episodes. As opposed to most robotics application, the reward design, observations and action space are considered fixed and cannot be changed to improve the performance of the algorithm.
This project was developped on Ubuntu 22.04.4 LTS.
Python dependencies are managed through conda (micromamba) and pip (when conda package not available). Due to old dependencies related to Unity ML Agents, Python version is 3.6.15.
Here are some gross instructions to install and run the project:
First, install the banana env binaries: Linux: click here.
Then, install the python dependencies.
conda create -n udacity-navigation python=3.6
conda activate udacity-navigation
pip install gym
pip install box2d-py
pip install box2d
git clone https://github.com/udacity/Value-based-methods.git
cd Value-based-methods/python
pip install .
pip install torch==0.4.0
python -m ipykernel install --user --name udacity-navigation --display-name "udacity-navigation"
jupyter lab
Do not forget to switch the kernel to the freshly created udacity-navigation one.
The main entry is the Navigation.ipynb file, you can train the agent in this notebook.
dqn_agent.py and model.py contain implementation of the DQN algorithm and the Q network implementation respectively.