piSTAR Lab Documentation
piSTAR Lab is an open source platform built to make AI experimentation accessible and fun.
Note
piSTAR Lab is in early development.
Who is piSTAR Lab for?
- AI Enthusiasts and Students:
Have fun experimenting with state-of-the-art AI Agents. No coding required.
Gain a better intuition about reinforcement learning by creating your own agents.
- Reinforcement Learning Researchers and Engineers:
Showcase your algorithms by creating extensions for other users to enjoy. No restrictive framework/library requirements.
Import your externally trained agents.
Use piSTAR Lab in your development workflow.
Try your algorithm on newly available environments with just a few clicks
- Game and Environment Developers:
Create Extensions” which make your game playable by piSTAR Agents.
Train your own Agents for your game’s AI
Features
Intuitive UI
Extension System for adding new agents, environments or tasks types
No framework requirements. Use any framework available in python (other languages coming)
Real-time streaming of observations during training
Single and Multi player environment support
Experiment tracking
Python API, anything you can do in the UI, you can do in Python as well
Uses Ray Project (https://ray.io/) under the hood for distributed processing
Coming soon (unordered)
Simpler installation and better support for Windows and Mac
Easily share your trained agents with friends or publicly
Docker based agents and environments
Remote agents and environments
Data environments for agents with support for learning from offline data
API for Data Interfaces
Support for browser based games and environments so human can interact directly with their agents.
Support for other programming languages
Human control mode for testing environments via the UI
Better workspace integration with VSCode and Jupyter
Interactive Debugger
Testing and quality verification of extensions
Contributing
piSTAR Lab is Open Source and welcomes your contributions.
Overview
piSTAR Lab is modular Agent Development Environment that can run on your PC or in the cloud.
Manage your own library of Agents and Environments.
Run experiments with a click of a button.
Watch your agent learn in realtime.
Screenshots
Installation
NOTE: Tested on Ubuntu and Windows 10. If you run into issues, we recommend using Docker.
with Pip
Install Anaconda or Miniconda
Visit https://www.anaconda.com/products/individual for instructions
- Install additional dependencies (Ubuntu only)
XVFB to render without display (No MS Windows Support)
ffmpeg for video processing
sudo apt-get install -y xvfb ffmpeg
Create Conda Virtual Environment
conda create -n pistarlab python=3.7 conda activate pistarlab conda install pip
Option 1 (Package only)
pip install https://github.com/pistarlab/pistarlab/archive/refs/heads/main.zip#egg=pistarlab[all]
Option 2 (Package + Source)
git clone --single-branch --depth=1 http://github.com/pistarlab/pistarlab/ cd pistarlab pip install -e .[all]
with Docker
- Install Docker:
Clone Repo
docker pull docker.pkg.github.com/pistarlab/pistarlab/image:latest
Usage
To launch piSTAR Lab, run:
pistarlab_launcher
Open your browser to http://localhost:7777
To launch the piSTAR Lab docker container, run:
mkdir pistarlab_docker ; docker run --rm -i -t -u `id -u` \
--shm-size=2g \
-p 7776:7776 \
-p 7777:7777 \
-p 7778:7778 \
-p 8265:8265 \
-v ${HOME}/pistarlab_docker:/home/ray/pistarlab pistarlab/pistarlab:latest --launcher_host="0.0.0.0"
Open your browser to http://localhost:7777
Extensions
Extensions are the primary mechanism for adding new Agents, Environments, Tasks and Components.
Below are extensions available via piSTAR Lab and the repo (https://github.com/pistarlab/pistarlab-repo/). Future versions will allow user created extensions to be used and shared as well.
- Gym Atari Environments:
- Gym Text Environments:
- Gym 2D Box and Classic Control Environments:
- piSTAR Landia Environment
- Ray RLlib Agents
Extension: https://github.com/pistarlab/pistarlab/tree/main/pistarlab/extensions/pistarlab-rllib
Project Page: https://docs.ray.io/en/latest/rllib.html
Note: extension only includes a subset of agent algorithms
- MiniGrid Envirnoments:
- Petting Zoo Environments (WIP)
Extension: https://github.com/pistarlab/pistarlab-repo/tree/main/src/pistarlab-petting-zoo
Project Page: https://www.pettingzoo.ml/
Note: extension only includes a small subset of environments
- Stable Baselines Agents (WIP)
Extension: https://github.com/pistarlab/pistarlab-repo/tree/main/src/pistarlab-stable-baselines
Project Page:
Note: extension only includes a small subset of agents available and currently does not support parameterization
- Unity ML-Agent Environments (Planned)
Extension: https://github.com/pistarlab/pistarlab-repo/tree/main/src/pistarlab-unity-envs
Project Page: https://github.com/Unity-Technologies/ml-agents
Note: currently unavailable in extension repo
- Minecraft RL Environments (Planned)
Extension: https://github.com/pistarlab/pistarlab-repo/tree/main/src/pistarlab-envs-minecraft
Project Page: https://github.com/minerllabs/minerl
Note: currently unavailable in extension repo
- MultiGrid MARL Environments (Planned)
Extension: https://github.com/pistarlab/pistarlab-repo/tree/main/src/pistarlab-multigrid
Project Page: https://github.com/ArnaudFickinger/gym-multigrid
Note: currently unavailable in extension repo
Development Notes
Creating a manifest
Manfiest files are used to speed up the installation of Extensions. They are especially useful for Environments where probing is required.
Example of creating a manifest
xvfb-run python pistarlab/extensions_tools.py --action=save_manifest --extension_path PATH_TO_EXTENSION/pistarlab-envs-gym-main
Advanced
Settings
Data and Configuration Path
By default pistarlab stores data and configuration in the $HOME/pistarlab/ directory. (eg: /home/$USER/pistarlab) This can be changed by using the PISTARLAB_ROOT environment variable
If you are using Docker, PISTARLAB_ROOT is under $HOME/pistarlab_docker/. This has been changed for Docker to avoid permission issues that Docker may instroduce.
Configuration is stored in the config.yaml file located at PISTARLAB_ROOT
Data is stored under PISTARLAB_ROOT/data
Debugging
Our current solution uses the remote_pdb python module to enable remote debuging over telnet.
To use debugging, do the following:
Insert the following code snip where you want the debugger to started and wait for your input
from ..utils.remote_pdb import set_trace
set_trace() # you'll see the port number in the logs
When the above code is executed, it will print the port number to connect to in the logs.
use telnet to connect and use pdb as usual: for example
`telnet 127.0.0.1 PORTNUM_HERE`
Cluster Mode
Warning
Experimental
Cluster mode is handled by Ray. For more information visit ray.io.
WIP, this documentation is incomplete and more testing needed.
Requirements * PostgreSQL to communicate over network (Temporary) * Python Version should be same on all nodes
with Docker
Switch to python env to 3.7.7
This is needed because the docker version of Ray uses 3.7.7 and multiple versions of python create problems when pickling.
conda install python=3.7.7
Docker Related Issue with using Ray
file permission issues when using docker. files copied using rsync get permissions
/tmp/ray_tmp_mount
without Docker
- Install MINICONDA
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
bash ~/Miniconda3-latest-Linux-x86_64.sh
login logout
conda install python=3.7.7
** Configuration **
Config file path: ~/pistarlab/cluster.yaml
TODO, see for details
Startup
Starting cluster
ray up -vvvv ~/pistarlab/cluster.yaml
Launch piSTAR Lab on head node
python pistarlab/launcher.py --ray_address="IP_ADDRESS:6379"
Install extensions on nodes
ray exec $HOME/pistarlab/cluster.yaml "pip install --user -e /home/pistarlabuser/app/pistarlab/extensions/pistarlab-envs-gym-main"
Sync Data with Nodes
Before each task run
ray rsync-up -vvv ~/pistarlab/cluster.yaml
After each task (or as needed)
ray rsync-down -vvv ~/pistarlab/cluster.yaml /home/pistarlabuser/pistarlab/data/ $HOME/pistarlab/data/
Troubleshooting
Missing .so error when running tensorflow
ensure LD_LIBRARY_PATH is correct
My GPU is suddenly not avilable (Ubuntu)
If your GPU was working previously, but suddenly is not accessable the the system. The following script may help the script at `scripts/fix_gpu.sh` may help.
Testing GPU
Check if torch is detecting the GPU
python -c “import torch; print(torch.cuda.is_available());”
Running Atari Environment on Windows
If you get this error:
FileNotFoundError: Could not find module '..\atari_py\ale_interface\ale_c.dll' (or one of its dependencies). Try using the full path with constructor syntax.------------
Reinstall Atari from here: https://github.com/Kojoley/atari-py/releases
Design
piSTAR Lab is build from on serveral excellent opensource technologies including:
Redis (redis.io) - in-memory data store
Ray (ray.io) - distributed processing framework to handle remote task execution
SQLite (sqlite.org)/ PostgreSQL (postgresql.org)
Vuejs (vuejs.org) - web development framework built on (nodejs)
Theia IDE (theia-ide.org) - web based IDE
Developer Notes
Setting up your environment
Tested on Ubuntu
Install Anaconda or Miniconda
Visit https://www.anaconda.com/products/individual for instructions
Create Conda Virtual Environment
conda create -n pistarlab python=3.7
Install PIP
conda install pip
Clone Repo and install
git clone https://github.com/pistarlab/pistarlab cd pistarlab pip install -e .[all]
- Install additional dependencies
XVFB to render without display (No MS Windows Support)
ffmpeg for video processing
sudo apt-get install -y xvfb ffmpeg
Install NPM/Nodejs for UI and Theia IDE
bash ./install_node.sh
Build Web IDE (optional)
bash ./build_ide.sh
on Windows [Experimental]
Limitation: no headless mode for many environments so rendering may open a window
Install Miniconda
Install GitBash
Follow Ubuntu Instructions
Troubleshooting
Building Theia IDE on Windows. * https://github.com/eclipse-theia/theia/blob/master/doc/Developing.md#building-on-windows
Install Scoop
See: https://github.com/lukesampson/scoop#installation
Invoke-Expression (New-Object System.Net.WebClient).DownloadString('https://get.scoop.sh') # or shorter iwr -useb get.scoop.sh | iex # IF SCOOP doesn't get added to path $env:Path += ";C:\Users\${USER}\scoop\shims"
Making changes to the UI
The UI is build using Vuejs cli and requires npm to run. Once setup, changes to the ui source code will be reflected immidiately in the browser.
Option 1:
# Start pistarlab with enable_dev_ui argument. Eg: `pistarlab_launcher --enable_dev_ui`
Option 2:
#. Run the UI using `npm run serve`
#. By default, changes will be reflected at http://localhost:8080
Building for Readonly Viewing
pip install -e . -no-deps pip install -r requirements-webreadonly.txt
Building for PiPy
Build Redis Server Binary and copy to pistarlab/thirdparty_lib
bash ./install_redis.sh_
Build and deploy UI in pistarlab/uidist/ package directory
bash ./build_ui.sh
Run Tests with tox
pip install tox tox
Building wheel and source distribution and view files
rm -rf build dist *.egg-info && python setup.py bdist_wheel && python -m build --sdist --wheel && unzip -l dist/*.whl
Uploading to PiPy
pip install twine twine upload dist/*
Building the Documentation
Install dependencies
pip install -r docs/requirements.txt
Rebuild API Docs
From the project root, run:
cd docs sphinx-apidoc -o source ../pistarlab
Update the HTML
cd docs make html
Building and Publishing a new Docker Image
Instructions on how to create a docker image from an Ubuntu environment
Make changes to docker file
Update requirements.txt
conda create -n pistarlab377 python=3.7.7 conda activate pistarlab377 pip install -e . pip freeze > requirements.txt
Run Docker Build
./build_docker
Roadmap
Warning
Under development
Planned Features
Composable Agents - agents built from reusable components
Public repositories for agents and component snapshots
Filter by observation/action space
Persistent environments
UI for easy hyper parameter exploration
Learning resource: videos, documentation, and tutorials
Missions which help users gain intuition about different aspects of reinforcement learning
Hosted competitions