Skip to content

gtatiya/gym-novel-gridworlds

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

63 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Gym Novel Gridworlds

Gym Novel Gridworlds are OpenAI Gym environments for developing and evaluating AI agents that can detect and adapt to unknown sudden novelties in their environments. In each environment, the agent needs to craft objects using multiple recipes, which requires performing certain steps in some sequence.

Installation

git clone https://github.com/gtatiya/gym-novel-gridworlds.git
cd gym-novel-gridworlds
pip install -e .

If you prefer, you can install directly from PyPI:

pip install gym-novel-gridworlds

BibTeX

@article{goel2021novelgridworlds,
  title={NovelGridworlds: A Benchmark Environment for Detecting and Adapting to Novelties in Open Worlds},
  author={Shivam Goel* and Gyan Tatiya* and Matthias Scheutz and Jivko Sinapov},
  journal={International Foundation for Autonomous Agents and Multiagent Systems, AAMAS},
  year={2021}
}

Development Environment

Python 3.7.9 and following packages were used for development:

Python Packages

pip install gym==0.18.0
pip install keyboard==0.13.5
pip install matplotlib==3.3.3
pip install numpy==1.19.4
pip install stable-baselines==2.10.1
pip install tensorflow-gpu==1.15.0

Running

import gym
import gym_novel_gridworlds

env = gym.make('NovelGridworld-Pogostick-v1')

done = False
while not done:
        env.render()
        action = env.action_space.sample()
        obs, reward, done, info = env.step(action)

env.close()

Play it yourself

Play the game using keyboard interface. To change the environment and to inject a novelty you must edit keyboard_interface.py.

python tests/keyboard_interface.py

Environments

In each environment, the agent needs to craft objects using multiple recipes, which requires performing certain steps in some sequence. The default observation space is of type gym.spaces.Dict that contains information about the map, agent_location, agent_facing_id, and inventory_items_quantity. Feel free to use Observation Wrappers to change observation space and Wrappers to limit action space and more.

NovelGridworld-Bow-v0 drawing NovelGridworld-Bow-v1 drawing
NovelGridworld-Pogostick-v0 drawing NovelGridworld-Pogostick-v1 drawing

Note that deprecated environments do not support any wrappers.

NovelGridworld-v0 drawing NovelGridworld-v1 drawing
NovelGridworld-v2 drawing NovelGridworld-v3 drawing
NovelGridworld-v4 drawing NovelGridworld-v5 drawing

Enjoy a Trained Agent

If the trained agent exists, then you can see it in action using:

python enjoy.py -env <env id> -episodes <number of episodes>

For example, enjoy PPO2 on NovelGridworld-v0 during 10 episodes:

python enjoy.py -env NovelGridworld-v0 -episodes 10

Novelties

How to inject a novelty?

import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.novelty_wrappers import inject_novelty

env_id = 'NovelGridworld-Pogostick-v1'
env = gym.make(env_id)

novelty_name = 'axe'
novelty_arg1 = 'iron'
novelty_arg2 = 'true'
difficulty = 'medium'

env = inject_novelty(env, novelty_name, difficulty, novelty_arg1, novelty_arg2)

List of Novelties

Novelty NameNovelty DetailsEasyMediumHardnovelty_arg1novelty_arg2
addchopA new action (Chop) is added. It's like break action, but instead of 1 item, agent will get 2 items, but step_cost will be higher (1.2 times).-----
additemA new item is introduced in the map.1-10% of the free locations are occupied by the new item.10-20% of the free locations are occupied by the new item.20-30% of the free locations are occupied by the new item.Any item name (e.g. arrow, spring)-
addjumpA new action (Jump) is added, when it's executed, the agent jumps 2 blocks forward.-----
axeAn axe is introduced. Using axe reduces the step_cost when Break action is used. If novelty_arg2 is set to true, the agent will get 2 items in inventory after break action instead of 1. There are 12 variants of this novelty: 3 difficulties x 2 attributes of axe x 2 quantities on breaking.An axe is in the inventory. The agent must select axe to use it.An axe in the map. The agent must go near the axe to grab it.A recipe to craft axe is given. The agent must craft the axe.Attribute of axe (e.g. wooden, iron)Optional: true or false.If true, the agent will get 2 items in inventory after break action otherwise 1.
axetobreakAn axe is introduced and axe required to break items. Using axe reduces the step_cost when Break action is used. There are 6 variants of this novelty: 3 difficulties x 2 attributes of axe.An axe is in the inventory. The agent must select axe to use it.An axe in the map. The agent must go near the axe to grab it.A recipe to craft axe is given. The agent must craft the axe.Attribute of axe (e.g. wooden, iron)-
breakincreaseThe agent gets 2 items in the inventory when the agent breaks that item instead of 1.---Optional: any existing item (e.g. tree_log), Default: all items-
crateA new item, crate, is introduced in the map. When the crate is broken, some ingredients of the goal item to craft are added in the inventory.100% of the ingredients of the goal item to craft are added in the inventory.50-90% of the ingredients of the goal item to craft are added in the inventory.10-50% of the ingredients of the goal item to craft are added in the inventory.--
extractincdecThe agent gets more/less items in the inventory when the agent extracts it instead of the default quantity.---increase or decrease-
fenceItems in the map are blocked by fence. The agent must break the fence to reach the blocked items. There are 6 variants of this novelty: 3 difficulties x 2 attributes of fence.20-50% items in the map are blocked by fence.50-90% items in the map are blocked by fence.90-100% items in the map are blocked by fence.Attribute of fence (e.g. wooden, iron)-
fencerestrictionRestrict breaking an item around fence until fence(s) are broken. All fences are always breakable. 50-90% items in the map are blocked by fence (fence medium novelty).Breaking one fence next to the item, will enable breaking that item. This is same as fence novelty.Breaking fences on one side of the item, will enable breaking that item.Breaking all fences around the item, will enable breaking that item.Attribute of fence (e.g. wooden, iron)-
firewallWalls are replaced by fire-walls, agent dies when it's next to fire-wall.-----
remapactionActions are remapped randomlyOnly manipulation actions (e.g. Forward, Left, Right, Break, Place_tree_tap, Extract_rubber) are remapped.Manipulation actions and craft actions (e.g. Craft_plank, Craft_stick, Craft_tree_tap, Craft_pogo_stick) are remapped.Manipulation actions, craft actions and select actions (e.g. Select_crafting_table, Select_plank, Select_pogo_stick, Select_rubber, Select_stick, Select_tree_log, Select_tree_tap) are remapped.--
replaceitemReplace an existing item with a new item.5-10% of existing item are replaced with the new item.40-90% of existing item are replaced with the new item.100% of existing item are replaced with the new item.Item to replace (e.g. wall). Must be an existing item.Item to replace with (e.g. stone-wall). Must be a new item.

Novelties Demo

addchop drawing additem drawing
addjump drawing axe drawing
axetobreak drawing breakincrease drawing
crate drawing extractincdec drawing
fence drawing fencerestriction drawing
firewall drawing remapaction drawing
replaceitem drawing

Wrappers

Observation Wrappers

How to use Observation Wrappers?

import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.observation_wrappers import LidarInFront

env_id = 'NovelGridworld-Pogostick-v1'
env = gym.make(env_id)

env = LidarInFront(env, num_beams=8)

List of Observation Wrappers

Observation NameObservation Details
AgentMapAgent's local view within a range (agent_view_size), agent_facing_id, inventory_items_quantity.
LidarInFrontSend several beans (num_beams) at equally spaced angles in 360 degrees in front of agent + agent's current inventory.

Wrappers

How to use Wrappers?

import gym
import gym_novel_gridworlds
from gym_novel_gridworlds.wrappers import LimitActions

env_id = 'NovelGridworld-Bow-v0'
env = gym.make(env_id)

env = LimitActions(env, {'Forward', 'Left', 'Right', 'Break', 'Craft_bow'})

List of Wrappers

Wrapper NameWrapper Details
LimitActionsLimit the actions in the environment.
SaveTrajectoriesSave agent trajectories in the environment.

About

Gym Novel Gridworlds are environments for OpenAI Gym.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages