diff --git a/Documentation/Changelog.md b/Documentation/Changelog.md
index b42326438..c1009cf2b 100644
--- a/Documentation/Changelog.md
+++ b/Documentation/Changelog.md
@@ -6,32 +6,45 @@
### `tdw` module
+- Added required modules: `pyinstaller` and `keyboard`
+- Added `tdw.keyboard_controller` A controller that can listen to keyboard input.
+
#### `Controller`
- Removed optional `display` parameter. It doesn't actually work; Linux users should instead launch the controller with a `DISPLAY` environment variable.
-### Frontend
-
-- Added `freeze.py`. "Freeze" your controller into a portable binary executable.
-
#### Backend
-- Added `controller.spec` (used for freezing controller code).
- Adjusted how Flatbuffers imports numpy so that frozen controller code works.
+### Example Controllers
+
+- Renamed `keyboard.py` to `keyboard_controller.py` to avoid a name clash with the `keyboard` module. Rewrote the code to use the `KeyboardController` class.
+
+### Build
+
+- Fixed: Segmentation colors are often non-unique.
+
+### Misc.
+
+- Added `freeze.py`. "Freeze" your controller into a portable binary executable.
+ - Added `controller.spec` (used for freezing controller code).
+
### Documentation
#### New Documentation
-| Document | Description |
-| ----------- | ------------------------------------------------------------ |
-| `freeze.md` | How to freeze your controller code into a binary executable. |
+| Document | Description |
+| ------------------------ | ------------------------------------------------------------ |
+| `freeze.md` | How to freeze your controller code into a binary executable. |
+| `keyboard_controller.md` | API for KeyboardController. |
#### Modified Documentation
| Document | Description |
| -------------------- | ---------------------------------------------------------- |
| `getting_started.md` | Fixed instructions for how to start a controller in Linux. |
+| `docker.md` | Fixed some broken links. |
## v1.6.4
diff --git a/Documentation/Docker/docker.md b/Documentation/Docker/docker.md
index 42c776259..8e82f5370 100644
--- a/Documentation/Docker/docker.md
+++ b/Documentation/Docker/docker.md
@@ -60,7 +60,7 @@ All Docker-related bash scripts are in [`tdw/Docker`](https://github.com/threedw
| ------------------------------------------------------------ | ------------------------------------------------------------ | ------------------------------------------------------------ |
| [`start_container.sh`](https://github.com/threedworld-mit/tdw/tree/master/Docker/start_container.sh) | | Start the container and run TDW. |
| [`start_container_xpra.sh`](https://github.com/threedworld-mit/tdw/tree/master/Docker/start_container_xpra.sh) | | Start the container with [Xpra](../misc_frontend/xpra.md) and run TDW. |
-| [`start_container_audio_video.sh VOLUME IPADDRESS PORT`](https://github.com/threedworld-mit/tdw/tree/master/Docker/start_container_audio.sh) | `VOLUME` Save audio to this volume
`IPADDRESS` The address of the build.
`PORT` The port of the build. | [Record audio and video](../misc_frontend/video.md) from TDW. |
+| [`start_container_audio_video.sh VOLUME IPADDRESS PORT`](https://github.com/threedworld-mit/tdw/tree/master/Docker/start_container_audio_video.sh) | `VOLUME` Save audio to this volume
`IPADDRESS` The address of the build.
`PORT` The port of the build. | [Record audio and video](../misc_frontend/video.md) from TDW. |
### Other scripts
@@ -69,7 +69,7 @@ All Docker-related bash scripts are in [`tdw/Docker`](https://github.com/threedw
| [`pull.sh`](https://github.com/threedworld-mit/tdw/tree/master/Docker/pull.sh) | | Try to download a Docker container from DockerHub with a tag that matches the version of TDW on this machine. |
| [`docker_tag.sh`](https://github.com/threedworld-mit/tdw/tree/master/Docker/docker_tag.sh) | | Get the tag of the TDW Docker image. |
| [`tdw_version.py`](https://github.com/threedworld-mit/tdw/tree/master/Docker/tdw_version.py) | | Get the version of TDW on this machine. |
-| [`record_audio_video.sh ADDRESS PORT WIDTH HEIGHT`](https://github.com/threedworld-mit/tdw/tree/master/Docker/record_audio.sh) | `ADDRESS` The network address of the build.
`PORT` The network port of the build
`WIDTH` The desired width of the video in pixels.
`HEIGHT` The desired height of the video in pixels. | Launch TDW and begin recording audio. |
+| [`record_audio_video.sh ADDRESS PORT WIDTH HEIGHT`](https://github.com/threedworld-mit/tdw/tree/master/Docker/record_audio_video.sh) | `ADDRESS` The network address of the build.
`PORT` The network port of the build
`WIDTH` The desired width of the video in pixels.
`HEIGHT` The desired height of the video in pixels. | Launch TDW and begin recording audio. |
## Docker within Docker
diff --git a/Documentation/python/example_controllers.md b/Documentation/python/example_controllers.md
index f0a3bca1a..4c0a47f55 100644
--- a/Documentation/python/example_controllers.md
+++ b/Documentation/python/example_controllers.md
@@ -23,7 +23,7 @@ Each TDW release includes many example controllers. They can be found in: `2. Add a camera and receive an image. |
| `hdri.py` | Create an object and avatar and capture images of the scene, rotating the HDRI skybox by 15 degrees
for each image. |
| `impact_sounds.py` | - Listen for collisions between objects.
- Generate an impact sound with py_impact upon impact and play the sound in the build. |
-| `keyboard.py` | Use WASD or arrow keys to move an avatar. |
+| `keyboard_controls.py` | Use WASD or arrow keys to move an avatar. |
| `local_object.py` | Create a local asset bundle and load it into TDW.
See `Documentation/misc_frontend/add_local_object.md` for how to run the Asset Bundle Creator. |
| `minimal.py` | A minimal example of how to connect to the build and receive data. |
| `minimal_remote.py` | A minimal example of how to use the launch binaries daemon to
start and connect to a build on a remote node. Note: the remote
must be running launch_binaries.py. |
diff --git a/Documentation/python/keyboard_controller.md b/Documentation/python/keyboard_controller.md
new file mode 100644
index 000000000..7d6818f3d
--- /dev/null
+++ b/Documentation/python/keyboard_controller.md
@@ -0,0 +1,78 @@
+# `keyboard_controller.py`
+
+## `KeyboardController(Controller)`
+
+`from tdw.keyboard_controller import KeyboardController`
+
+Listen for keyboard input to send commands.
+
+Usage:
+
+```python
+from tdw.keyboard_controller import KeyboardController
+from tdw.tdw_utils import TDWUtils
+
+def stop():
+ done = True
+
+done = False
+c = KeyboardController()
+c.start()
+
+# Quit.
+c.listen(key="esc", commands=None, function=stop)
+
+# Equivalent to c.start()
+c.listen(key="r", commands={"$type": "load_scene", "scene_name": "ProcGenScene"}, function=None)
+
+while not done:
+ # Receive data. Load the scene when r is pressed. Quit when Esc is pressed.
+ c.communicate([])
+# Stop the build.
+c.communicate({"$type": "terminate"})
+```
+
+***
+
+#### `stop()`
+
+def __init__(self, port: int = 1071, check_version: bool = True, launch_build: bool = True):
+
+***
+
+#### `__init__(self, port: int = 1071, check_version: bool = True, launch_build: bool = True)`
+
+Create the network socket and bind the socket to the port.
+
+| Parameter | Description |
+| --- | --- |
+| port | The port number. |
+| check_version | If true, the controller will check the version of the build and print the result. |
+| launch_build | If True, automatically launch the build. If one doesn't exist, download and extract the correct version. Set this to False to use your own build, or (if you are a backend developer) to use Unity Editor. |
+
+***
+
+#### `communicate(self, commands: Union[dict, List[dict]]) -> list`
+
+Listen for when a key is pressed and send commands.
+
+| Parameter | Description |
+| --- | --- |
+| key | The keyboard key. |
+| commands | Commands to be sent when the key is pressed. |
+| function | A function to be invoked when the key is pressed. |
+
+***
+
+#### `listen(self, key: str, commands: Union[dict, List[dict]] = None, function=None) -> None`
+
+Listen for when a key is pressed and send commands.
+
+| Parameter | Description |
+| --- | --- |
+| key | The keyboard key. |
+| commands | Commands to be sent when the key is pressed. |
+| function | A function to be invoked when the key is pressed. |
+
+***
+
diff --git a/Documentation/python/py_impact.md b/Documentation/python/py_impact.md
index 7d1b55d1b..217bfd2ac 100644
--- a/Documentation/python/py_impact.md
+++ b/Documentation/python/py_impact.md
@@ -327,20 +327,20 @@ c.start()
# Request the required output data (do this at the start of the simulation, not per frame).
resp = c.communicate([{"$type": "send_collisions",
-"enter": True,
-"exit": False,
-"stay": True,
-"collision_types": ["obj", "env"]},
-{"$type": "send_rigidbodies",
-"frequency": "always"}])
+ "enter": True,
+ "exit": False,
+ "stay": True,
+ "collision_types": ["obj", "env"]},
+ {"$type": "send_rigidbodies",
+ "frequency": "always"}])
# Parse the output data and get collision type data.
ctof = CollisionTypesOnFrame(object_id, resp)
# Read the dictionaries of collidee IDs and collision types.
for collidee_id in ctof.collisions:
-collision_type = ctof.collisions[collidee_id]
-print(collidee_id, collision_type)
+ collision_type = ctof.collisions[collidee_id]
+ print(collidee_id, collision_type)
# Check the environment collision.
print(ctof.env_collision_type)
diff --git a/Documentation/python/tdw.md b/Documentation/python/tdw.md
index e0105ad1f..c7399cd4b 100644
--- a/Documentation/python/tdw.md
+++ b/Documentation/python/tdw.md
@@ -12,15 +12,16 @@ pip3 install tdw
### Frontend
-| | Description |
-| ------------------------------------------------------------ | ----------------------------------------------------------- |
-| [Controller](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/controller.md) | Base class for all controllers. |
-| [TDWUtils](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw_utils.md) | Utility class. |
-| [AssetBundleCreator](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/asset_bundle_creator.md) | Covert 3D models into TDW-compatible asset bundles. |
-| [PyImpact](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/py_impact.md) | Generate impact sounds at runtime. |
-| [DebugController](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/debug_controller.md) | Child class of `Controller` that has useful debug features. |
-| [Librarian](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/librarian/librarian.md) | "Librarians" hold asset bundle metadata records. |
-| [FluidTypes](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/fluid_types.md) | Access different NVIDIA Flex fluid types. |
+| | Description |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| [Controller](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/controller.md) | Base class for all controllers. |
+| [TDWUtils](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw_utils.md) | Utility class. |
+| [AssetBundleCreator](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/asset_bundle_creator.md) | Covert 3D models into TDW-compatible asset bundles. |
+| [PyImpact](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/py_impact.md) | Generate impact sounds at runtime. |
+| [DebugController](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/debug_controller.md) | Child class of `Controller` that has useful debug features. |
+| [KeyboardController](keyboard_controller.md) | Child class of `Controller` that can listen for keyboard input. |
+| [Librarian](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/librarian/librarian.md) | "Librarians" hold asset bundle metadata records. |
+| [FluidTypes](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/fluid_types.md) | Access different NVIDIA Flex fluid types. |
### Backend
diff --git a/Python/example_controllers/keyboard.py b/Python/example_controllers/keyboard.py
deleted file mode 100644
index b4f6cdfbb..000000000
--- a/Python/example_controllers/keyboard.py
+++ /dev/null
@@ -1,85 +0,0 @@
-from tdw.controller import Controller
-from tdw.tdw_utils import TDWUtils
-from platform import system
-
-if system() == "Windows":
- from msvcrt import getch
-else:
- from getch import getch
-
-
-"""
-Use WASD or arrow keys to move an avatar.
-"""
-
-
-class KeyboardControls(Controller):
- def run(self, force=80):
- """
- :param force: The force magnitude used to move the avatar.
- """
-
- print("W, up-arrow = Move forward")
- print("S, down-arrow = Move backward")
- print("A, left-arrow = Turn counterclockwise")
- print("D, right-arrow = Turn clockwise")
- print("Esc = Quit")
-
- self.start()
- self.communicate(TDWUtils.create_empty_room(12, 12))
- self.communicate(TDWUtils.create_avatar(avatar_type="A_Img_Caps"))
-
- # Set high drag values so it doesn't feel like the avatar is sliding on ice.
- self.communicate({"$type": "set_avatar_drag",
- "drag": 10,
- "angular_drag": 20})
-
- # Change the floor material so it's easier to visualize the avatar's movement.
- self.communicate([self.get_add_material("parquet_alternating_orange", library="materials_high.json"),
- {"$type": "set_proc_gen_floor_material",
- "name": "parquet_alternating_orange"},
- {"$type": "set_proc_gen_floor_texture_scale",
- "scale": {"x": 8, "y": 8}}])
-
- done = False
- while not done:
- ch = getch()
- ch_int = ord(ch)
- forward = 0
- torque = 0
- # Move forward with w or up-arrow.
- if ch == b'w' or ch_int == 72:
- forward = force
- # Move backward with s or down-arrow.
- elif ch == b's' or ch_int == 80:
- forward = -force
- # Rotate counterclockwise with d or right-arrow.
- elif ch == b'd' or ch_int == 77:
- torque = force
- # Rotate clockwise with a or left-arrow.
- elif ch == b'a' or ch_int == 75:
- torque = -force
- # Quit with esc.
- elif ch_int == 27:
- done = True
- continue
-
- commands = []
- # If there was any directional input, apply a directional force.
- if abs(forward) > 0:
- commands.append({"$type": "move_avatar_forward_by",
- "magnitude": forward})
- # If there was any rotational input, apply a torque.
- if abs(torque) > 0:
- commands.append({"$type": "turn_avatar_by",
- "torque": torque})
-
- # If there was no input, do nothing and advance the simulation.
- # (This isn't necessary in this example, but you'll want this if anything else is moving in the scene).
- if len(commands) > 0:
- commands.append({"$type": "do_nothing"})
-
- self.communicate(commands)
-
-
-KeyboardControls().run()
diff --git a/Python/example_controllers/keyboard_controls.py b/Python/example_controllers/keyboard_controls.py
new file mode 100644
index 000000000..1fa0f1ae2
--- /dev/null
+++ b/Python/example_controllers/keyboard_controls.py
@@ -0,0 +1,86 @@
+from tdw.keyboard_controller import KeyboardController
+from tdw.tdw_utils import TDWUtils
+
+
+"""
+Use WASD or arrow keys to move an avatar.
+"""
+
+
+class KeyboardControls(KeyboardController):
+ def __init__(self, port: int = 1071):
+ super().__init__(port=port)
+ self.done = False
+
+ def run(self, force=80, torque=100):
+ """
+ :param force: The force magnitude used to move the avatar.
+ :param torque: The torque magnitude to turn the avatar by.
+ """
+
+ print("W, up-arrow = Move forward")
+ print("S, down-arrow = Move backward")
+ print("A, left-arrow = Turn counterclockwise")
+ print("D, right-arrow = Turn clockwise")
+ print("Esc = Quit")
+
+ # Listen for keyboard input for movement.
+ self.listen("w", commands={"$type": "move_avatar_forward_by",
+ "magnitude": force,
+ "avatar_id": "a"})
+ self.listen("up", commands={"$type": "move_avatar_forward_by",
+ "magnitude": force,
+ "avatar_id": "a"})
+ self.listen("s", commands={"$type": "move_avatar_forward_by",
+ "magnitude": -force,
+ "avatar_id": "a"})
+ self.listen("down", commands={"$type": "move_avatar_forward_by",
+ "magnitude": -force,
+ "avatar_id": "a"})
+ self.listen("d", commands={"$type": "turn_avatar_by",
+ "torque": torque,
+ "avatar_id": "a"})
+ self.listen("right", commands={"$type": "turn_avatar_by",
+ "torque": torque,
+ "avatar_id": "a"})
+ self.listen("a", commands={"$type": "turn_avatar_by",
+ "torque": -torque,
+ "avatar_id": "a"})
+ self.listen("left", commands={"$type": "turn_avatar_by",
+ "torque": -torque,
+ "avatar_id": "a"})
+ # Listen for keyboard input to quit.
+ self.listen("esc", function=self.stop)
+
+ self.start()
+ # Create the room.
+ commands = [TDWUtils.create_empty_room(12, 12)]
+ # Create the avatar.
+ commands.extend(TDWUtils.create_avatar(avatar_type="A_Img_Caps", avatar_id="a"))
+ # 1. Set high drag values so it doesn't feel like the avatar is sliding on ice.
+ # 2. Set the room's floor material.
+ commands.extend([{"$type": "set_avatar_drag",
+ "drag": 10,
+ "angular_drag": 20,
+ "avatar_id": "a"},
+ self.get_add_material("parquet_alternating_orange", library="materials_high.json"),
+ {"$type": "set_proc_gen_floor_material",
+ "name": "parquet_alternating_orange"},
+ {"$type": "set_proc_gen_floor_texture_scale",
+ "scale": {"x": 8, "y": 8}}])
+ self.communicate(commands)
+ while not self.done:
+ # Listen for keyboard input to add other commands.
+ self.communicate([])
+ self.communicate({"$type": "terminate"})
+
+ def stop(self):
+ """
+ Stop the controller and the build.
+ """
+
+ self.done = True
+
+
+if __name__ == "__main__":
+ KeyboardControls().run()
diff --git a/Python/setup.py b/Python/setup.py
index 14426a80d..33cfebf85 100644
--- a/Python/setup.py
+++ b/Python/setup.py
@@ -31,5 +31,5 @@
include_package_data=True,
keywords='unity simulation ml machine-learning',
install_requires=['pyzmq', 'numpy', 'scipy', 'pillow', 'tqdm', 'psutil', 'boto3', 'botocore', 'requests',
- "pyinstaller"],
+ 'pyinstaller', 'keyboard'],
)
diff --git a/Python/tdw/keyboard_controller.py b/Python/tdw/keyboard_controller.py
new file mode 100644
index 000000000..a73bb9f78
--- /dev/null
+++ b/Python/tdw/keyboard_controller.py
@@ -0,0 +1,84 @@
+from typing import List, Union
+import keyboard
+from tdw.controller import Controller
+
+
+class KeyboardController(Controller):
+ """
+ Listen for keyboard input to send commands.
+
+ Usage:
+
+ ```python
+ from tdw.keyboard_controller import KeyboardController
+ from tdw.tdw_utils import TDWUtils
+
+ def stop():
+ done = True
+
+ done = False
+ c = KeyboardController()
+ c.start()
+
+ # Quit.
+ c.listen(key="esc", commands=None, function=stop)
+
+ # Equivalent to c.start()
+ c.listen(key="r", commands={"$type": "load_scene", "scene_name": "ProcGenScene"}, function=None)
+
+ while not done:
+ # Receive data. Load the scene when r is pressed. Quit when Esc is pressed.
+ c.communicate([])
+ # Stop the build.
+ c.communicate({"$type": "terminate"})
+ ```
+ """
+
+ def __init__(self, port: int = 1071, check_version: bool = True, launch_build: bool = True):
+ """
+ Create the network socket and bind the socket to the port.
+
+ :param port: The port number.
+ :param check_version: If true, the controller will check the version of the build and print the result.
+ :param launch_build: If True, automatically launch the build. If one doesn't exist, download and extract the correct version. Set this to False to use your own build, or (if you are a backend developer) to use Unity Editor.
+ """
+
+ # Commands that should be added due to key presses on this frame.
+ self.on_key_commands: List[dict] = []
+
+ super().__init__(port=port, check_version=check_version, launch_build=launch_build)
+
+ def communicate(self, commands: Union[dict, List[dict]]) -> list:
+ if isinstance(commands, dict):
+ commands = [commands]
+ # Add commands from key presses.
+ commands.extend(self.on_key_commands[:])
+ # Clear the on-key commands.
+ self.on_key_commands.clear()
+ return super().communicate(commands)
+
+ def listen(self, key: str, commands: Union[dict, List[dict]] = None, function=None) -> None:
+ """
+ Listen for when a key is pressed and send commands.
+
+ :param key: The keyboard key.
+ :param commands: Commands to be sent when the key is pressed.
+ :param function: A function to be invoked when the key is pressed.
+ """
+
+ if commands is not None:
+ keyboard.on_press_key(key, lambda e: self._set_frame_commands(commands))
+ if function is not None:
+ keyboard.on_press_key(key, lambda e: function())
+
+ def _set_frame_commands(self, commands: Union[dict, List[dict]]) -> None:
+ """
+ Set the next frame's commands.
+
+ :param commands: The commands to send on this frame.
+ """
+
+ if isinstance(commands, dict):
+ commands = [commands]
+ self.on_key_commands = commands
+
diff --git a/README.md b/README.md
index 1f8211537..35e38746d 100644
--- a/README.md
+++ b/README.md
@@ -29,16 +29,17 @@
##### Frontend
-| Document | Description |
-| ------------------------------------------------------------ | ----------------------------------------------------------- |
-| [`tdw` module](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw.md) | Overview of the Python `tdw` module. |
-| [Controller](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/controller.md) | Base class for all controllers. |
-| [TDWUtils](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw_utils.md) | Utility class. |
-| [AssetBundleCreator](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/asset_bundle_creator.md) | Covert 3D models into TDW-compatible asset bundles. |
-| [PyImpact](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/py_impact.md) | Generate impact sounds at runtime. |
-| [DebugController](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/debug_controller.md) | Child class of `Controller` that has useful debug features. |
-| [Librarian](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/librarian/librarian.md) | "Librarians" hold asset bundle metadata records. |
-| [FluidTypes](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/fluid_types.md) | Access different NVIDIA Flex fluid types. |
+| Document | Description |
+| ------------------------------------------------------------ | ------------------------------------------------------------ |
+| [`tdw` module](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw.md) | Overview of the Python `tdw` module. |
+| [Controller](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/controller.md) | Base class for all controllers. |
+| [TDWUtils](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/tdw_utils.md) | Utility class. |
+| [AssetBundleCreator](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/asset_bundle_creator.md) | Covert 3D models into TDW-compatible asset bundles. |
+| [PyImpact](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/py_impact.md) | Generate impact sounds at runtime. |
+| [DebugController](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/debug_controller.md) | Child class of `Controller` that has useful debug features. |
+| [KeyboardController](Documentation/python/keyboard_controller.md) | Child class of `Controller` that can listen for keyboard input. |
+| [Librarian](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/librarian/librarian.md) | "Librarians" hold asset bundle metadata records. |
+| [FluidTypes](https://github.com/threedworld-mit/tdw/blob/master/Documentation/python/fluid_types.md) | Access different NVIDIA Flex fluid types. |
##### Backend