Output data classes are found in output_data.py. These classes are wrappers for classes auto-generated by flatc, which in turn are wrappers for accessing byte arrays serialized according to Flatbuffer schemas; none of these files should be edited.
Output data is sent from the build as a list of serialized byte arrays of Flatbuffer objects. The last element of the list is the frame number.
from tdw.output_data import OutputData, Images
from tdw.controller import Controller
c = Controller()
# Initialize your scene here.
resp = c.Communicate(commands)
# Iterate through all output data.
# Ignore the last element (frame count).
for i in range(len(resp) - 1):
# Parse the byte array here.You can't directly cast a byte array to a Flatbuffer type. However, each byte array begins with a 4-character identifier:
r_id = OutputData.get_data_type_id(r)
if r_id == "imag":
i = Images(r)This document includes each output data type's identifier.
Objects in arrays can't be directly accessed (this is due to how the backend code is structured). Instead, each output data type has functions with an index parameter:
# Get the image, pass mask, and file extension of each pass.
for i in range(i.get_num_passes()):
img = i.get_image(i)
pass_mask = i.get_pass_mask(i)
ext = i.get_extension(i)| Data Type | Description | Identifier |
|---|---|---|
| AlbedoColors | The albedo colors of each object in the scene. | acol |
| AudioSourceDone | Output data that announces that an audio source is done playing. | ausd |
| AudioSources | Audio data for each object in a scene. Note that this will only tell you if any audio is playing; it won't return the audio itself (use an external program to record audio). | audi |
| AvatarKinematic | Data of a kinematic (non-physics) avatar. | avki |
| AvatarNonKinematic | Data of a non-kinematic (physics-enabled) avatar with a single body object. | avnk |
| AvatarSegmentationColor | Color segmentation data for an avatar. | avsc |
| AvatarSimpleBody | Data of a SimpleBodyAvatar. | avsb |
| AvatarTransformMatrices | 4x4 transform matrices for avatars and sensor containers. | atrm |
| Bounds | Rotated bounds data. | boun |
| CameraMatrices | An avatar's camera matrix data. Each matrix is arranged as m00, m01... m10, m11, etc. | cama |
| Categories | Color segmentation data for object categories. | cate |
| Collision | Data for a collision between objects occurring on this frame. | coll |
| Containment | The IDs of every object that a shape overlaps plus parent IDs and the semantic containment tag. | cont |
| Drones | Data for each drone in the scene. | dron |
| DynamicCompositeObjects | Dynamic data for composite objects. | dcom |
| DynamicEmptyObjects | The position of each empty object in the scene. | dyem |
| DynamicRobots | Dynamic robot output data. | drob |
| EnvironmentColliderIntersection | Data for an whose colliders are intersecting with an environment collider such as a wall. | enci |
| EnvironmentCollision | Data for a collision between and object and the scene environment on this frame. | enco |
| EulerAngles | The rotation of each object in the scene expressed as Euler angles. | eule |
| FieldOfView | An avatar's camera field of view and focal length. | fofv |
| FlexParticles | NVIDIA Flex data. | flex |
| Fove | FOVE headset and eye tracking data. | fove |
| Framerate | Framerate data. | fram |
| IdPassGrayscale | The average grayscale value of the _id pass. | idgs |
| IdPassSegmentationColors | All segmentation colors in an _id pass. | ipsc |
| Images | The images and associated metadata that were captured by an avatar. | imag |
| ImageSensors | The names of each ImageSensor component attached to an avatar, and whether they are enabled. | imse |
| IsOnNavMesh | Data regarding whether a position is on the NavMesh. Invoked by first sending the command send_is_on_nav_mesh. |
isnm |
| Keyboard | Keyboard input. Note that in order to receive keyboard input, the build must be the focused window and running on the same computer as the keyboard. | keyb |
| LeapMotion | Data for Leap Motion hands. | leap |
| Lights | Data for all lights in the scene. | ligh |
| LocalTransforms | Data about the Transform component of objects (position and rotation) relative to its parent objects. | ltra |
| LogMessage | A log message sent by the build. | logm |
| Magnebot | Data for a Magnebot. | magn |
| MagnebotWheels | A message sent when a Magnebot arrives at a target. | mwhe |
| Meshes | Mesh data from readable objects. | mesh |
| Models | Model names and URLs per object. | mode |
| Mouse | Data for mouse input and movement. | mous |
| NavMeshPath | A path on the scene's NavMesh. | path |
| ObiParticles | Obi particle data. | obip |
| ObjectColliderIntersection | Data for two objects whose colliders are intersecting. | obci |
| ObjectScales | The spatial scale of each object in the scene. | osca |
| Occlusion | To what extent parts of the scene environment (such as walls) are occluding objects. | occl |
| OccupancyMap | A grid of positions denoting whether a space is occupied, free, or out of bounds. | occu |
| OculusTouchButtons | Which Oculus Touch controller buttons have been pressed. | octb |
| Overlap | The IDs of every object that a shape overlaps. | over |
| PostProcess | Post-processing values. | post |
| QuitSignal | A message sent by the build when it quits. | quit |
| Raycast | A ray cast from an origin to a destination and what, if anything, it hit. | rayc |
| Replicants | Data about each Replicant in the scene. | repl |
| ReplicantSegmentationColors | Color segmentation data for Replicants in the scene. | rseg |
| Rigidbodies | Dynamic rigibody data (velocity, angular velocity, etc.) for objects in the scene. | rigi |
| RobotJointVelocities | Velocity for a robot in the scene. | rojv |
| Scene | The scene name and URL of the asset bundle. | scen |
| SceneRegions | Data regarding the scene regions. | sreg |
| ScreenPosition | A position on the screen converted from a worldspace position. | scre |
| SegmentationColors | Color segmentation data for objects in the scene. | segm |
| StaticCompositeObjects | Static data for composite objects. | scom |
| StaticEmptyObjects | Static data for empty objects in the scene. | stem |
| StaticOculusTouch | Static data for the Oculus Touch rig. | soct |
| StaticRigidbodies | Static rigibody data (mass, kinematic state, etc.) for objects in the scene. | srig |
| StaticRobot | Static data for a robot in the scene. | srob |
| Substructure | The substructure of a model. This should be used mainly for backend debugging. | subs |
| SystemInfo | System and hardware information. | syst |
| TransformMatrices | 4x4 transform matrices for each object in the scene. | trma |
| Transforms | Data about the Transform component of objects (position and rotation). | tran |
| TriggerCollision | Data for a non-physics trigger collision event. | trco |
| Version | The build version and Unity version. | vers |
| Volumes | Spatial volume data for objects in the scene. | volu |
| VRRig | Data about the VR rig currently in the scene. | vrri |
| AvatarIds | The IDs of each avatar in the scene. | avid |
| FastAvatars | Fast, fixed-length avatar transform data. Use this in conjunction with FastAvatarIds: the order of the IDs matches the order of this data. | fava |
| FastImageSensors | Fast, fixed-length avatar image sensor transform data. Use this in conjunction with FastAvatarIds: the order of the IDs matches the order of this data. | fims |
| FastTransforms | None | None |
| ObjectIds | The IDs of all Rigidbody objects (models and composite sub-objects) in the scene. | obid |
a = AlbedoColors(byte_array)
Identifier: acol
The albedo colors of each object in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_color(index) |
The color. | np.ndarray |
a = AudioSourceDone(byte_array)
Identifier: ausd
Output data that announces that an audio source is done playing.
| Function | Description | Return type |
|---|---|---|
get_id() |
The audio source ID. | int |
a = AudioSources(byte_array)
Identifier: audi
Audio data for each object in a scene. Note that this will only tell you if any audio is playing; it won't return the audio itself (use an external program to record audio).
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_object_id(index) |
The ID of the object. | int |
get_is_playing(index) |
If true, the audio source is currently playing a sound. | bool |
get_samples() |
Audio samples from the audio listener. | np.ndarray |
a = AvatarKinematic(byte_array)
Identifier: avki
Data of a kinematic (non-physics) avatar.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_position() |
The local bottom-center position of the avatar. | Tuple[float, float, float] |
get_rotation() |
The current rotation of the avatar. | Tuple[float, float, float, float] |
get_forward() |
The forward directional vector of the avatar. | Tuple[float, float, float] |
a = AvatarNonKinematic(byte_array)
Identifier: avnk
Data of a non-kinematic (physics-enabled) avatar with a single body object.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_position() |
The local bottom-center position of the avatar. | Tuple[float, float, float] |
get_rotation() |
The current rotation of the avatar. | Tuple[float, float, float, float] |
get_forward() |
The forward directional vector of the avatar. | Tuple[float, float, float] |
get_velocity() |
The directional velocity. | Tuple[float, float, float] |
get_angular_velocity() |
The angular velocity. | Tuple[float, float, float] |
get_mass() |
The mass. | float |
get_sleeping() |
True if the rigidbody is sleeping. | bool |
a = AvatarSegmentationColor(byte_array)
Identifier: avsc
Color segmentation data for an avatar.
| Function | Description | Return type |
|---|---|---|
get_id() |
The ID of the avatar. | str |
get_segmentation_color() |
The segmentation color. | Tuple[float, float, float] |
a = AvatarSimpleBody(byte_array)
Identifier: avsb
Data of a SimpleBodyAvatar.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_position() |
The local bottom-center position of the avatar. | Tuple[float, float, float] |
get_rotation() |
The current rotation of the avatar. | Tuple[float, float, float, float] |
get_forward() |
The forward directional vector of the avatar. | Tuple[float, float, float] |
get_velocity() |
The directional velocity. | Tuple[float, float, float] |
get_angular_velocity() |
The angular velocity. | Tuple[float, float, float] |
get_mass() |
The mass. | float |
get_sleeping() |
True if the rigidbody is sleeping. | bool |
get_visible_body() |
The name of the current visible body. | str |
a = AvatarTransformMatrices(byte_array)
Identifier: atrm
4x4 transform matrices for avatars and sensor containers.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | str |
get_avatar_matrix(index) |
The matrix of the avatar. | np.array |
get_sensor_matrix(index) |
The matrix of the sensor. | np.array |
b = Bounds(byte_array)
Identifier: boun
Rotated bounds data.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_front(index) |
The front. | np.ndarray |
get_back(index) |
The back. | np.ndarray |
get_left(index) |
The left. | np.ndarray |
get_right(index) |
The right. | np.ndarray |
get_top(index) |
The top. | np.ndarray |
get_bottom(index) |
The bottom. | np.ndarray |
get_center(index) |
The center. | np.ndarray |
c = CameraMatrices(byte_array)
Identifier: cama
An avatar's camera matrix data. Each matrix is arranged as m00, m01... m10, m11, etc.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_sensor_name() |
The name of the sensor that captured the image. | str |
get_projection_matrix() |
The projection matrix. | np.ndarray |
get_camera_matrix() |
The camera matrix. | np.ndarray |
c = Categories(byte_array)
Identifier: cate
Color segmentation data for object categories.
| Function | Description | Return type |
|---|---|---|
get_num_categories() |
The number of categories. | int |
get_category_name(index) |
The name of the category. | str |
get_category_color(index) |
The color of the category. | Tuple[float, float, float] |
c = Collision(byte_array)
Identifier: coll
Data for a collision between objects occurring on this frame.
| Function | Description | Return type |
|---|---|---|
get_collider_id() |
The ID of the object that registered the collision. | int |
get_collidee_id() |
The ID of the object that collided with the collider. | int |
get_relative_velocity() |
The relative linear velocity of the collision. | Tuple[float, float, float] |
get_impulse() |
The total impulse applied to the pair of objects to resolve the collision. | Tuple[float, float, float] |
get_state() |
The state of the collision. | str |
get_num_contacts() |
The number of contacts. | int |
get_contact_normal(index) |
The normal of the contact. | Tuple[float, float, float] |
get_contact_point(index) |
The point of the contact. | Tuple[float, float, float] |
c = Containment(byte_array)
Identifier: cont
The IDs of every object that a shape overlaps plus parent IDs and the semantic containment tag.
| Function | Description | Return type |
|---|---|---|
get_object_id() |
The ID of the object. | int |
get_container_id() |
The ID of the container. | int |
get_tag() |
The semantic tag. | ContainerTag |
get_overlap_ids() |
The IDs of every object in the overlap shape. | np.ndarray |
get_env() |
If true, the overlap shape includes at least one environment object (such as the floor). | bool |
get_walls() |
If true, the overlap shape includes at least one environment object that isn't the floor. | bool |
d = Drones(byte_array)
Identifier: dron
Data for each drone in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_raycast_hit(index) |
The hit of the raycast. | bool |
get_raycast(index) |
The raycast. | np.ndarray |
get_motor_on(index) |
If true, the drone's motor is on. | bool |
d = DynamicCompositeObjects(byte_array)
Identifier: dcom
Dynamic data for composite objects.
| Function | Description | Return type |
|---|---|---|
get_num_hinges() |
The number of hinges. | int |
get_hinge_parent_id(index) |
The ID of the hinge parent. | int |
get_hinge_id(index) |
The ID of the hinge. | int |
get_hinge_angle(index) |
The angle of the hinge. | float |
get_hinge_velocity(index) |
The velocity of the hinge. | float |
get_num_lights() |
The number of lights. | int |
get_light_parent_id(index) |
The ID of the light parent. | int |
get_light_id(index) |
The ID of the light. | int |
get_light_is_on(index) |
The on of the light is. | bool |
d = DynamicEmptyObjects(byte_array)
Identifier: dyem
The position of each empty object in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of positions. | int |
get_position(index) |
The position. | np.ndarray |
d = DynamicRobots(byte_array)
Identifier: drob
Dynamic robot output data.
| Function | Description | Return type |
|---|---|---|
get_immovable(index) |
An array indicating whether the root object of each robot is immovable. | bool |
get_robot_position(index) |
The position of the robot in the order of (x, y, z). | np.ndarray |
get_robot_rotation(index) |
The rotation of the robot in the order (x, y, z, w). | np.ndarray |
get_robot_forward(index) |
The forward of the robot. | np.ndarray |
get_joint_position(index) |
The position of the joint in the order of (x, y, z). | np.ndarray |
get_joint_angles(index) |
The angles of the joint. | np.ndarray |
get_joint_sleeping(index) |
The sleeping of the joint. | bool |
e = EnvironmentColliderIntersection(byte_array)
Identifier: enci
Data for an whose colliders are intersecting with an environment collider such as a wall.
| Function | Description | Return type |
|---|---|---|
get_object_id() |
The ID of the object. | int |
get_distance() |
The distance along direction that is required to separate the colliders apart. | float |
get_direction() |
The direction along which the translation required to separate the colliders apart is minimal. | Tuple[float, float, float] |
e = EnvironmentCollision(byte_array)
Identifier: enco
Data for a collision between and object and the scene environment on this frame.
| Function | Description | Return type |
|---|---|---|
get_object_id() |
The ID of the object. | int |
get_state() |
The state of the collision. | str |
get_num_contacts() |
The number of contacts. | int |
get_contact_normal(index) |
The normal of the contact. | Tuple[float, float, float] |
get_contact_point(index) |
The point of the contact. | Tuple[float, float, float] |
get_floor() |
If True, this is the floor. | bool |
e = EulerAngles(byte_array)
Identifier: eule
The rotation of each object in the scene expressed as Euler angles.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_rotation(index) |
The rotation. | np.ndarray |
f = FieldOfView(byte_array)
Identifier: fofv
An avatar's camera field of view and focal length.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_sensor_name() |
The name of the sensor that captured the image. | str |
get_fov() |
The field of view. | float |
get_focal_length() |
The focal length. | float |
f = FlexParticles(byte_array)
Identifier: flex
NVIDIA Flex data.
| Function | Description | Return type |
|---|---|---|
get_num_objects() |
The number of objects. | int |
get_particles(index) |
The Flex particle positions and inverse mass. | np.ndarray |
get_velocities(index) |
The Flex particle velocities. | np.ndarray |
get_id(index) |
The unique ID of the object. | int |
f = Fove(byte_array)
Identifier: fove
FOVE headset and eye tracking data.
| Function | Description | Return type |
|---|---|---|
get_eye_direction(index) |
The direction of the eye. | np.ndarray |
get_eye_state(index) |
The state of the eye. | EyeState |
get_hit(index) |
The hit. | bool |
get_object_hit(index) |
The hit of the object. | bool |
get_object_id(index) |
The ID of the object. | int |
get_hit_position(index) |
The position of the hit in the order of (x, y, z). | np.ndarray |
get_combined_depth() |
The combined depth value. | float |
get_is_calibrated() |
The calibrated of the is. | bool |
f = Framerate(byte_array)
Identifier: fram
Framerate data.
| Function | Description | Return type |
|---|---|---|
get_target_framerate() |
The target framerate. | int |
get_frame_dt() |
The time elapsed for this frame in seconds. | float |
get_physics_timestep() |
The timestep of the physics. | float |
i = IdPassGrayscale(byte_array)
Identifier: idgs
The average grayscale value of the _id pass.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar that captured the image. | str |
get_sensor_name() |
The name of the sensor that captured the image. | str |
get_grayscale() |
The average grayscale value. A higher value means that more of the image pixels are non-black. | float |
i = IdPassSegmentationColors(byte_array)
Identifier: ipsc
All segmentation colors in an _id pass.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar that captured the image. | str |
get_num_segmentation_colors() |
The number of segmentation colors. | int |
get_segmentation_color(index) |
The color of the segmentation. | np.ndarray |
i = Images(byte_array)
Identifier: imag
The images and associated metadata that were captured by an avatar.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar that captured the image. | str |
get_sensor_name() |
The name of the sensor that captured the image. | str |
get_num_passes() |
The number of passes. | int |
get_pass_mask(index) |
The ID of the environment. | str |
get_image(index) |
The image. | np.ndarray |
get_extension(index) |
The image file extension (.png or .jpg). | str |
get_width() |
The width of the screen in pixels. | int |
get_height() |
The height of the screen in pixels. | int |
i = ImageSensors(byte_array)
Identifier: imse
The names of each ImageSensor component attached to an avatar, and whether they are enabled.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar. | str |
get_num_sensors() |
The number of sensors. | int |
get_sensor_name(index) |
The name of the sensor. | str |
get_sensor_on(index) |
The on of the sensor. | bool |
get_sensor_rotation(index) |
The rotation of the sensor in the order (x, y, z, w). | Tuple[float, float, float, float] |
get_sensor_forward(index) |
The forward of the sensor. | Tuple[float, float, float] |
get_sensor_field_of_view(index) |
The view of the sensor field of. | float |
i = IsOnNavMesh(byte_array)
Identifier: isnm
Data regarding whether a position is on the NavMesh. Invoked by first sending the command send_is_on_nav_mesh.
| Function | Description | Return type |
|---|---|---|
get_position() |
A valid position on the NavMesh. If is_on == False, this is [0, 0, 0]. | np.ndarray |
get_is_on() |
True if position is a valid position on the NavMesh. |
bool |
get_id() |
The ID of this output data object. | int |
k = Keyboard(byte_array)
Identifier: keyb
Keyboard input. Note that in order to receive keyboard input, the build must be the focused window and running on the same computer as the keyboard.
| Function | Description | Return type |
|---|---|---|
get_num_pressed() |
The number of pressed. | int |
get_pressed(index) |
Keys that were first pressed between the previous frame and this frame. | str |
get_num_held() |
The number of held. | int |
get_held(index) |
Keys that are being pressed down. | str |
get_num_released() |
The number of released. | int |
get_released(index) |
Keys that were released between the previous frame and this frame. | str |
l = LeapMotion(byte_array)
Identifier: leap
Data for Leap Motion hands.
| Function | Description | Return type |
|---|---|---|
get_num_collisions_per_bone() |
The number of collisions per bone. | int |
get_position(index, bone_index) |
The position. | np.ndarray |
get_rotation(index, bone_index) |
The rotation. | np.ndarray |
get_forward(index, bone_index) |
The forward. | np.ndarray |
get_is_collision(index, bone_index, collision_index) |
The collision of the is. | bool |
get_collision_id(index, bone_index, collision_index) |
The ID of the collision. | int |
get_angles(index, start_bone_index, end_bone_index) |
The angles in degrees of each bone drive axis. | np.ndarray |
get_is_button_pressed(index) |
The pressed of the is button. | bool |
l = Lights(byte_array)
Identifier: ligh
Data for all lights in the scene.
| Function | Description | Return type |
|---|---|---|
get_num_directional_lights() |
The number of directional lights. | int |
get_directional_light_intensity(index) |
The intensity of the directional light. | float |
get_directional_light_color(index) |
The color of the directional light. | Tuple[float, float, float] |
get_directional_light_rotation(index) |
The rotation of the directional light in the order (x, y, z, w). | Tuple[float, float, float, float] |
get_num_point_lights() |
The number of point lights. | int |
get_point_light_intensity(index) |
The intensity of the point light. | float |
get_point_light_color(index) |
The color of the point light. | Tuple[float, float, float] |
get_point_light_position(index) |
The position of the point light in the order of (x, y, z). | Tuple[float, float, float] |
get_point_light_range(index) |
The range of the point light. | float |
l = LocalTransforms(byte_array)
Identifier: ltra
Data about the Transform component of objects (position and rotation) relative to its parent objects.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_position(index) |
The position of the object's pivot point, in the order (x, y, z). | np.ndarray |
get_forward(index) |
The forward directional vector, in worldspace rotational coordinates. | np.ndarray |
get_rotation(index) |
The rotation. | np.ndarray |
get_euler_angles(index) |
The [x, y, z] Euler angles of each object. |
np.ndarray |
l = LogMessage(byte_array)
Identifier: logm
A log message sent by the build.
| Function | Description | Return type |
|---|---|---|
get_message() |
The message. | str |
get_message_type() |
The type of message. | str |
get_object_type() |
The type of object that logged this message. | str |
m = Magnebot(byte_array)
Identifier: magn
Data for a Magnebot.
| Function | Description | Return type |
|---|---|---|
get_id() |
The unique ID of the Magnebot. | int |
get_held_left() |
The IDs of the objects held by the left magnet. | np.ndarray |
get_held_right() |
The IDs of the objects held by the right magnet. | np.ndarray |
get_top() |
The position of the top point of the Magnebot. | Tuple[float, float, float] |
m = MagnebotWheels(byte_array)
Identifier: mwhe
A message sent when a Magnebot arrives at a target.
| Function | Description | Return type |
|---|---|---|
get_id() |
The ID of the Magnebot. | int |
get_success() |
If True, the Magnebot arrived at the target. | bool |
m = Meshes(byte_array)
Identifier: mesh
Mesh data from readable objects.
| Function | Description | Return type |
|---|---|---|
get_object_id(index) |
The ID of the object. | int |
get_num() |
The number of objects. | int |
get_vertices(index) |
The (x, y, z) coordinates of each vertex. | np.ndarray |
get_triangles(index) |
Each triangle of the mesh. | np.ndarray |
m = Models(byte_array)
Identifier: mode
Model names and URLs per object.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_name(index) |
The name. | str |
get_url(index) |
The url. | str |
m = Mouse(byte_array)
Identifier: mous
Data for mouse input and movement.
| Function | Description | Return type |
|---|---|---|
get_position() |
The (x, y) screen position of the mouse. | np.ndarray |
get_scroll_delta() |
The (x, y) delta of the scroll wheel. | np.ndarray |
get_is_left_button_pressed() |
The pressed of the is left button. | bool |
get_is_left_button_held() |
The held of the is left button. | bool |
get_is_left_button_released() |
The released of the is left button. | bool |
get_is_middle_button_pressed() |
The pressed of the is middle button. | bool |
get_is_middle_button_held() |
The held of the is middle button. | bool |
get_is_middle_button_released() |
The released of the is middle button. | bool |
get_is_right_button_pressed() |
The pressed of the is right button. | bool |
get_is_right_button_held() |
The held of the is right button. | bool |
get_is_right_button_released() |
The released of the is right button. | bool |
n = NavMeshPath(byte_array)
Identifier: path
A path on the scene's NavMesh.
| Function | Description | Return type |
|---|---|---|
get_state() |
The state of the path: "complete", "partial", or "invalid". | str |
get_path() |
Waypoints on the path as a numpy array of (x, y, z) coordinates. | np.ndarray |
get_id() |
The ID of this path. Use this to differentiate between different NavMeshPaths. | int |
o = ObiParticles(byte_array)
Identifier: obip
Obi particle data.
| Function | Description | Return type |
|---|---|---|
get_num_solvers() |
The number of solvers. | int |
get_positions(index) |
The positions of the particles. | np.ndarray |
get_velocities(index) |
The velocities of the particles. | np.ndarray |
get_num_objects() |
The number of objects. | int |
get_object_id(index) |
The ID of the object. | int |
get_solver_id(index) |
The ID of the actor's solver. | int |
get_count(index) |
The number of active particles. | int |
get_solver_indices(index) |
Indices of the particles in the solver. | np.ndarray |
o = ObjectColliderIntersection(byte_array)
Identifier: obci
Data for two objects whose colliders are intersecting.
| Function | Description | Return type |
|---|---|---|
get_object_id_a() |
The ID of the first object. | int |
get_object_id_b() |
The ID of the second object. | int |
get_distance() |
The distance along direction that is required to separate the colliders apart. | float |
get_direction() |
The direction along which the translation required to separate the colliders apart is minimal. | Tuple[float, float, float] |
o = ObjectScales(byte_array)
Identifier: osca
The spatial scale of each object in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_scale(index) |
The scale. | np.ndarray |
o = Occlusion(byte_array)
Identifier: occl
To what extent parts of the scene environment (such as walls) are occluding objects.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar that captured the image. | str |
get_occluded() |
An integer between 0 and 255 describing what fraction of the image's pixels are occupied by objects, as opposed to background meshes. A value of 255 would mean that the image is totally, or near-totally, occupied by objects. | int |
get_unoccluded() |
An integer between 0 and 255 describing what fraction of the image's pixels are occupied by objects if background meshes aren't rendered. A value of 255 would mean that the image is totally, or near-totally, occupied by objects. | int |
o = OccupancyMap(byte_array)
Identifier: occu
A grid of positions denoting whether a space is occupied, free, or out of bounds.
| Function | Description | Return type |
|---|---|---|
get_shape() |
Reshape the occupancy map array to this shape. | np.ndarray |
get_map() |
The occupancy map. | np.ndarray |
get_positions() |
An array of (x, z) positions on the occupancy map. | np.ndarray |
o = OculusTouchButtons(byte_array)
Identifier: octb
Which Oculus Touch controller buttons have been pressed.
| Function | Description | Return type |
|---|---|---|
get_left() |
Bitwise sum of buttons pressed on the left controller. | List[OculusTouchButton] |
get_right() |
Bitwise sum of buttons pressed on the right controller. | List[OculusTouchButton] |
get_left_axis() |
Left control stick direction as an [x, y] vector (values range from -1 to 1). |
np.ndarray |
get_right_axis() |
Right control stick direction as an [x, y] vector (values range from -1 to 1). |
np.ndarray |
o = Overlap(byte_array)
Identifier: over
The IDs of every object that a shape overlaps.
| Function | Description | Return type |
|---|---|---|
get_id() |
The identity of this overlap (useful if you've requested multiple Overlap objects). | int |
get_object_ids() |
The IDs of every object in the overlap shape. | np.ndarray |
get_env() |
If true, the overlap shape includes at least one environment object (such as the floor). | bool |
get_walls() |
If true, the overlap shape includes at least one environment object that isn't the floor. | bool |
p = PostProcess(byte_array)
Identifier: post
Post-processing values.
| Function | Description | Return type |
|---|---|---|
get_enabled() |
Whether post-processing is enabled. | bool |
get_ambient_occlusion_intensity() |
The intensity (darkness) of the Ambient Occulusion effect. | float |
get_ambient_occlusion_thickness_modifier() |
The spread of the Ambient Occlusion effect from corners. | float |
get_aperture() |
The depth-of-field aperture. | float |
get_focus_distance() |
The depth-of-field focus distance. | float |
get_contrast() |
The color grading contrast. | float |
get_post_exposure() |
The post-exposure value, which affects brightness. | float |
get_saturation() |
The color grading saturation. | float |
get_screen_space_reflections() |
Whether screen-space reflections are enabled. | float |
get_vignette() |
Whether the vignette is enabled. | float |
q = QuitSignal(byte_array)
Identifier: quit
A message sent by the build when it quits.
| Function | Description | Return type |
|---|---|---|
get_ok() |
If True, the build quit gracefully. If False, the build quit because there was an error or exception. | bool |
r = Raycast(byte_array)
Identifier: rayc
A ray cast from an origin to a destination and what, if anything, it hit.
| Function | Description | Return type |
|---|---|---|
get_raycast_id() |
The identity of this raycast. | int |
get_hit() |
If true, the raycast hit something. | bool |
get_hit_object() |
If true, the raycast hit an object. | bool |
get_object_id() |
The ID of the object hit by the raycast. Ignore this if hit_object is false. | Optional[int] |
get_normal() |
The normal of the surface that the raycast hit. | Tuple[float, float, float] |
get_point() |
The point that the raycast hit. | Tuple[float, float, float] |
r = Replicants(byte_array)
Identifier: repl
Data about each Replicant in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_position(index) |
The position. | np.ndarray |
get_forward(index) |
The forward. | np.ndarray |
get_rotation(index) |
The rotation. | np.ndarray |
get_body_part_id(index, body_part_index) |
The ID of the body part. | int |
get_body_part_position(index, body_part_index) |
The position of the body part in the order of (x, y, z). | np.ndarray |
get_body_part_rotation(index, body_part_index) |
The rotation of the body part in the order (x, y, z, w). | np.ndarray |
get_body_part_forward(index, body_part_index) |
The forward of the body part. | np.ndarray |
get_is_holding_left(index) |
The left of the is holding. | bool |
get_held_left(index) |
The left of the held. | int |
get_is_holding_right(index) |
The right of the is holding. | bool |
get_held_right(index) |
The right of the held. | int |
get_is_collision(index, body_part_index, collision_index) |
The collision of the is. | bool |
get_collision_id(index, body_part_index, collision_index) |
The ID of the collision. | int |
get_status(index) |
The status. | ActionStatus |
get_num_body_parts() |
The number of body parts per Replicant. | int |
r = ReplicantSegmentationColors(byte_array)
Identifier: rseg
Color segmentation data for Replicants in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_segmentation_color(index) |
The color of the segmentation. | np.ndarray |
r = Rigidbodies(byte_array)
Identifier: rigi
Dynamic rigibody data (velocity, angular velocity, etc.) for objects in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_velocity(index) |
The velocity. | np.ndarray |
get_angular_velocity(index) |
The angular velocity. | np.ndarray |
get_sleeping(index) |
The sleeping. | bool |
r = RobotJointVelocities(byte_array)
Identifier: rojv
Velocity for a robot in the scene.
| Function | Description | Return type |
|---|---|---|
get_id() |
The ID of the robot. | int |
get_num_joints() |
The number of joints. | int |
get_joint_id(index) |
The ID of the joint. | int |
get_joint_velocity(index) |
The velocity of the joint. | np.ndarray |
get_joint_angular_velocity(index) |
The angular velocity of the joint. | np.ndarray |
get_joint_sleeping(index) |
The sleeping of the joint. | bool |
s = Scene(byte_array)
Identifier: scen
The scene name and URL of the asset bundle.
| Function | Description | Return type |
|---|---|---|
get_name() |
The name of the scene. | str |
get_url() |
The URL of the asset bundle. | str |
s = SceneRegions(byte_array)
Identifier: sreg
Data regarding the scene regions.
| Function | Description | Return type |
|---|---|---|
get_center(index) |
The centerpoint of the region. | Tuple[float, float, float] |
get_bounds(index) |
The size of the region. | Tuple[float, float, float] |
get_id(index) |
The ID of the region. | int |
get_num() |
The number of regions. | int |
s = ScreenPosition(byte_array)
Identifier: scre
A position on the screen converted from a worldspace position.
| Function | Description | Return type |
|---|---|---|
get_avatar_id() |
The ID of the avatar that is rendered the screen. | str |
get_sensor_name() |
The name of the sensor that rendered the screen. | str |
get_id() |
An identifier for the screen position to help you map it back to the original world position. | int |
get_screen() |
The position in screenspace coordinates. From the Unity documentation: Screenspace is defined in pixels. The bottom-left of the screen is (0,0); the right-top is (pixelWidth,pixelHeight). The z position is in world units from the camera. | Tuple[float, float, float] |
get_world() |
The position in worldspace coordinates. | Tuple[float, float, float] |
s = SegmentationColors(byte_array)
Identifier: segm
Color segmentation data for objects in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_object_id(index) |
The ID of the object. | int |
get_object_color(index) |
The color of the object. | np.ndarray |
get_object_name(index) |
The name of the object. | str |
get_object_category(index) |
The category of the object. | str |
s = StaticCompositeObjects(byte_array)
Identifier: scom
Static data for composite objects.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_object_id(index) |
The ID of the object. | int |
get_num_non_machines(index) |
The number of non machines. | int |
get_non_machine_id(index, non_machine_index) |
The ID of the non machine. | int |
get_num_lights(index) |
The number of lights. | int |
get_light_id(index, light_index) |
The ID of the light. | int |
get_num_hinges(index) |
The number of hinges. | int |
get_hinge_id(index, hinge_index) |
The ID of the hinge. | int |
get_hinge_has_limits(index, hinge_index) |
The limits of the hinge has. | bool |
get_hinge_min_limit(index, hinge_index) |
The limit of the hinge min. | float |
get_hinge_max_limit(index, hinge_index) |
The limit of the hinge max. | float |
get_hinge_axis(index, hinge_index) |
The axis of the hinge. | Tuple[float, float, float] |
get_num_motors(index) |
The number of motors. | int |
get_motor_id(index, motor_index) |
The ID of the motor. | int |
get_motor_has_limits(index, hinge_index) |
The limits of the motor has. | bool |
get_motor_min_limit(index, hinge_index) |
The limit of the motor min. | float |
get_motor_max_limit(index, hinge_index) |
The limit of the motor max. | float |
get_motor_axis(index, hinge_index) |
The axis of the motor. | Tuple[float, float, float] |
get_motor_force(index, motor_index) |
The force of the motor. | float |
get_num_springs(index) |
The number of springs. | int |
get_spring_id(index, spring_index) |
The ID of the spring. | int |
get_spring_has_limits(index, hinge_index) |
The limits of the spring has. | bool |
get_spring_min_limit(index, hinge_index) |
The limit of the spring min. | float |
get_spring_max_limit(index, hinge_index) |
The limit of the spring max. | float |
get_spring_axis(index, hinge_index) |
The axis of the spring. | Tuple[float, float, float] |
get_spring_force(index, spring_index) |
The force of the spring. | float |
get_spring_damper(index, spring_index) |
The damper of the spring. | float |
get_num_prismatic_joints(index) |
The number of prismatic joints. | int |
get_prismatic_joint_id(index, prismatic_joint_index) |
The ID of the prismatic joint. | int |
get_prismatic_joint_limit(index, prismatic_joint_index) |
The limit of the prismatic joint. | float |
get_prismatic_joint_axis(index, prismatic_joint_index) |
The axis of the prismatic joint. | Tuple[float, float, float] |
s = StaticEmptyObjects(byte_array)
Identifier: stem
Static data for empty objects in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of ids. | int |
get_object_id(index) |
The ID of the object. | int |
get_empty_object_id(index) |
The ID of the empty object. | int |
s = StaticOculusTouch(byte_array)
Identifier: soct
Static data for the Oculus Touch rig.
| Function | Description | Return type |
|---|---|---|
get_body_id() |
The ID of the body. | int |
get_left_hand_id() |
The ID of the left hand. | int |
get_right_hand_id() |
The ID of the right hand. | int |
get_human_hands() |
If True, the hands are human hands. | bool |
s = StaticRigidbodies(byte_array)
Identifier: srig
Static rigibody data (mass, kinematic state, etc.) for objects in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_mass(index) |
The mass. | float |
get_kinematic(index) |
The kinematic states of each object's Rigidbody. | bool |
get_dynamic_friction(index) |
The friction of the dynamic. | float |
get_static_friction(index) |
The friction of the static. | float |
get_bounciness(index) |
The bounciness. | float |
s = StaticRobot(byte_array)
Identifier: srob
Static data for a robot in the scene.
| Function | Description | Return type |
|---|---|---|
get_id() |
The ID of this robot. | int |
get_num_joints() |
The number of joints. | int |
get_joint_id(index) |
The ID of the joint. | int |
get_joint_segmentation_color(index) |
The color of the joint segmentation. | Tuple[float, float, float] |
get_joint_mass(index) |
The mass of the joint. | float |
get_is_joint_immovable(index) |
The immovable of the is joint. | bool |
get_is_joint_root(index) |
The root of the is joint. | bool |
get_joint_parent_id(index) |
The ID of the joint parent. | int |
get_joint_name(index) |
The name of the joint. | str |
get_joint_type(index) |
The type of joint. | str |
get_num_joint_drives(index) |
The number of joint drives. | int |
get_joint_drive_axis(index, drive_index) |
The axis of the joint drive. | str |
get_joint_drive_limits(index, drive_index) |
The limits of the joint drive. | bool |
get_joint_drive_lower_limit(index, drive_index) |
The limit of the joint drive lower. | float |
get_joint_drive_upper_limit(index, drive_index) |
The limit of the joint drive upper. | float |
get_joint_drive_force_limit(index, drive_index) |
The limit of the joint drive force. | float |
get_joint_drive_stiffness(index, drive_index) |
The stiffness of the joint drive. | float |
get_joint_drive_damping(index, drive_index) |
The damping of the joint drive. | float |
get_num_non_moving() |
The number of non moving. | int |
get_non_moving_id(index) |
The ID of the non moving. | int |
get_non_moving_name(index) |
The name of the non moving. | str |
get_non_moving_segmentation_color(index) |
The color of the non moving segmentation. | Tuple[float, float, float] |
get_joint_indices() |
The joint IDs and their indices in the static data. | np.ndarray |
get_robot_index() |
The index of the robot. | int |
s = Substructure(byte_array)
Identifier: subs
The substructure of a model. This should be used mainly for backend debugging.
| Function | Description | Return type |
|---|---|---|
get_num_sub_objects() |
The number of sub objects. | int |
get_sub_object_name(index) |
The name of the sub object. | str |
get_num_sub_object_materials(index) |
The number of sub object materials. | int |
get_sub_object_material(index, material_index) |
The material of the sub object. | str |
s = SystemInfo(byte_array)
Identifier: syst
System and hardware information.
| Function | Description | Return type |
|---|---|---|
get_os() |
The operating system. | str |
get_cpu() |
The processor name. | str |
get_browser() |
The web browser. WebGL only. | str |
get_gpu() |
The name of the GPU. | str |
get_graphics_api() |
The graphics API. | str |
get_time_since_startup() |
The time in ticks (100-nanosecond intervals) since startup. | int |
get_screen_width() |
The pixel width of the application window. | int |
get_screen_height() |
The pixel height of the application window. | int |
t = TransformMatrices(byte_array)
Identifier: trma
4x4 transform matrices for each object in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_matrix(index) |
The matrix. | np.array |
t = Transforms(byte_array)
Identifier: tran
Data about the Transform component of objects (position and rotation).
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | int |
get_position(index) |
The position of the object's pivot point, in the order (x, y, z). | np.ndarray |
get_forward(index) |
The forward. | np.ndarray |
get_rotation(index) |
The rotation. | np.ndarray |
t = TriggerCollision(byte_array)
Identifier: trco
Data for a non-physics trigger collision event.
| Function | Description | Return type |
|---|---|---|
get_collidee_id() |
The ID of the object that collided with the trigger collider. | int |
get_collider_id() |
The ID of the object that has the trigger collider. | int |
get_trigger_id() |
The ID of the trigger collider. | int |
get_state() |
The state of the collision. | str |
v = Version(byte_array)
Identifier: vers
The build version and Unity version.
| Function | Description | Return type |
|---|---|---|
get_unity_version() |
The version of the unity. | str |
get_tdw_version() |
The version of the tdw. | str |
get_standalone() |
True if the build is a standalone binary. False if the build is Unity Editor. | bool |
v = Volumes(byte_array)
Identifier: volu
Spatial volume data for objects in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_object_id(index) |
The ID of the object. | int |
get_volume(index) |
The approximate volume in cubic meters of each object. | float |
v = VRRig(byte_array)
Identifier: vrri
Data about the VR rig currently in the scene.
| Function | Description | Return type |
|---|---|---|
get_position() |
The rig's position. | Tuple[float, float, float] |
get_rotation() |
The rig's rotation. | Tuple[float, float, float, float] |
get_forward() |
The rig's forward directional vector. | Tuple[float, float, float] |
get_left_hand_position() |
The position of the left hand in the order of (x, y, z). | Tuple[float, float, float] |
get_left_hand_rotation() |
The rotation of the left hand in the order (x, y, z, w). | Tuple[float, float, float, float] |
get_left_hand_forward() |
The forward of the left hand. | Tuple[float, float, float] |
get_right_hand_position() |
The position of the right hand in the order of (x, y, z). | Tuple[float, float, float] |
get_right_hand_rotation() |
The rotation of the right hand in the order (x, y, z, w). | Tuple[float, float, float, float] |
get_right_hand_forward() |
The forward of the right hand. | Tuple[float, float, float] |
get_head_position() |
The position of the head in the order of (x, y, z). | Tuple[float, float, float] |
get_head_rotation() |
The rotation of the head in the order (x, y, z, w). | Tuple[float, float, float, float] |
get_head_forward() |
The forward of the head. | Tuple[float, float, float] |
get_held_left() |
The IDs of the objects held by the left hand. | np.ndarray |
get_held_right() |
The IDs of the objects held by the right hand. | np.ndarray |
a = AvatarIds(byte_array)
Identifier: avid
The IDs of each avatar in the scene.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of objects. | int |
get_id(index) |
The id. | str |
get_type(index) |
The type. | str |
f = FastAvatars(byte_array)
Identifier: fava
Fast, fixed-length avatar transform data. Use this in conjunction with FastAvatarIds: the order of the IDs matches the order of this data.
| Function | Description | Return type |
|---|
f = FastImageSensors(byte_array)
Identifier: fims
Fast, fixed-length avatar image sensor transform data. Use this in conjunction with FastAvatarIds: the order of the IDs matches the order of this data.
| Function | Description | Return type |
|---|---|---|
get_num() |
The number of rotations. | int |
get_rotation(index) |
The rotation. | np.ndarray |
f = FastTransforms(byte_array)
Identifier: None
None
| Function | Description | Return type |
|---|
o = ObjectIds(byte_array)
Identifier: obid
The IDs of all Rigidbody objects (models and composite sub-objects) in the scene.
| Function | Description | Return type |
|---|---|---|
get_ids() |
The ID of each object and composite sub-object in the scene. This array is sorted. | List[int] |