Simulator

Action simulator combines simulator interface with action mappers.

Simulator takes a list of tasks at initialization, builds agent-readable representation for them, and provides interface to run simulation in a specified action tier.

phyre.ACTION_TIERS = ('ball', 'two_balls')

List of action tiers in phyre.

phyre.initialize_simulator(task_ids: Sequence[str], action_tier: str) → phyre.action_simulator.ActionSimulator

Initialize ActionSimulator for given tasks and tier.

class phyre.ActionSimulator(tasks: Union[Sequence[phyre.interface.task.ttypes.Task], Mapping[str, phyre.interface.task.ttypes.Task]], action_mapper, no_goals: bool = True)

Interface to query the simulator with actions within the tier.

property action_space_dim

Return dimension of the actions space.

build_discrete_action_space(max_actions, seed=1) → Sequence[Union[Sequence[float], numpy.ndarray]]

Build a mapping from the given number of action to the action space.

Parameters

  • max_actions – int, maximum number of discrete actions.

  • seed – int, random seed to generate the subspace.

Returns

tuple of actions of length max_actions.

Return type

discrete_actions

property goals

Represents goals for each task.

uint8 array with shape (task, 3). Each goal is encoded with three numbers: (obj_type1, obj_type2, rel). All three are less than MAX_GOAL. To be more precise, obj_types are less than MAX_OBJECT_TYPE and rel is less than MAX_RELATION.

property initial_featurized_objects

Inital scene objects featurized for each task before agent input.

List (length tasks) of FeaturizedObjects containing float arrays of size (number scene objects, OBJECT_FEATURE_SIZE).

property initial_scenes

Represents intial scene for each task before agent input.

uint8 array with shape (task, height, width).

sample(valid_only=True, rng=None) → Union[Sequence[float], numpy.ndarray]

Sample a random (valid) action from the action space.

simulate_action(task_index: int, action: Union[Sequence[float], numpy.ndarray], *, need_images: bool = True, need_featurized_objects: bool = False, stride: int = 60, stable: bool = False) → phyre.simulation.Simulation

Runs simluation for the action.

Parameters

  • task_index – index of the task.

  • action – numpy array or list of self.action_space_dim floats in [0, 1].

  • need_images – whether simulation images are needed.

  • need_featurized_objects – whether simulation featurized_objects are needed.

  • stride – int, defines the striding for the simulation images array. Higher striding will result in less images and less compute. Note, that this parameter doesn’t affect simulation FPS. Ignored if need_images is False.

  • stable – if True, will simulate a few actions in the neigborhood of the actions and return STABLY_SOLVED status iff all neigbour actions are either SOLVED or INVALID. Otherwise UNSTABLY_SOLVED is returned. SOLVED is never returned if stable is set.

Returns

  • phyre.simulation.Simulation object containing the result of

    the simulation.

  • SimulationStatus, images, and featurized_objects are easily

    accesible with simulation.simulation_status, simulation.images, and simulation.featurized_objects.

simulate_single(task_index: int, action: Union[Sequence[float], numpy.ndarray], need_images: bool = True, stride: int = 60, stable: bool = False) → Tuple[phyre.action_simulator.SimulationStatus, Optional[numpy.ndarray]]

Deprecated in version 0.2.0 in favor of simulate_action. Runs simluation for the action.

Parameters

  • task_index – index of the task.

  • action – numpy array or list of self.action_space_dim floats in [0, 1].

  • need_images – whether simulation images are needed.

  • stride – int, defines the striding for the simulation images array. Higher striding will result in less images and less compute. Note, that this parameter doesn’t affect simulation FPS. Ignored if need_images is False.

  • stable – if True, will simulate a few actions in the neigborhood of the actions and return STABLY_SOLVED status iff all neigbour actions are either SOLVED or INVALID. Otherwise UNSTABLY_SOLVED is returned. SOLVED is never returned if stable is set.

Returns

  • If need_images is True, returns a pair (status, images).

    • If status is INVALID_INPUT images is None.

    • Otherwise images is an array contains intermediate observations.

  • If need_images is False: returns (status, None).

property task_ids

Tuple of task ids in simulator.

class phyre.SimulationStatus

Status that ActionSimulator returns given a task and an action.

is_invalid() → bool

whether the action is invalid for the task.

is_not_solved() → bool

Whether the action is valid, but doesn’t solve the task.

is_solved() → bool

Whether the action solved the task.

is_stably_solved() → bool

Whether the action is stable solution for the task.

This only applies if the simulation was called with stable flag.

class phyre.Simulation(*, status=None, images: Optional[numpy.ndarray] = None, featurized_objects: Optional[numpy.ndarray] = None)

Interface for the result of an ActionSimulator simulation.

Featurized objects and images are returned in the same order, such that simulation.images[i] is the pixel representation of simulation.featurized_objects[i].

If self.status is INVALID_INPUT self.images, and self.featurized_objects are both None.

Variables

  • images – Initial pixel representation of intermeidate obervations.

  • featurized_objects – Object representation of intermediate observations. FeaturizedObjects containing information about object features and state.

  • status – SimulationStatus of simulation.

class phyre.FeaturizedObjects(featurized_objects: numpy.ndarray)

Featurization of objects in a series of scene, such as from a simulation. Returned by either ActionSimulator.intial_featurized_objects, or ActionSimulator.simulate_action if need_featurized_objects=True. Note, for object order, user input objects (if any) are always last. :ivar features: Featurs of objects of observations for a set (or one) timesteps.

TxNx14 np.array where T is the number of timestes, N is the number of objects in the scene and 14 is the feature vector size. The features are by index:

  • 0: x in pixels of center of mass divided by SCENE_WIDTH

  • 1: y in pixels of center of mass divided by SCENE_HEIGHT

  • 2: angle of the object between 0 and 2pi divided by 2pi

  • 3: diameter in pixels of object divided by SCENE_WIDTH

  • 4-8: One hot encoding of the object shape, according to order:

    ball, bar, jar, standing sticks

  • 8-14: One hot encoding of object color, according to order:

    red, green, blue, purple, gray, black

Variables

  • shapes – List(str) of length number of objects of the shape types of the objects in order. Values are members of scene_if.ShapeType

  • shapes_one_hot – np.array of size (T, N, 4) corresponding to one hot encoding of shapes. Features 4-8 shape types of the objects in order. Values are members of scene_if.ShapeType

  • colors – List(str) of length number of objects of the colors of the objects in order. Values are members of shared_if.Colors

  • shapes_one_hot – np.array of size (T, N, 6) corresponding to one hot encoding of colors. Features 8-14

  • diameters – np.ndarray of dtype=float of shape(num objects, ) containing the object diameter in pixels divided by SCENE_WIDTH in order

  • states –

    np.array of size (T, N, 3) where T is the number of timesteps, N is the number of objects and the remaining 3 features are:

    • 0: x in pixels of center of mass divided by SCENE_WIDTH

    • 1: y in pixels of center of mass divided by SCENE_HEIGHT

    • 2: angle of the object in [0, 2pi] divided by 2pi

  • num_user_inputs – (int) Number of user input objects in the simulation

  • num_objects – (int) Number of objects in the simulation_states

  • num_scene_obejcts – (int) Number of scene objects in the simulation.

property num_objects

Number of objects in the scene.

property num_scene_objects

Number of scene objects in the scene.