Move dev_wrappers and functional to experimental (openai#159)

.pre-commit-config.yaml

@@ -15,7 +15,7 @@ repos:
     hooks:
       - id: flake8
         args:
-          - '--per-file-ignores=*/__init__.py:F401 gymnasium/envs/registration.py:E704 docs/tutorials/*.py:E402'
+          - '--per-file-ignores=*/__init__.py:F401 gymnasium/envs/registration.py:E704 docs/tutorials/*.py:E402 gymnasium/experimental/wrappers/__init__.py:E402'
           - --ignore=E203,W503,E741
           - --max-complexity=30
           - --max-line-length=456

docs/api/experimental.md

@@ -0,0 +1,214 @@
+---
+title: Experimental
+---
+
+# Experimental
+
+```{toctree}
+:hidden:
+experimental/functional
+experimental/wrappers
+experimental/vector
+experimental/vector_wrappers
+```
+
+## Functional Environments
+
+The gymnasium ``Env`` provides high flexibility for the implementation of individual environments however this can complicate parallelism of environments. Therefore, we propose the :class:`gymnasium.experimental.FuncEnv` where each part of environment has its own function related to it. 
+
+## Wrappers
+
+Gymnasium already contains a large collection of wrappers, but we believe that the wrappers can be improved to
+
+ * Support arbitrarily complex observation / action spaces. As RL has advanced, action and observation spaces are becoming more complex and the current wrappers were not implemented with these spaces in mind.
+ * Support for numpy, jax and pytorch data. With hardware accelerated environments, i.e. Brax, written in Jax and similar pytorch based programs, numpy is not the only game in town anymore. Therefore, these upgrades will use Jumpy for calling numpy, jax and torch depending on the data.
+ * More wrappers. Projects like Supersuit aimed to bring more wrappers for RL however wrappers can be moved into Gymnasium.
+ * Versioning. Like environments, the implementation details of wrapper can cause changes agent performance. Therefore, we propose adding version numbers with all wrappers.
+
+ * In v28, we aim to rewrite the VectorEnv to not inherit from Env, as a result new vectorised versions of the wrappers will be provided.
+
+### Lambda Observation Wrappers
+```{eval-rst}
+.. py:currentmodule:: gymnasium
+
+.. list-table:: 
+    :header-rows: 1
+    
+    * - Old name
+      - New name
+      - Vector version
+      - Tree structure
+    * - :class:`wrappers.TransformObservation`
+      - :class:`experimental.wrappers.LambdaObservationV0`
+      - VectorLambdaObservation
+      - No
+    * - :class:`wrappers.FilterObservation`
+      - FilterObservation
+      - VectorFilterObservation (*)
+      - Yes
+    * - :class:`wrappers.FlattenObservation`
+      - FlattenObservation
+      - VectorFlattenObservation (*)
+      - No
+    * - :class:`wrappers.GrayScaleObservation`
+      - GrayscaleObservation
+      - VectorGrayscaleObservation (*)
+      - Yes
+    * - :class:`wrappers.PixelObservationWrapper`
+      - PixelObservation 
+      - VectorPixelObservation (*)
+      - No
+    * - :class:`wrappers.ResizeObservation`
+      - ResizeObservation
+      - VectorResizeObservation (*)
+      - Yes
+    * - Not Implemented
+      - ReshapeObservation
+      - VectorReshapeObservation (*)
+      - Yes
+    * - Not Implemented
+      - RescaleObservation
+      - VectorRescaleObservation (*)
+      - Yes
+    * - Not Implemented
+      - DtypeObservation
+      - VectorDtypeObservation (*)
+      - Yes
+    * - :class:`NormalizeObservation`
+      - NormalizeObservation 
+      - VectorNormalizeObservation
+      - No
+    * - :class:`TimeAwareObservation`
+      - TimeAwareObservation
+      - VectorTimeAwareObservation
+      - No
+    * - :class:`FrameStack`
+      - FrameStackObservation
+      - VectorFrameStackObservation
+      - No
+    * - Not Implemented
+      - DelayObservation
+      - VectorDelayObservation
+      - No
+    * - :class:`AtariPreprocessing`
+      - AtariPreprocessing
+      - Not Implemented
+      - No
+```
+
+### Lambda Action Wrappers
+```{eval-rst}
+.. py:currentmodule:: gymnasium
+
+.. list-table:: 
+    :header-rows: 1
+    
+    * - Old name
+      - New name
+      - Vector version
+      - Tree structure
+    * - Not Implemented
+      - :class:`experimental.wrappers.LambdaActionV0`
+      - VectorLambdaAction
+      - No
+    * - :class:`wrappers.ClipAction`
+      - ClipAction
+      - VectorClipAction (*)
+      - Yes
+    * - :class:`wrappers.RescaleAction`
+      - RescaleAction
+      - VectorRescaleAction (*)
+      - Yes
+    * - Not Implemented
+      - NanAction
+      - VectorNanAction (*)
+      - Yes
+    * - Not Implemented
+      - StickyAction
+      - VectorStickyAction
+      - No
+```
+
+### Lambda Reward Wrappers
+```{eval-rst}
+.. py:currentmodule:: gymnasium
+
+.. list-table:: 
+    :header-rows: 1
+    
+    * - Old name
+      - New name
+      - Vector version
+    * - :class:`wrappers.TransformReward`
+      - :class:`experimental.wrappers.LambdaRewardV0`
+      - VectorLambdaReward
+    * - Not Implemented
+      - :class:`experimental.wrappers.ClipRewardV0`
+      - VectorClipReward (*)
+    * - Not Implemented
+      - RescaleReward
+      - VectorRescaleReward (*)
+    * - :class:`wrappers.NormalizeReward`
+      - NormalizeReward
+      - VectorNormalizeReward
+```
+
+### Common Wrappers
+```{eval-rst}
+.. py:currentmodule:: gymnasium
+
+.. list-table:: 
+    :header-rows: 1
+    
+    * - Old name
+      - New name
+      - Vector version
+    * - :class:`wrappers.AutoResetWrapper`
+      - AutoReset
+      - VectorAutoReset
+    * - :class:`wrappers.PassiveEnvChecker`
+      - PassiveEnvChecker
+      - VectorPassiveEnvChecker
+    * - :class:`wrappers.OrderEnforcing`
+      - OrderEnforcing
+      - VectorOrderEnforcing (*)  
+    * - :class:`wrappers.EnvCompatibility`
+      - Moved to `shimmy <https://github.com/Farama-Foundation/Shimmy/blob/main/shimmy/openai_gym_compatibility.py>`_
+      - Not Implemented
+    * - :class:`RecordEpisodeStatistics`
+      - RecordEpisodeStatistics
+      - VectorRecordEpisodeStatistics
+    * - :class:`RenderCollection`
+      - RenderCollection
+      - VectorRenderCollection
+    * - :class:`HumanRendering`
+      - HumanRendering
+      - Not Implemented
+    * - Not Implemented
+      - JaxToNumpy
+      - VectorJaxToNumpy
+    * - Not Implemented
+      - JaxToTorch
+      - VectorJaxToTorch
+```
+
+### Vector Only Wrappers
+```{eval-rst}
+.. py:currentmodule:: gymnasium
+
+.. list-table:: 
+    :header-rows: 1
+    
+    * - Old name
+      - New name
+    * - :class:`wrappers.VectorListInfo`
+      - VectorListInfo
+```
+
+## Vector Environment
+
+These changes will be made in v0.28
+
+## Wrappers for Vector Environments
+
+These changes will be made in v0.28

docs/api/experimental/functional.md

@@ -0,0 +1,36 @@
+---
+title: Functional
+---
+
+# Functional Environment
+
+## gymnasium.experimental.FuncEnv
+
+```{eval-rst}
+.. autoclass:: gymnasium.experimental.FuncEnv
+
+.. autofunction:: gymnasium.experimental.FuncEnv.initial
+.. autofunction:: gymnasium.experimental.FuncEnv.transition
+
+.. autofunction:: gymnasium.experimental.FuncEnv.observation
+.. autofunction:: gymnasium.experimental.FuncEnv.initial
+
+.. autofunction:: gymnasium.experimental.FuncEnv.observation
+.. autofunction:: gymnasium.experimental.FuncEnv.reward
+.. autofunction:: gymnasium.experimental.FuncEnv.terminal
+
+.. autofunction:: gymnasium.experimental.FuncEnv.state_info
+.. autofunction:: gymnasium.experimental.FuncEnv.step_info
+
+.. autofunction:: gymnasium.experimental.FuncEnv.transform
+
+.. autofunction:: gymnasium.experimental.FuncEnv.render_image
+.. autofunction:: gymnasium.experimental.FuncEnv.render_init
+.. autofunction:: gymnasium.experimental.FuncEnv.render_close
+```
+
+## gymnasium.experimental.func2env.FunctionalJaxCompatibilityEnv
+
+```{eval-rst}
+... autoclass:: gymasnium.experimental.func2env.FunctionalJaxCompatibilityEnv
+```

docs/api/experimental/vector.md

@@ -0,0 +1,15 @@
+---
+title: Vector
+---
+
+# Vectorizing Environment
+
+## gymnasium.experimental.VectorEnv
+
+## gymnasium.experimental.vector.AsyncVectorEnv
+
+## gymnasium.experimental.vector.SyncVectorEnv
+
+## Custom Vector environments
+
+## EnvPool

docs/api/experimental/vector_wrappers.md

@@ -0,0 +1,15 @@
+---
+title: Vector Wrappers
+---
+
+# Vector Environment Wrappers
+
+## Vector Lambda Observation Wrappers
+
+## Vector Lambda Action Wrappers
+
+## Vector Lambda Reward Wrappers
+
+## Vector Common Wrappers
+
+## Vector Only Wrappers

docs/api/experimental/wrappers.md

@@ -0,0 +1,26 @@
+# Wrappers
+
+## Lambda Observation Wrappers
+
+```{eval-rst}
+.. autoclass:: gymnasium.experimental.wrappers.LambdaObservationV0
+```
+
+## Lambda Action Wrappers
+
+```{eval-rst}
+.. autoclass:: gymnasium.experimental.wrappers.LambdaActionV0
+```
+
+## Lambda Reward Wrappers
+
+```{eval-rst}
+.. autoclass:: gymnasium.experimental.wrappers.LambdaRewardV0
+.. autoclass:: gymnasium.experimental.wrappers.ClipRewardV0
+```
+
+## Common Wrappers
+
+```{eval-rst}
+
+```

docs/index.md

@@ -48,6 +48,7 @@ api/spaces
 api/wrappers
 api/vector
 api/utils
+api/experimental
 ```
 
 ```{toctree}

gymnasium/__init__.py

@@ -10,10 +10,8 @@
 )
 from gymnasium.spaces.space import Space
 from gymnasium.envs.registration import make, spec, register, registry, pprint_registry
-from gymnasium import envs, spaces, utils, vector, wrappers, error, logger
+from gymnasium import envs, spaces, utils, vector, wrappers, error, logger, experimental
 
-import os
-import sys
 
 __all__ = [
     # core classes
@@ -37,6 +35,7 @@
     "wrappers",
     "error",
     "logger",
+    "experimental",
 ]
 __version__ = "0.26.3"
 
@@ -45,6 +44,9 @@
 #   pygame
 # DSP is far more benign (and should probably be the default in SDL anyways)
 
+import os
+import sys
+
 if sys.platform.startswith("linux"):
     os.environ["SDL_AUDIODRIVER"] = "dsp"
 

gymnasium/envs/phys2d/__init__.py

@@ -1,2 +1,2 @@
-from gymnasium.envs.phys2d.cartpole import CartPoleF
-from gymnasium.envs.phys2d.pendulum import PendulumF
+from gymnasium.envs.phys2d.cartpole import CartPoleFunctional
+from gymnasium.envs.phys2d.pendulum import PendulumFunctional