Mujoco Unity Plugin (Mujoco Unity 插件)

Doris232 · 发表于 2022-6-30 20:27

mujoco python related module：

dm_control：TBD
mujoco: mujoco open source library, including native c++ engine and python interface
mujoco_py: This is just a python wrapper on mujoco, abandoned by deepmind, so you should use mujoco.
gym： This has some mujoco playground environments, and it uses mujoco or mujoco-py to communicate to mujoco backend.

reset(): it first uses mujoco mj_resetData and then use reset_model.

mujoco unity plugin：
This plugin directly communicates with mujoco c++ backend. Using c# to read mjcf xml file and load all resources, it won&#39;t involve any unity physical system and just sync all data from mujoco to unity after each timestep.
Since it is a unity plugin, the whole running process is under unity execution order. It is possible to use another unity plugin mlagent to interact with our python training code. User needs to write there own code to extract data from mujoco and also send actions to mujoco.
The pipline is like (Your Python training code) → (Your C# script based on mlagent) → (mujoco unity plugin) → (mujoco backend)

unity execution order link： https://docs.unity3d.com/Manual/ExecutionOrder.html
mujoco mjcf xml explanation：https://zhuanlan.zhihu.com/p/380716224

site: Site可以认为是零质量的geom。其定义了刚体坐标系中感兴趣的位置点，通常用于定义一些对象（如传感器、肌腱路径点、滑块终点）的位置。不参与碰撞检测。
inertial: This element specifies the mass and inertial properties of the body

Mujoco Unity Plugin Debug：

MjImporterEditorWindow： Main entrance, get XML path and give it to ImportFile
MjImporterWithAssets：

ImportFile： Use mujoco c++ interface to read XML and it will re-save model as xml file to a tmp folder. PS: if tag&#39;s parameters are all default values, this tag will be ignored during re-saving. Then go to ImportString.
ImportString： It just creates a file path and links tmp folder with project folder, and also selects mujoco compiler path. Next to ImportXml
ImportXml： Select angle type, create a gameObject as root, then go to ParseRoot. XML tree is changed to <mujoco> tag as the parent node.
ParseAssets:
- Import Assets，only <mesh> and <material> tag
  - ParseMesh
  - ParseMaterial
    - PS: It will change &#34;/&#34; in name to &#34;_&#34;
    - PS: It will save file as .mat.
base.ParseRoot：
- apply <option> and <size> tag, <option> is related to mujoco psycial system and <size> is related to other tags data. It will create a gameObject called &#34;global setting&#34; under root node.
- Parse <worldbody> tag, then go to ParseBodyChildren
  - ParseBodyChildren：
    - ApplyModifiersToElement：Copy parameters from <default> tag and create one singe default dictionary contain all values. And then copy other default parameters setting for class to the dictionary. Finally, apply all values to actual child tag.
    - ParseBodyChild： Create real gameObjects, attach related scripts and values for child tags
      - ParseMjcf： create mesh data for gameobjects
- contact tag
  - Related to collision detection but now only support setting no collision.
- tendon tag
- equality tag
- actuator/sensor tag
  - create actuator/sensor gameobjects

MjScene (when running project):

If there has any MjComponent, it has a start function that will create a MjScene gameObject and attach MjScene script to it when unity starts running.
CreateScene
- It will firstly transfer XML tree structure to list and use GenerateSceneMjcf to create how unity scene.
StepScene
- just use mj c++ interface to step environment and sync data to unity
RecreateScene
- There is a variable named SceneRecreationAtLateUpdateRequested in LateUpdate. If there is any new added MjComponent, LateUpdate will rebuild the whole scene for new component.

		自动登录	找回密码
密码			立即注册