Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically saving good-quality simulation videos using a Python (.py) script rather than mediapy in a Jupyter notebook (.ipynb) #1650

Closed
kurtenkera opened this issue May 11, 2024 · 4 comments
Closed

Automatically saving good-quality simulation videos using a Python (.py) script rather than mediapy in a Jupyter notebook (.ipynb) #1650

kurtenkera opened this issue May 11, 2024 · 4 comments
Labels
question Request for help or information

Comments

@kurtenkera
Copy link

kurtenkera commented May 11, 2024 •
edited

Hi,

I am accessing MuJoCo via the Python bindings. I currently run my simulations in a Jupyter notebook. Using the code below, I can simulate, render and save the rendered videos. However, to save the videos, I have to manually click "Download" on the resulting video (whereas I want to automate this saving process in a .py file?):

# Simulate and display video.
frames = []
while data.time < duration:
  mujoco.mj_step(model, data)
  if len(frames) < data.time * framerate:
    renderer.update_scene(data, "closeup")
    pixels = renderer.render()
    frames.append(pixels)

media.show_video(frames, fps=framerate)

I understand mediapy's show_video function only works in a Jupyter notebook, but I eventually want to build my code in a Python script (.py).

How can I run and save good-quality simulations in an automated way using a .py script rather than a Jupyter notebook? Ideally, I want to also have the option to run my simulations headless, but save the resulting good-quality videos to a file directory also?

Ideally, I would like the simulation videos to also have the same quality as seen below in the MuJoCo GUI:

good_quality_mjsim.mp4

Note that the videos produced using mediapy in the Jupyter notebook environment have noticeably worse quality:

bad_quality_mediapy_mjsim.mp4

Summarising my questions:

(1) How can I run and save good-quality simulations in an automated way using a .py script rather than a Jupyter notebook?
(2) How can I optionally run the simulations in the OpenGL, good-quality rendering GUI or headless from a .py script?
(3) How to ensure that the simulations run from a .py file have high-quality resolution, and not bad quality as produced using `mediapy'?
@kurtenkera kurtenkera added the question Request for help or information label May 11, 2024
@kurtenkera kurtenkera changed the title Simulating and Rendering using a Python (.py) script rather than a Jupyter notebook (.ipynb) Saving good-quality simulation videos using a Python (.py) script rather than mediapy in a Jupyter notebook (.ipynb) May 12, 2024
@kurtenkera kurtenkera changed the title Saving good-quality simulation videos using a Python (.py) script rather than mediapy in a Jupyter notebook (.ipynb) Automatically saving good-quality simulation videos using a Python (.py) script rather than mediapy in a Jupyter notebook (.ipynb) May 12, 2024
@kevinzakka
Copy link
Collaborator

media.write_video compresses the video by default. You can specify additional parameters to increase the quality, as documented here.

@kurtenkera
Copy link
Author

Thanks @kevinzakka.

Using media.write_video, I can run the .py script below (which is a copy of the Contacts example in tutorial.ipynb):

import mujoco
import mediapy
from mediapy import VideoWriter
import numpy as np

free_body_MJCF = """
<mujoco>
  <asset>
    <texture name="grid" type="2d" builtin="checker" rgb1=".1 .2 .3" 
    rgb2=".2 .3 .4" width="300" height="300" mark="edge" markrgb=".2 .3 .4"/>
    <material name="grid" texture="grid" texrepeat="2 2" texuniform="true"
    reflectance=".2"/>
  </asset>

  <worldbody>
    <light pos="0 0 1" mode="trackcom"/>
    <geom name="ground" type="plane" pos="0 0 -.5" size="2 2 .1" material="grid" solimp=".99 .99 .01" solref=".001 1"/>
    <body name="box_and_sphere" pos="0 0 0">
      <freejoint/>
      <geom name="red_box" type="box" size=".1 .1 .1" rgba="1 0 0 1" solimp=".99 .99 .01"  solref=".001 1"/>
      <geom name="green_sphere" size=".06" pos=".1 .1 .1" rgba="0 1 0 1"/>
      <camera name="fixed" pos="0 -.6 .3" xyaxes="1 0 0 0 1 2"/>
      <camera name="track" pos="0 -.6 .3" xyaxes="1 0 0 0 1 2" mode="track"/>
    </body>
  </worldbody>
</mujoco>
"""
model = mujoco.MjModel.from_xml_string(free_body_MJCF)
data = mujoco.MjData(model)

n_frames = 200
height = 240
width = 320
# for video writing
shape = (height,width)
frames = []
renderer = mujoco.Renderer(model, height, width)

# visualize contact frames and forces, make body transparent
options = mujoco.MjvOption()
mujoco.mjv_defaultOption(options)
options.flags[mujoco.mjtVisFlag.mjVIS_CONTACTPOINT] = True
options.flags[mujoco.mjtVisFlag.mjVIS_CONTACTFORCE] = True
options.flags[mujoco.mjtVisFlag.mjVIS_TRANSPARENT] = True

# tweak scales of contact visualization elements
model.vis.scale.contactwidth = 0.1
model.vis.scale.contactheight = 0.03
model.vis.scale.forcewidth = 0.05
model.vis.map.force = 0.3

# random initial rotational velocity:
mujoco.mj_resetData(model, data)
np.random.seed(1)
data.qvel[3:6] = 5*np.random.randn(3)

# simulate and render
for i in range(n_frames):
  while data.time < i/120.0: #1/4x real time
    mujoco.mj_step(model, data)
  renderer.update_scene(data, "track", options)
  frame = renderer.render()
  frames.append(frame)

with VideoWriter('test.mp4', shape, fps=30) as writer:
    for frame in frames:
        writer.add_image(frame)

While this code successfully saves a video test.mp4 of the simulation, I also get the following error/warning messages in my terminal:

(base) enk001@drift-ph:~/Documents/PHD/PHD-Code/Projects/Mujoco-ArbMeshContacts$ /home/enk001/miniconda3/envs/mujoco-tut/bin/python /home/enk001/Documents/PHD/PHD-Code/Projects/Mujoco-ArbMeshContacts/test.py
Exception ignored in: <function Renderer.__del__ at 0x7a5a3aa77240>
Traceback (most recent call last):
  File "/home/enk001/miniconda3/envs/mujoco-tut/lib/python3.12/site-packages/mujoco/renderer.py", line 330, in __del__
  File "/home/enk001/miniconda3/envs/mujoco-tut/lib/python3.12/site-packages/mujoco/renderer.py", line 316, in close
  File "/home/enk001/miniconda3/envs/mujoco-tut/lib/python3.12/site-packages/mujoco/glfw/__init__.py", line 35, in free
TypeError: 'NoneType' object is not callable
Exception ignored in: <function GLContext.__del__ at 0x7a5b461ad580>
Traceback (most recent call last):
  File "/home/enk001/miniconda3/envs/mujoco-tut/lib/python3.12/site-packages/mujoco/glfw/__init__.py", line 41, in __del__
  File "/home/enk001/miniconda3/envs/mujoco-tut/lib/python3.12/site-packages/mujoco/glfw/__init__.py", line 35, in free
TypeError: 'NoneType' object is not callable

I believe these warning messages occur due to this line of code:

renderer = mujoco.Renderer(model, height, width)

Since the code still runs and produces the saved simulation video, should I just ignore these messages? Or do I need to modify my code so that I don't get these error messages?

@yuvaltassa
Copy link
Collaborator

Does this go away if you add renderer.close() at the end of your script?

@kevinzakka, @kbayes, is there a bug in the implementation of close, __del__ or __exit__, similar things seem to keep happening, but it's not clear to me why...

@kurtenkera
Copy link
Author

Yes @yuvaltassa thankyou very much! It goes away if I add renderer.close() or renderer.__del__()

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Request for help or information
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yuvaltassa @kevinzakka @kurtenkera