Cannot reset simulator for maps other than Canyons

[subhash]

Repro code:

import sim
import time
import config as c

env = sim.start(map=c.CANYONS_MAP_NAME)
while True:
    time.sleep(5)
    print("Resetting")
    res = env.reset()
    print("Reset ", res)

This resets as expected. But when the map is changed to any of the others, the simulator fails to reset.

@crizCraig This makes it hard to end an episode for RL training

[crizCraig]

Hi @subhash - thanks, we'll work on resolving this asap. In the name of faster iteration, I should also mention that I have made quite a bit of progress on reinforcement learning performance using a 2D environment using spinningup's PPO. Some improvements to the PPO baseline in the 3D world are

1. Incorporation of g-force into reward function (in addition to speed). This is a difficult tradeoff to achieve in practice, but I've gotten a pretty good balance now
2. 100x faster than realtime training
3. All physics in Python and can be synced / decoupled from rendering and be made deterministic
4. Much smoother steering (PPO excelled here vs SAC spinningup or SAC pytorch )

If we can train and transfer to 3D deepdrive, this would be a promising test of real transfer.

[subhash]

@crizCraig I see what you mean. I was initially intending to try SAC with VAE feature extraction. But if you see promising results in the 2D environment, that may be worth pursuing more.

Your idea is very compelling. After all, regular MPC control works well in 2D, so even if that could be replaced with a RL component, that would be a good start. I was able to run the simulator but couldn't figure out how to bootstrap actions from it. Also, I see that you have a script to run the policy but is there a way to see the trained results?

I would like to participate in the 2D effort if you could help me jumpstart.

[crizCraig]

Hi @subhash - the 2D repo is in quite a bit of flux at the moment, so I apologize upfront for the unconventional adhoc setup guid, but as a first step running the examples in my spinningup fork will verify that the RL half is working

https://github.com/crizCraig/spinningup

Then checkout the run configurations here:
https://github.com/deepdrive/deepdrive-2d/blob/selfplay/.idea/runConfigurations/play_dd2d_all_friction_one_waypoint_static_obstacle_DISABLE_GAME_OVER_1.xml

From which you can use the SCRIPT_NAME and PARAMETERS to see what commands I'm running to do things. Or if you use PyCharm, you can use these directly.

[subhash]

Hi @crizCraig

I ran into an issue with the latest spinningup repo. I backed up to an earlier commit and I am now able to run the one waypoint test case you mention above.

I also see some issues logged in the GitHub repo. Maybe I can lend a hand in attacking them first? Hopefully, that'll get me started up.

Could we sync up on a chat sometime so that I can come up to speed on the concepts? Please let me know.