Gazebo racecar suddenly stopped

[jinbo-huang]

OS Version: Ubuntu 18.04
Gazebo Version: Melodic
Runtime: Python 3.0

Hi, I'm building a gym env wrapper, but I found sometimes the racecar in Gazebo suddenly stopped even I kept publishing new actions. However, It continued running after I clicked the racecar through Gazebo GUI or I paused and played Gazebo. But I want to do some RL training, it's impossible that I keep monitoring Gazebo while it's training.

Here is my launch file (pretty much the same to q_learning.launch):

<?xml version="1.0" encoding="UTF-8"?>
<launch>
    <arg name="world" default="racetrack_decorated"/>
    <arg name="use_gpu" default="true"/>
    <arg name="gui" default="true"/>
    <arg name="realtime_simulation" default="true"/>
    <!-- <arg name="plot_window" default="200"/> -->

    <param name="world_name" value="$(arg world)"/>

    <include file="$(find racer_world)/launch/racer_gazebo.launch">
        <arg name="world" value="$(arg world)"/>
        <arg name="gui" value="$(arg gui)"/>
        <arg name="debug" value="false"/>
        <arg name="verbose" value="false"/>
        <arg name="use_gpu" value="$(arg use_gpu)"/>
        <arg name="extra_gazebo_args" value="--profile fast_simulation" unless="$(arg realtime_simulation)"/>
        <!-- Warning: Gazebo calculates inaccurate scans if the resolution is too small. The smaller the resolution, the
        greater the error becomes. Don't make the resolution too small, even if you only need a few samples. -->
        <arg name="laser_sample_count" value="128"/>
    </include>

    <!-- <node name="rviz" pkg="rviz" type="rviz" args="-d $(find racer_world)/launch/rviz_config.rviz"/> -->

    <!-- <node name="lap_timer" pkg="simulation_tools" type="lap_timer.py" output="screen"/> -->

    <include file="$(find vesc_sim)/launch/vesc_sim.launch"/>
    <include file="$(find car_control)/launch/car_control.launch">
        <arg name="mode_override" value="2"/>
    </include>

    <node pkg="simulation_tools" type="crash_detector" name="crash_detector" output="screen">
    </node>
</launch>

gym reset function:

    def reset(self):
        for joint in EFFORT_JOINTS:
            self.clear_forces_client(joint)
        self.reset_racecar()
        self.episode_length = 0
        self.is_terminal_step = False
        _ = self.image_queue.get(block=True, timeout=None)
        self.set_next_state()
        return self.next_state

gym step function:

    def step(self, action_index):
        angle, velocity = self.actions[action_index]
        message = drive_param()
        message.angle = angle
        message.velocity = velocity
        self.drive_parameters_publisher.publish(message)
        self.set_next_state()
        # cv2.imshow('image', self.next_state)
        # cv2.waitKey(1)
        r = 0       # reward
        self.episode_length += 1
        return self.next_state, r, self.is_terminal_step, {}

how I execute:

if __name__ == "__main__":
    env = AutoRacingEnv()
    while True:
        env.reset()
        done = False
        while not done:
            s_next, r, done, _ = env.step(np.random.randint(3))

I also tried a variety of settings (sleep after step() or reset()), but none of them worked.