ME5413 - Homework 1: Perception

Author: Cao Chenyu

(I have removed all the data files on the remote branch, but only reserved the implementations and the report.)

Introduction

This is the repository for the homework 1 of the course ME5413: Perception. The homework is divided into three tasks:

• Task 1: Single Object Detection
• Task 2: Multiple Object Detection
• Bonus Task: Publish and Subscribe Object Detection Results

Project Structure

Homework1_Perception
├─ README.md
├─ Task 1
│  ├─ task1.ipynb
│  ├─ data
│  └─ utils
│     ├─ __init__.py
│     ├─ evaluate.py
│     ├─ load.py
│     └─ save.py
├─ Task 2
│  ├─ task2.ipynb
│  ├─ data
│  ├─ utils
│  │  ├─ __init__.py
│  │  └─ plot.py
├─ Bonus Task
│  ├─ bonus_task_ws -> ROS workspsace for bonus task
│     ├─ run.sh -> shell script for running the ROS nodes
│     ├─ src
│     │  └─ track_node -> node for publishing KF tracking results and subscribing the object detection results
│     │     ├─ data
│     │     ├─ launch
│     │     │  └─ track.launch
│     │     ├─ scripts
│     │     │  ├─ track_pub_p.py
│     │     │  └─ track_sub_p.py
│     │     └─ track.bag -> bag file
├─ requirements.txt
└─ ME5413_HW1_Report.pdf

Usage

Pre-requisites

You need to create a virtual environment and install the required packages. You can do this by running the following commands:

python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt

You have to install ROS 1 Noetic on Ubuntu 20 in order to run the Bonus Task. You can install ROS 1 Noetic by following the instructions in the official website.

Task 1: Single Object Detection

Select the prepared virtual environment, then click Run All in the task1.ipynb to run the code. The code will load the dataset, perform the template matching and Kalman Filtering, and visualize all the results.

Task 2: Multiple Object Detection

Select the prepared virtual environment, then click Run All in the task2.ipynb to run the code. The code will load the dataset, perform the object detection using different models, calculate the ADE and FDE, and visualize the results.

Bonus Task

cd into the bonus_task_ws and run the following commands:

./run.sh

which is a shell script that will catkin_make the workspace, and roslaunch the publisher and subscriber nodes.

The exported bag file is in the track_node folder, and the bag file is named track.bag. You can play the bag file by running the following command:

rosbag play ./src/track_node/track.bag

TODO list

• Finish the Task 1
• Finish the Task 2
• Finish the Bonus Task

Task 1

• Use template matching method to track the object
• Use Kalman Filtering (based on template matching measurements) to track the object
• Improve the performance of template matching by selecting the best templete based on the different dataset.
• Tune the search region of template matching to improve the performance of tracking
• Tune the parameters of Kalman Filtering to improve the performance of tracking, in order that using Kalman Filter would get a better performance than using only template matching.
• Adjust the width and height of the template to improve the performance of tracking.

Task 2

• Use constant velocity model and calculate the ADE and FDE
• Use constant acceleration model and calculate the ADE and FDE
• Use CTRV model and calculate the ADE and FDE
• Compare the performance of different models
• Visualize the dynamic process of the object detection
• Compare the performance when using the average velocity (and acceleration) of the last 1 second frames (Because by default I use the current frame's velocity and acceleration)

Bonus

• Read from the bag files
• Deploy the Kalman Filter Algorithm Successfully
• Topic Message Publishing and Subscribing
• Rosbag Recording (and Playback)