Chaokang jiang

About Me (蒋超康)

I was born in 1998 and currently work as a Computer Vision (CV) Algorithm Engineer at PhiGent Robotics in Beijing. I obtained my Master’s degree in Engineering in 2023, during which I was jointly trained by China University of Mining and Technology and Shanghai Jiao Tong University. I was fortunate to be guided by Professor Hesheng Wang, Professor Yanzi Miao and to collaborate with my senior, Dr. Guangming Wang from the University of Cambridge. At the IRMV Lab, I participated in numerous engineering projects, such as lawn mowing robots and 4D automatic annotation, which significantly enhanced my practical engineering skills.

Research Interests

• Computer Vision: image recognition, image generation, video captioning
• World Models: model-based automatic driving scene generation
• 2D/3D Object Detection: pure vision, pure LiDAR and 2D-3D fusion deep learning algorithms
• Deep Learning on Point Clouds: feature extraction, matching and fusion, graph network
• 3D Scene Flow: unsupervised learning, point cloud processing
• LiDAR Odometry: large-scale point cloud registration, motion estimation

News

• [May. 2024] Our paper “NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation” is available on arXiv.
• [May. 2024] Our paper “MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models” is available on arXiv.
• [Feb. 2024] Our paper “3DSFLabelling: Boosting 3D Scene Flow Estimation by Pseudo Auto-labelling” is accepted to CVPR 2024.
• [Feb. 2024] Our paper “DifFlow3D: Toward Robust Uncertainty-Aware Scene Flow Estimation with Iterative Diffusion-Based Refinement” is accepted to CVPR 2024.

Publications

Projects

Autonomous Intelligent Lawn Mower Robot (2021-09 ~ 2022-11)

Collaboration Project between SJTU IRMV & Positec Technology

Lawn mower going up and downhill

Lawn mower going up and downhill

3D obstacle detection

2D detection and 3D clustering fusion

2D segmentation and 3D clustering fusion

2D bird's eye view detection for safety

Responsibilities:

• Developed slope detection for lawn mowers using radar and depth camera, including point cloud clustering, segmentation, fitting, and detection techniques
• Developed 3D obstacle detection using point cloud and depth camera on the TX2 platform for various complex scenarios in flat and grassy environments
• Designed color texture feature encoding methods and efficient fusion with dense 3D point cloud features

The lawn mower robot project encompasses AI vision obstacle classification and recognition, multi-sensor offline and online calibration, visual-inertial odometry, global pose estimation based on multi-sensor fusion, obstacle detection techniques using LiDAR and depth cameras, and multi-sensor fusion for 3D obstacle detection.

Related News

Multimodal Fusion Technology in Autonomous Intelligent Lawn Mowing Robots (2021-09 ~ 2023-03)

Collaboration Project between SJTU IRMV & Positec Technology

FFPA-Net Efficient Feature Fusion with Projection Awareness for 3D Object Detection

The Main Network Structures of The Bi-modality Fusion Pipeline S&H-Fusion

Responsibilities:

• A new data pre-processing method is proposed to achieve more efficient fusion of the different signal features. The indexes between different sensor signals are established in advance and stored in a map, while synchronized sampling provides fast and accurate query correspondence for feature fusion.
• Multiple methods are explored to achieve cross-modal feature fusion more reasonably and efficiently, including soft query weights with perceiving the Euclidean distance of bimodal features, and fusion modules based on dual attention correlating the geometric features and texture features of the scene.
• A bi-modality feature fusion module with both hard and soft components is proposed, which guides the network to refine more accurate 3D positions and orientations of objects in the second stage. The proposed method achieves advanced performance on the nuScenes dataset, especially demonstrating powerful performance for small object detection with degraded image quality and objects with few LiDAR signals.

This is a preliminary research project aimed at exploring data-driven methods for image and LiDAR fusion. We focused on two 2D-3D fusion approaches to address the high latency and low accuracy of current 3D obstacle detection models. Our goal is to accelerate the practical application of these models.

FFPA-Net Soft and Hard Associations in Bi-modality Fusion Are All You Need

Integrated Network for Perception, Planning and Decision-making (2021-08~2022-10)

Independent Innovation Joint Fund Project of The Future Laboratory of The Second Aerospace Academy

Comprehensive demonstration of reconnaissance and strike missions in a simulated environment

Integrated Neural Network Technology Research for Perception, Planning, and Decision-making

Monocular Visual SLAM Real-time Dense Map Construction

Monocular Visual SLAM Real-time Dense Map Construction

Responsibilities:

• Developed perception tasks in simulation environments by integrating the Webots robot simulator with deep learning for 2D tracking and detection.
• Deployed depth estimation and mapping in SLAM models, and 3D semantic segmentation models.
• Deployed real-time dense mapping for monocular visual SLAM on ROS.

The project focuses on designing an integrated reinforcement learning network encompassing perception, planning, and decision-making. The perception module includes depth estimation, semantic segmentation, odometry estimation, loop closure detection, dense mapping, and object detection and tracking.

Video Offline 4D Automatic Labelling (2022-11 ~ 2023-04)

Collaboration Project between Hozonauto and SJTU IRMV

The Basic Framework of 4D Automatic Labelling Scheme

Automatic Labelling Scheme for Ground Static Elements

Responsibilities:

• Perception team leader, responsible for reporting, summarizing, and controlling the phased progress of each sub project. Timely follow up and assist in resolving technical issues encountered.

Imitating Tesla’s 4D automatic labeling solution, the process includes the following parts, input signal processing, standard scene perception tasks, ground element reconstruction and labeling, static scene reconstruction and 3D dynamic object labeling, overall 4D automatic labeling summary, badcase simulation.

Services

Conference Reviewers

• IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024
• IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2024
• Conference on Neural Information Processing Systems (NIPS) 2024

Journal Reviewers

• IEEE Transactions on Intelligent Vehicles (T-IV)
• IEEE Transactions on Robotics (TRO)

Powered by Jekyll and Minimal Light theme.