1. UAV and Robotics

We focus on environment sensing and autonomous control for UAV and robotics. Our research fileds: SLAM, Realtime mapping, 3D modeling, and autonomous control. The results and open-source codes are summarized as:

Map2DFusion: Real-time Incremental UAV Image Mosaicing based on Monocular SLAM


We present a real-time approach to stitch large-scale aerial images incrementally. A monocular SLAM system is used to estimate camera position and attitude, and meanwhile 3D point cloud map is generated. When GPS information is available, the estimated trajectory is transformed to WGS84 coordinates after time synchronized automatically. Therefore, the output orthoimage retains global coordinates without ground control points. The final image is fused and visualized instantaneously with a proposed adaptive weighted multiband algorithm. To evaluate the effectiveness of the proposed method, we create a publicly available aerial image dataset with sequences of different environments. The experimental results demonstrate that our system is able to achieve high efficiency and quality compared to state-of-the-art methods.

SDTAM: Semi-direct tracking and mapping for RGB-D cameras


We present a novel semi-direct tracking and mapping (SDTAM) approach for RGB-D cameras which inherits the advantages of both direct and feature based methods, and consequently it achieves high efficiency, accuracy, and robustness. The inputted RGB-D frames are tracked with a direct method and keyframes are refined by minimizing a proposed measurement residual function which takes both geometric and depth information into account. A local optimization is performed to refine the local map while global optimization detects and corrects loops with the appearance based bag of words and a co-visibility weighted pose graph.

NPU Drone-Map Dataset

NPU DroneMap Dataset

This dataset contains several aerial video sequences captured in different terrains and heights, which is used to evaluate the effectiveness of our proposed open-source Map2DFusion system. Until now, the dataset is consisted of several sequences recorded in different locations with over 100GB data and more captured sequences will be added.

NPU RGB-D Dataset


In order to evaluate the RGB-D SLAM algorithms of handling large-scale sequences, we recorded this dataset which contains several sequences in the campus of Northwestern Polytechnical University with a Kinect for XBOX 360. This is a challenging dataset since it contains fast motion, rolling shutter, repetitive scenes, and even poor depth informations.



Hands-Free is an open source robot hardware and software platform for robot research and development. It has well designed embedded hardware and software architecture. Through the platform, you can create your robot rapidly. This system include navigation, SLAM, computer vision modules, which also support other open-source platform such as ROS, MRPT, Pixhawk.



This program is a simple ground control station program. You can use the program to show MAV's position and attitude information through Mavlink protocol. Current, this program only tested for ArduCopter, although it may support other flight controller which use Mavlink. The map widget used in this program is based on opmapcontrol.



This program is a map widget which is based on opmapcontrol. We have fixed some minor problems of the implementation of opmapcontrol. There is a demo program show how to use the map widget. The map control has the functions of downloading offline map, showing UAV position, and editing waypoints.



This program is a pure Qt implementation of flight instrument widgets, including attitude display indicator (ADI), compass, and list viewer. These widgets are easy to use and modify.

ArDrone Autoflight Toolkit


This program is a automatic flight toolkit for ArDrone 2. We implenented 5 flight tasks. They are [0] tracking color block, [1] tracking Aruco marker [2] Flight along corridor , [3] Recorder video, [4] Auto landing.

Dynamic Path Planning


This program demonstrates how to dynamically plan a path from a begin point to an end point, where the environment is incrementally scanned. The simplest A* algorithm is used to plan a path from current position to destination using currently scanned map. Different with traditional path planning, it can plan route for completely unknown environment.





2. Machine learning and computer vision

The fundamental of intelligence robotics is artificial intelligence. Therefore, deep learning, scene parsing are our research interests. The results and open-source code are summarized as:



This program is a C++ toolbox for extracting CNN feature map from image. For achieving high-performance computation, it support CUDA acceleration. The average computation of 3 layers feature maps of a 640x480 image is 20 ms. You can easly integrated the code to your program.



This program is a C++ toolbox for generating superpixel from image. For achieving high-performance computation, it support CUDA acceleration. The average time for a 640x480 image is 40 ms. You can easily integrated the code to you embedded program.

Convolutional Deep Belief Networks


This program is an implementation of Convolutional Deep Belief Networks with 'MATLAB','MEX','CUDA' versions. In this code, the binary and Gaussian visable types are both supported. In addition, CUDA acceleration is also included. We provide some demo programs to show the usage of the code.

DeepNet with CUDA


In this project, Shaoguang Cheng implement a deep learning toolbox (DeepNet) including RBM, DBN, Multi-modal DBN with Python, in which the majority of matrix operations are carried on GPU by using the Cudamat to speed up the calculation process. There are some examples to show how to use this package.

camshiftKalman - An object tracking project using camshift and Kalman Filter based on OpenCV


We implement an object tracking algorithm using camshift and Kalman Filter. Three features, hue, saturation and rotation invariant Local Binary Pattern, are used to model the tracking object. Kalman Filter is employed to smooth the motion trajectory and predict the next position when the object is occluded.



This program can align a face to front view. This system is an implementation of paper "Effective Face Frontalization in Unconstrained Images, Tal Hassner, Shai Harel, Eran Paz, Roee Enbar"

EICAM: An Efficient C++ Implimentation of Different Camera Models

The EICAM is the a part of PIL library, which includes some pretty useful tools for C++ programing especially in the areas of mobile robotics and CV (computer vision). Since cameras projection and unprojection actions are often needed in CV or other area such as Robot localization like SLAM (simultaneous localization and mapping), we provided EICAM for efficient implementation of different camera models including PinHole, ATAN, OpenCV, OCAM .etc.

Comments are closed.