Centeye Prototypes Vision-Based System for Nano Drones


The post below is written by Centeye President Geof Barrows. Centeye is a member of the TandemNSI community.

The next frontier in small UAV operations is the near-Earth environment, where GPS is inadequate and the structure cannot be known beforehand. As part of an Air Force funded project, Centeye has prototyped a vision-based system to allow small drones to both hover in place without GPS and avoid collisions with nearby obstacles.

This system was tested on a “nano” unmanned aerial vehicles (UAV) that weighs about an ounce and can fit in the palm of one’s hand. The video below shows sample flights in an indoor residence.

The vision system comprises four stereo camera pairs, each less than a half gram, arranged for 360-degree stereo vision. The vision system uses optical flow to measure self-motion and prevent drift and stereo vision to avoid nearby objects. The UAV implements “supervised autonomy” in which the operator provides high level control (e.g. “ascend” or “move right”) and the UAV computes and executes the maneuver while avoiding obstacles. All sensing and processing is performed on-board.

Centeye claims that very small UAV platforms have inherent advantages over larger platforms. Obviously they are easier to carry and more discreet. However, such small UAVs are more maneuverable and can simply fit into tight spaces that larger platforms cannot. The low mass also makes the UAV safe and physically robust. it can handle dozens of collisions without damaging itself or other objects.

The biggest challenge with nano drones is fitting the electronics within the tight payload constraints. Centeye achieved this through a holistic approach to designing the system. We developed the sensor circuit boards and software, and designed the camera chips and their optics. The only item not designed by Centeye was the UAV platform itself, which is an open-source nano quadrotor that was minimally modified to receive control input from the vision system.

Although the results so far are encouraging, there is more work to be done. As shown in the video, the current stereo vision system does not respond to blank walls and flies only at low speeds. It also will not work in the dark. However, Centeye has already prototyped techniques to address these issues and has a roadmap to incorporate them over the next year.

This work is part of a greater effort to study how flying insects use vision to perceive and fly through an environment, and use these insights to create new autonomy technologies. The techniques for processing omnidirectional optical flow for flight stabilization were drawn from earlier work, sponsored by the Air Force Office of Scientific Research, that studied how specific neurons in a fly’s visual system respond to motion in different directions and rotation around different axes.

We believes that “mother nature” has a rich set of “intellectual property” that may be used to complement more conventional techniques and ultimately provide new autonomy techniques for operation in GPS-denied environments.

Centeye believes this work, when completed, will open up new applications for small and “nano” air vehicles by allowing safe flight both near buildings, vegetation, and other clutter, as well as deep inside buildings or other enclosed spaces.


6 responses to Centeye Prototypes Vision-Based System for Nano Drones

  1. Geoffrey L. Barrows posted a blog post… | Drones News Magazine

    […] (More info and full post here)I’ve been experimenting with putting 360 degree vision, including stereo vision, onto a Crazyflie nano quadrotor to assist with flight in near-Earth and indoor environments. Four stereo boards, each holding two image sensor chips and lenses, together see in all directions except up and down. We developed the image sensor chips and lenses in-house for this work, since there is nothing available elsewhere that is suitable for platforms of this size. The control processor (on the square PCB in the middle) uses optical flow for position control and stereo vision for obstacle avoidance. The system uses a “supervised autonomy” control scheme in which the operator gives high level commands via control sticks (e.g. “move this general direction”) and the control system implements the maneuver while avoiding nearby obstacles. All sensing and processing is performed on board. The Crazyflie itself was unmodified other than a few lines of code in it’s firmware to get the target Euler angles and throttle from the vision system.Below is a video from a few flights in an indoor space. This is best viewed on a laptop or desktop computer to see the annotations in the video. The performance is not perfect, but much better than the pure “hover in place” systems I had flown in the past since obstacles are now avoided. I would not have been able to fly in the last room without the vision system to assist me! There are still obvious shortcomings- for example the stereo vision currently does not respond to blank walls- but we’ll address this soon…See More […]

Leave a reply