Centeye Prototypes Vision-Based System for Nano Drones
The post below is written by Centeye President Geof Barrows. Centeye is a member of the TandemNSI community.
The next frontier in small UAV operations is the near-Earth environment, where GPS is inadequate and the structure cannot be known beforehand. As part of an Air Force funded project, Centeye has prototyped a vision-based system to allow small drones to both hover in place without GPS and avoid collisions with nearby obstacles.
This system was tested on a “nano” unmanned aerial vehicles (UAV) that weighs about an ounce and can fit in the palm of one’s hand. The video below shows sample flights in an indoor residence.
The vision system comprises four stereo camera pairs, each less than a half gram, arranged for 360-degree stereo vision. The vision system uses optical flow to measure self-motion and prevent drift and stereo vision to avoid nearby objects. The UAV implements “supervised autonomy” in which the operator provides high level control (e.g. “ascend” or “move right”) and the UAV computes and executes the maneuver while avoiding obstacles. All sensing and processing is performed on-board.
Centeye claims that very small UAV platforms have inherent advantages over larger platforms. Obviously they are easier to carry and more discreet. However, such small UAVs are more maneuverable and can simply fit into tight spaces that larger platforms cannot. The low mass also makes the UAV safe and physically robust. it can handle dozens of collisions without damaging itself or other objects.
The biggest challenge with nano drones is fitting the electronics within the tight payload constraints. Centeye achieved this through a holistic approach to designing the system. We developed the sensor circuit boards and software, and designed the camera chips and their optics. The only item not designed by Centeye was the UAV platform itself, which is an open-source nano quadrotor that was minimally modified to receive control input from the vision system.
Although the results so far are encouraging, there is more work to be done. As shown in the video, the current stereo vision system does not respond to blank walls and flies only at low speeds. It also will not work in the dark. However, Centeye has already prototyped techniques to address these issues and has a roadmap to incorporate them over the next year.
This work is part of a greater effort to study how flying insects use vision to perceive and fly through an environment, and use these insights to create new autonomy technologies. The techniques for processing omnidirectional optical flow for flight stabilization were drawn from earlier work, sponsored by the Air Force Office of Scientific Research, that studied how specific neurons in a fly’s visual system respond to motion in different directions and rotation around different axes.
We believes that “mother nature” has a rich set of “intellectual property” that may be used to complement more conventional techniques and ultimately provide new autonomy techniques for operation in GPS-denied environments.
Centeye believes this work, when completed, will open up new applications for small and “nano” air vehicles by allowing safe flight both near buildings, vegetation, and other clutter, as well as deep inside buildings or other enclosed spaces.