«An Autonomous Vision-Guided Helicopter Omead Amidi August 1996 Department of Electrical and Computer Engineering Carnegie Mellon University ...»
The visual odometer’s capabilities can be expanded to track particular objects of interest to perform autonomous search missions by the helicopter. Instead of locking on to an arbitrary object, the visual odometer can be guided by a secondary object detector to establish a lock onto a particular object and track it thereafter. In addition, the odometer could be configured recognize known environments such as a particular landing site by tracking known ground features to estimate helicopter position. In the following subsections, these two scenarios are investigated through preliminary experiments.
6.2.1 Object Tracking
Similar to the visual odometer, an object tracker can detect and track a particular object by template matching. A major difficulty of this approach is that object orientation is unknown and therefore all possible object orientations must be searched for in order to locate the object. Methods such as Karhunen-Loeve expansion can be used to reduce computational complexity and storage of necessary templates. Another problem stems from varying helicopter altitude which will change the size of the object in the image. Close regulation and measurement of helicopter altitude is necessary to further reduce the complexity of this search.
Rotated template images differ slightly from each other and are highly correlated. Therefore, the image vector subspace required for their effective representation can be defined by a small number of eigenvectors or eigenimages. The eigenimages which best account for the distribution of image template vectors can be derived by Karhunen-Loeve expansion .
Such principal component analysis methods can dramatically reduce the number of templates required to locate a particular object template. In fact, an experimental system based on the work presented in [5 11 and  could detect a section of a toy Jeep which was placed under the indoor testbed helicopter within +/- 80 degree orientation discrepancy using only 4 1 rotated (32x32) templates. The Jeep and the tracked template are shown in the two images of Figure 6- 1.
The processing frequency for searching the entire image is 2.5 Hz, using three C40 processors.
Although relatively slow, this object search process can provide a resetting mechanism for the lower visual odometer which cycles at field rate. The detected object, the Jeep in this case, can serve as one of the tracked templates of the visual odometer, therefore positioning the helicopter relative to the object. This relative positioning can then be employed for aerial tracking while the helicopter is controlled in relation to the object.
6.2.2 Helicopter Positioning using Known Environments In many common applications helicopters must fly close to known objects or land on predetermined landing pads. In most cases, it is reasonable to assume that special markings or features can be placed in view of the helicopter cameras to provide feedback for automatic close proximity hovering, landing, and take off. Aside from especially painted landing pads, it is desirable to use existing easy-todetect features which may be dispersed irregularly, but at known positions, for relative helicopter position estimation. The traditional approach to this problem is to back project known 3-D world features on the image plane and match them with the 2-D image features to estimate 3-D camera pose .Other methods use projective transforms and geometric invariants  for direct pose recovery from image features.
Chapter 6. Conclusions and Future Work 127Future Work Following the back projection approach, experiments were conducted with easy-to-detect known ground features for helicopter positioning. A number of easily detectable ground features, white dots, are placed at known locations on a black background under the testbed helicopter as shown in Figure 6-2.
Figure 6-2. Experiment with known ground features
A feature detector located the white dots in the image by image thresholding. Figure 6-3 shows the detector’s input and output images.The output image shows squares around detected features after lens calibration. The squares around features near the image boundaries do not line up perfectly with the raw features due to the larger lens distortion in these areas. The known ground features were then projected onto the image plane for matching. The proximity of image features to projected features was used as a matching criteria. In experimental trials, the closest 5- 10 features were selected for position updates.
The transformation from the world to the image plane is nonlinear but continuous and well behaved, allowing linear functions to locally approximate it. Using the current helicopter location, a Jacobian was constructed to iteratively approximate the change in helicopter position and attitude given image feature displacement. Linear extrapolation by Newton-Raphson was employed to update helicopter position between successive images.
Experimental trials demonstrated that the hovering helicopter’s movement in one field was small enough ( 10 pixeldfeature) to yield satisfactory ( 5 cm accuracy on average) position update with one or two iterations at 60 Hz processing frequency. Positioning accuracy proved satisfactory for future work in helicopter landings and takeoffs,
Robot helicopters are beginning to show their potential in an increasing number of applications. A small autonomous helicopter can perform aerial surveillance by transmitting images from on-board cameras taken at different altitudes and vantage points. This “eye-in-the-sky”capability, together with precise maneuvering of the helicopter, can provide a comprehensive picture of the environment central to scouting operations, site inspection, and movie production. Figure 6-5 shows a dangerous live power transmission line inspection and repair by an electrical worker sitting on a human piloted helicopter.’ As dangerous as they are, such applications are proving to be sufficiently cost effective to risk human lives, thus making a strong case for the necessity of autonomous robotic helicopters.
This dissertation has presented promising results in helicopter control using vision and it is my ultimate goal to apply these results to the development of robot helicopters for future real world applications.
1.Used with permission of Beyond 2000 television program.
Helicopter control is difficult; careful experimentation is essential to build a working prototype robot helicopter. For calibrated experiments, research described in this dissertation led to the development of a six Degree-Of-Freedom (6-DOF) testbed for safe indoor helicopter flight. The testbed measures ground truth helicopter position and attitude as well as working as a safety device for preventing crashes and out-of-control flight.
As shown in Figure A- 1, the testbed supports an electrical model helicopter which can fly freely in a cone-shaped volume six feet wide and five feet tall. The helicopter is fastened to six poles by rods which are free to move through two-degree-of-freedom (2-DOF) joints. The joint angles are measured by shaft encoders and are used by the computer to calculate the helicopter's ground truth position and attitude during flight tests.
An important issue concerns the effects of the testbed components on the helicopter dynamics in free flight. To minimize inertial variations, the testbed is built from light-weight metal and composite materials custom-designed and fabricated to minimize weight and friction. Minimizing friction is especially critical; friction has a tendency to significantly dampen helicopter movement. This gives a false sense of control system stability on the testbed compared to untethered flight which is undesirable.
Figure A- 1.6-DOF testbed
A.1 Testbed Helicopter
The testbed helicopter is an electric model, Kalt Whisper, as shown in Figure A-2. The helicopter is modified in several respects for testbed operation. Its power plant, a small DC motor, is built for high power, using a custom wound armature to operate more efficiently at higher voltages. The motor dissipates 0.5 HP and provides enough power to lift the helicopter, the rods, and the on-board sensors.
Because of its small size and high power dissipation rate, the motor is actively cooled by forced air to prevent premature failure. In addition, the motor voltage is regulated using a servo loop to keep rotor revolutions as constant as possible with varying helicopter loads.
A stereo pair of light-weight camera heads is mounted on the helicopter for vision along with vertical and directional gyroscopes for attitude sensing. The on-board sensors are shielded from helicopter vibration by a small suspension system built into the helicopter body. The suspension also dampens helicopter oscillations from the energy stored in the support rods just before takeoff.
The testbed helicopter is fastened to a planar light-weight structure made of three equal length aluminum tubes meeting at one point, designated as the testbed origin, and spread equally to span the area of an equilateral triangle as shown in Figure A-3.
134 Appendix A. Six Degree-Of-Freedom Testbed Mounting sites at the edges of the aluminum structure connect to support rods which travel through bearings at each 2-DOF joint, as illustrated in Figure A-4. Two rods are connected to each mounting site and three of their four joint angles, (a, e), are measured by shaft encoders to determine p, the 3D position of each mounting site. The helicopter position and orientation are then computed from the measured mounting site 3D locations.
Figure A-4. Testbed geometry
The support rods, 2-DOF joints, and helicopter catching mechanism are shown in Figure A-5.
The support rods are made from graphite arrows generally used for archery. The rods move through frictionless air bearings at each joint and are terminated by a spring-loaded stopper. The stopper cushions collisions as the helicopter reaches the rod’s extreme. A catcher mechanism connects each rod to the mounting site using support pins. The catcher mechanism stops the helicopter as it falls and the replaceable support pins absorb the impact energy by bending to prevent the rods from fracturing.
I have learned several valuable lessons while developing an autonomous vision-guided helicopter at Carnegie Mellon University. Others who strive to build complex robotic systems may benefit from
1. Follow an incremental and systematic design and evaluation approach: I started my graduate work by diving in and performing a series of outdoor experiments with model helicopters. I equipped a model helicopter with an array of sensors and performed a number of outdoor experiments which produced little concrete results. I was plagued with malfunctions, did not have a clear idea of system deficiencies, and most importantly had no way of quantifying system performance. Learning the hard way, I began to follow an incremental and systematic approach by building a number of indoor calibrated testbeds to design and evaluate each system component independently. The testbeds significantly reduced the chance of failure by verifying each system component before it was integrated into the final system. In spite of the seemingly extra amount of effort, this incremental approach to building complex integrated systems appears to be the most favorable in the long run.
This is especially true for helicopter control experiments in which one malfunction can cause a crash with major loss of time, resources, and safety.
2. Carefully verify that the traditional approach to the problem is not suficient before developing a new theory or approach: I learned this lesson while developing a helicopter control system. The highly unstable nature of helicopters led me to propose research in new adaptive control methods based on fuzzy logic and unsupervised learning techniques for my thesis work. To investigate the training capabilities of these methods, I designed a simple PD-based controller to stabilize and collect performance data from a model helicopter. To my surprise, the classical PD-controller worked quite well for hovering and low speed maneuvers which were the main modes of operation I was envisioning. The favorable performance was largely due to the high rate of positioning feedback and eliminated the need for rigorous system modeling. Therefore, instead of controls and system modeling, I focused on high-speed helicopter state estimation.
3. Reach a well-dejined ultimate goal through a series of manageable sub-goals: Throughout my graduate work, I maintained a single goal of stabilizing helicopters with on-board vision. I discovered that I reached this goal by reaching a number of short, manageable, and well-defined subgoals. The sub-goals maintained my high level of energy and reduced the large burden of what at times felt like an impossible task. Without clear sub-goals, it is easy to lose focus and attack interesting research problems for the sake of “research” alone.
4. Build what you need if you can ’ t j n d it or can ’taflord it: I wasted valuable time and resources by trying to use what was available to me instead of what I needed. I learned the hard way that just because a certain component is available it should not necessarily be part of the system. Instead of spending time on adapting available technology, I discovered that I could build certain components, such as image processing hardware, customized to my specifications and at a lower cost.
5. Give equal importance to every system component: The components of an integrated system can be thought of links in a chain. Every component is assembled for a purpose and must be treated with equal respect for successful operation of the entire system. At times, I have paid a high price for impatient implementation and improper use of seemingly unimportant system components.
Successful systems are built with patience and a consistent level of craftsmanship for every single component.