Orbital Express Advanced Video Guidance Sensor

In May 2007 the first US-sponsored fully autonomous rendezvous and capture was successfully performed by DARPA's Orbital Express (OE) mission. For the following three months, the Boeing ASTRO spacecraft and the Ball Aerospace NEXTSat performed multiple rendezvous and docking maneuvers to demonstrate some of the technologies needed for satellite servicing. MSFC's advanced video guidance sensor (AVGS) was a near-field proximity operations sensor integrated into ASTRO's Autonomous Rendezvous and Capture Sensor System (ARCSS), which provided relative state knowledge to the ASTRO GN&C system. AVGS was one of the primary docking sensors included in ARCSS. This paper provides an overview of the AVGS sensor that flew on orbital express, a summary of the AVGS ground testing, and a discussion of AVGS performance on-orbit for OE. The AVGS is a laser-based system that is capable of providing bearing at midrange distances and full six degree- of-freedom (6-DOF) knowledge at near ranges. The sensor fires lasers of two different wavelengths to illuminate retro- reflectors on the long range target (LRT) and the Short Range Target (SRT) mounted on NEXTSat. The retro- reflector filters allow one laser wavelength to pass through and be reflected, while blocking the other wavelength. Subtraction of one return image from the other image removes extraneous light sources and reflections from anything other than the corner cubes on the LRT and SRT. The very bright spots that remain in the subtracted image are processed to provide bearing or 6-DOF relative state information. AVGS was operational during the Orbital Express unmated scenarios and the sensor checkout operations. The OE unmated scenarios ranged from 10 meters to 7 kilometers ending in either a docking or a free-flyer capture. When the target was pointed toward the AVGS and in the AVGS operating range and field-of-view (i.e. along the approach corridor of the NEXTSat), the AVGS provided full 6-DOF measurements. The AVGS performed very well during the sensor check-out operations, effectively tracking beyond its 10-degree Pitch and Yaw limit-specifications. AVGS also provided excellent performance during the unmated operations, effectively tracking its targets, and showing good agreement between the SRT and LRT data. The AVGS consistently exceeded the tracking range expectations for both the SRT and LRT. During the approach to re-mate in scenario 3-1 recovery the AVGS began tracking the LRT at 150 m, well beyond the OE specified operational range of 120 meters, and functioned as the primary sensor for the autonomous rendezvous and docking. For all scenarios, the AVGS was used while ASTRO was in the approach corridor to NEXTSat, and during close proximity operations and docking.

The AVGS is a laser-based system that is capable of providing bearing at midrange distances and full six degreeof-freedom (6-DOF) knowledge at near ranges. The sensor fires lasers of two different wavelengths to illuminate retroreflectors on the Long Range Target (LRT) and the Short Range Target (SRT) mounted on NEXTSat.
The retroreflector filters allow one laser wavelength to pass through and be reflected, while blocking the other wavelength. Subtraction of one return image from the other image removes extraneous light sources and reflections from anything other than the comer cubes on the LRT and SRT.
The very bright spots that remain in the subtracted image are processed to provide bearing or 6-DOF relative state information.
AVGS was operational during the Orbital Express unmated scenarios and the sensor checkout operations.
The OE unmated scenarios ranged from 10 meters to 7 kilometers ending in either a docking or a free-flyer capture. When the target was pointed toward the AVGS and in the AVGS operating range and Field-of-View (i.e. along the Approach Corridor of the NEXTSat), the AVGS provided full 6-DOF measurements.
The AVGS performed very well during the sensor check-out operations, effectively tracking beyond its 10-degree Pitch and Yaw limit-specifications.
AVGS also provided excellent performance during the unmated operations, effectively tracking its targets, and showing good agreement between the SRT and LRT data. The approach to re-mate in Scenario 3-1 Recovery the AVGS began tracking the LRT at 150 m, well beyond the OE specified operational range of 120 meters, and functioned as the primary sensor for the autonomous rendezvous and docking.
For all scenarios, the AVGS was used while ASTRO was in the approach corridor to NEXTSat, and during close proximity operations and docking. suite of sensors were part of the ASTRO spacecraft GN&C system aiding AR&D efforts.
The ARCSS sensors consisted of a set of two visible light cameras, an infrared camera, a laser rangefinder, and the OE AVGS. The Boeing camera data was routed to a computer that processed the camera images using Boeing-developed algorithms, and then the computer combined that output with the information from the AVGS and the laser rangefinder.
The integrated solution was stored on both the data bus for the primary mission computer and a solid-state recorder, for later transmission to the ground.

AVGS FUNCTIONS AND SPECIFICATIONS
The AVGS, as built for the OE mission, was designed to guide a spacecraft in to a docked position with another spacecraft. The AVGS consists of two sets of laser diodes at wavelengths of 800 and 850 nanometers, a mirror through which the lasers fire, a camera that images the return from the lasers, and hardware, software, and firmware that process the returned images into relative position and attitude data. The sensor is designed to interact with a retro-reflective target.
The target has filters that allow one wavelength of AVGS laser to pass through and be reflected while blocking the other wavelength.
The target retro-reflectors are arranged in a pattern known to the AVGS software.
The sensor fires one set of lasers and captures an image, then it fires the second set of lasers and captures a second image. When this second image is subtracted from the first image and an intensity threshold is used, virtually all of the background clutter is eliminated. This feature played a significant role for Orbital Express in challenging lighting conditions. The remaining data is converted into a set of spots, and the spots are compared to the target pattern. Once a set of spots matching the target is found, the software computes the relative position and attitude between the target and the sensor. On Orbital Express, this data was output from the sensor and fed into the ARCSS computer for use by the Guidance and Relative Navigation (G&RN) algorithms, and was stored for telemetry to the ground. Figure 1 illustrates the laser illumination and processing sequence that the AVGS follows.
The overlap region for tracking the two targets simultaneously is nominally from 10 meters to 30 meters. The data output rate of the OE AVGS was 5 Hz, although the sensor internally tracked the target at 10 Hz. The sensor had a field-of-view (FOV) of +8 degrees, and was required to track the target while it was within a seven degree cone about the center of the FOV.

AVGS GROUND TESTING
There are several modes of operation for the AVGS. The primary AVGS modes of operation used during the OE mission were the following: 1) Standby mode, in which the sensor sends out status messages while awaiting further commands 2) Acquisition mode, in which the sensor is actively seeking a target, and 3) Track mode, in which the sensor is actively tracking a target. In addition, several other modes of operation were used occasionally or briefly. The Reset mode gave information every time the AVGS was powered on or issued a Reset command. The Maintenance mode was used to provide minor updates to the initial load (ILOAD) parameters during the mission.
Further information about the AVGS can be found in [1] and [2].

Advanced Video Guidance Sensor Specifications
The accuracy specifications for the OE AVGS are shown in Table 1. Due to the limited trajectories planned, the AVGS was never expected to be used beyond 120 meters, and the primary test facility was limited to 100 meters, so that was the maximum range tested on the ground. The specifications become more stringent as range decreases, because the greatest accuracy is required at docking. AVGS exhibits these characteristics, since it uses an imager chip and a fixed-focus lens, and the sensor's accuracy improves as the range decreases.
There are two specifications from 10 to 30 meters, since the AVGS OE target has both a Long Range Target (LRT) and Short Range Target (SRT), which are tracked over different ranges of operation. Testing was performed on the AVGS during every phase of its development.
The tests included sub-system testing, building and testing an engineering development unit, optical characterization testing, environmental testing on the flight unit, software testing, and final performance testing.
While the OE AVGS Engineering Development Unit (EDU) and Flight Unit were being built, the optical components were tested prior to final assembly.
The laser output power was measured for both sets of lasers.
In addition, the imager was exercised to ensure that it met its specifications.
Once the boxes were assembled, the unit was focused by using a spherical mirror to reflect the laser source back into the imager to measure the size of the spot. Then the full optical train was tested by taking pictures of corner cubes at different locations in the FOV illuminated by the sensor's laser diodes. The EDU was then shipped to MSFC.
The EDU was used for software development as well as optical characterization testing (OCT). The OCT was used to determine the optimal operating parameters (integration time, foreground and background laser power levels, and subtraction thresholds) used at each range.
The testing occurred from ranges of 1 meter out to 100 meters. Once the testing was complete and a set of operating parameters had been determined, the performance of the unit was tested at ranges between those used for OCT.
Once the flight unit had passed its assembly tests, it underwent standard environmental testing: electro-magnetic interference (EMI), electro-magnetic compatibility (EMC), vibration, shock, and thermal vacuum testing. This testing ensured that the unit was prepared to withstand the rigors of launch and of use in space.
After the environmental testing was complete, the flight unit was shipped m MSFC for final software development, installation, and further optical characterization and performance testing. During tests prior to the performance testing, it was noted that there were some problems with the optics as well as the performance (due in part to the optical issues.) Methods were developed to correct for or work around the optical issues, and then the performance testing began. The performance tests utilized a laser tracker with an accuracy of 0.001" (0.0254 nun) to ensure that the test setup was far more accurate than the specifications. The AVGS outputs its Range data in units of 1 nun, so truth data accuracy of 0.0254 mm was sufficient. The performance tests showed that the AVGS performed better than its specifications in every category.

FLIGHT PERFORMANCE
A number of different scenarios and operations were carried out with the AVGS powered on during the OE mission. All operations contained a segment during which the AVGS was in a static, fixed position relative to the SRT, with the majority of the scenarios including dynamic motion of the target relative to the sensor.
Some dynamic motion occurred while the two spacecraft were attached by a robotic arm, but most dynamic motion occurred during the unmated scenarios while the two spacecraft were free-flying relative to one another. During the Orbital Express mission, the AVGS performed extremely well. When the AVGS was powered on it was primarily in Track Mode or Acquisition Mode. During the AVGS warm-up the sensor was in Standby Mode. Throughout the mission the Diagnostic, Maintenance, and Reset Modes were occasionally used. These modes operated successful1y each time they were called. While AVGS was in Track Mode, the output data rate was consistently 5 Hz (as required). Twice during the mission a new ILOAD was developed on the ground and sent to the AVGS to be implemented. This process was run with ease and efficiency both times. The AVGS spent over 56.8 hours in Track Mode or Acquisition Mode while on orbit. The AVGS was powered on for over 76.9 hours during the whole of the Orbital Express mission.
Two modes of operation were never used on orbit: the Spot mode (which enables more flexible control of the AVGS optical parameters) and Segment mode (which allows pictUres to be captured by the sensor and transmitted through the serial port.) Neither of those modes were actual1y planned for use on orbit, but they were very useful for ground testing prior to flight. During the recovery from the anomaly in Scenario 3-1, Spot mode was considered for providing bearing information at long ranges, but the OE navigation filters were not set up to accept anything but Track data. .   The data from the first twelve meters for each of the six scenarios were separated out. Figure 3 has the AVGS range measured from the SRT for the six unmated operations. This shows how consistent the scenarios were and gives a feel for how the AVGS operated. All the scenarios started TOughly the same, which allows for a decent comparison. Figure 4 is the Azimuth for all six Scenarios, and Figure 5 shows the Elevation for all six Scenarios.

A VGS Performance Envelope
The region where the AVGS was able to track its target was a critical element in the performance analysis, including the maximum and minimum ranges for which the Short Range Target (SRT) and the Long Range Target (LRT) were able to be successfully tracked. The SRT was required to be tracked from mated range out to 30 meters. On departure 5 the SRT steadily tracked to between 32.2 and 34.2 meters, which is beyond the requirement. During approach the SRT consistently began tracking between 31.8 and 32.3 meters. The minimum range requirement for LRT tracking was 10 meters. On departure the AVGS tracked the LRT starting between 8.7 and 8.9 meters, while during approach the LRT tracking ended between 8.7 and 9.0 meters. Inside the critical proximity operations range of 60 meters, while the target was in the approach corridor, the AVGS solidly tracked its targets, greatly contributing to each successful docking. The AVGS typically tracked the LRT out to ranges between 100 and 110 meters, as long as the LRT was in the FOV. Most OE scenarios were not designed to keep the AVGS in the targets' FOV past this range. Throughout the mission, the AVGS remained in Track until a few minutes prior to leaving the departure corridor, or began tracking within a few minutes of entering the approach corridor. Due to anomalies early in Scenario 3-1, the AVGS target was pointed at the AVGS earlier than planned during re-mating operations, and the AVGS began tracking the LRT at 150 meters on the approach. The AVGS worked well out to 150 meters, even though pre-flight testing took place at a maximum range of 100 meters.
The AVGS successfully tracked a large range of attitudes, azimuths and elevations. The mission was not designed to test the AVGS tracking envelope, other than during ARCSS checkout. The data available for analysis encompasses a range of Pitch values from -26 through 26 degrees, Yaw from -23 through 10 degrees, Azimuth from -7 through 7 degrees and Elevation from -6 to 8 degrees. The maximum tilt angle on three scenarios exceeded the 25 degree requirement at ranges greater than 60 meters.

Scenario l-lA ARCSS Checkout
In Scenario 1-1 A, the Robotic Arm maneuvered the NEXTSat spacecraft into a variety of positions and attitudes relative to the ASTRa spacecraft. Excursions included points within the operating corridor and beyond. Two different automated arm motion scripts were run; one examined sensor performance at the expected edges of operation.
The other script went beyond expected operational capabilities for the sensors, so that sensor performance could be evaluated during target loss and recovery. Each script was performed twice. The ARCSS checkout was designed to test the operational limits of the other ARCSS sensors, but it benefited the AVGS as well.
In the ARCSS checkout, the AVGS performed very well, effectively tracking beyond its 10 degree tilt limit specifications at ranges closer than 30 m. The AVGS repeatedly reached a ten degree tilt angle, and successfully tracked at a tilt angle more than double the requirement. In addition to the tilt angle, the pitch and yaw were both tracked close to ten degrees individually. Figure 6 shows Azimuth vs. Elevation for one of the ARCSS checkout scripts. The motion went to the edges of the sensor's operation and a little bit beyond in Azimuth.

Positions of the SRT in the FOV (Azimuth and Elevation)
were compared between the two executions of the same script. The match was excellent, with slight variations that can be attributed to robotic arm vibration and noise, and demonstrated AVGS repeatability. Statistics were developed for the fifteen mated data samples. These statistics represent a combination of the repeatability of the AVGS and the repeatability of the docking mechanism. The mated data standard deviation was an order of magnitude smaller than the specifications, and the mean and median also fell within one sigma. When docked, the specifications encompass the bias from the desired zero solution as well as noise from the sensor. Biases can be taken out with ILOAD updates, but the biases seen during operations on orbit were not deemed by the program management team to be large enough to remove (thus all of the Azimuth and Elevation data is biased slightly in the negative direction). Figure 7 is two plots combined: Azimuth vs. Time and Elevation vs. Time, for one of the mated data sets. Figure 8 is another plot of Azimuth and Elevation versus Time, for the longest set of mated data that was taken on-orbit (5.2 hours). The data in these plots does not vary much about the mean. The sensor resolution is 0.000573 degrees. unmated operations. These three opportunities were the longest sets of mated static data available, including one 5.2 hour sample. The remainder of the mated data was taken immediately before the vehicles undocked, or while the sensor collected data after the two vehicles were rigidly docked.
The mated data is crucial to assessing the performance of the AVGS at close ranges; the docking mechanism used for OE had a very tight tolerance that essentially was below the threshold of the AVGS to detect. These mated opportunities provided an excellent source of ''truth data." The largest Elevation standard deviation was less than 15% of the specification. The largest standard deviation for Pitch, added to the largest mean, was approximately 60% of the 0.2 deg specification. This encompasses the bias and the noise of the sensor. Yaw was at 80% of the 0.2 deg specification for the same combination. The biases could have been corrected through an ILOAD, but that correction was not considered necessary for mission operations. The roll bias varied by 0.09 degrees, and is under the specification of 0.15 deg. The largest roll standard deviation was under 20% of the specification. Overall, very few data points fell outside the one sigma boundaries while in the mated configuration, well within the required three sigma specification.
The mated data showed high repeatability in Avas' performance.

Dual Track
Between the ranges of 9 meters and 32 meters, the Avas tracked both the LRT and SRT simultaneously. This region is commonly referred to as Dual Track. During Dual Track the Avas is measuring the relative position and attitude of NEXTSat through both the SRT and LRT. At the same point in time, the two solutions should be the same, since the spacecraft cannot physically have two attitudes and positions.
The Avas solutions from the LRT and SRT were closely correlated and remained consistent throughout the scenarios. The SRT solution was subtracted from the LRT solution during the dual track regions, and statistics were calculated on the solution difference. Dual Track covers a large range, and two different targets are tracked, which provide four requirements specifications for this region. The tightest specification, which was the LRT between 10 and 30 meters, was used for comparison. The range noise, represented by the standard deviation, was less than half of the specification of ±0.150 m. The largest noise for Azimuth and Elevation was slightly over half of the ±0.027 degree specification. Pitch and Yaw noise were also under half of their specification of ±0.7 degrees, with the largest Roll noise being close to 80% of the ±0.15 degree speci fication.
The plot in Figure 9 shows the range between the ASTRO and the NEXTSat as observed by Avas throughout Scenario 2-1. The solutions from Avas observing both the LRT and the SRT are included. The lower subplot is the difference between the LRT solution and the SRT solution for the dual track region. During Scenario 2-1, the two spacecraft separated to a distance slightly greater than 10 meters, held position there for an hour and a half, and then approached and docked. The two different targets were tracked simultaneously while the range exceeded approximately 8.8 meters, and the data from the two targets is nearly indistinguishable. Figure 10 and Figure 11 are similar to Figure 9, but examines the Azimuth and Elevation for Scenario 2-1. As with the Range, the LRT and SRT solutions are hard to distinguish as separate solutions when plotted together for the Azimuth and Elevation plots. Since there was no absolute "truth" data from the Orbital Express flight, the two different solutions were compared to one another. Looking at the plot of the difference, the range was easily less than the required specification.
The difference plots for the Azimuth and Elevation appear close to the one sigma specification, though when the actual standard deviation is calculated, it is less than half of the specification for Azimuth and Elevation. The last de-mate was the End of Life (EOL), in which the vehicles only backed away from one another, performed relative maneuvers and then the spacecraft went to separate orbits. There was no approach and re-mate for this final scenario. AVGS was powered on during the initial de-mate and separation from NEXTSat, though AVGS was only able to track while the ASTRO was on the departure corridor. The AVGS Range from EOL is depicted in Figure 12, with both the SRT and LRT solutions, roughly corresponding to the time spent in the departure corridor. The difference between the SRT and LRT solutions is in the lower subplot. Dual Track was only for a small portion of the EOL scenario. Azimuth and Elevation during EOL is depicted in Figure 13 and Figure 14, with the LRT and SRT difference in the lower subplots. As with Scenario 2-1, the standard deviations of the differences fell within the specifications, though that is not as clear in the subplots.

LESSONS LEARNED
In general, the AVGS performed extremely well, but there were some problems that could have been prevented or corrected on orbit, and there were some areas in which performance could have been improved.
The most glaring error that happened was a software problem that was first noticed during Scenario 5-1. The AVGS appeared to be sending out bad housekeeping data (temperatures, voltages, currents, etc.) and not putting out proper data during Acquisition mode. The problem was traced to an "AP Frame Overrun" erroran error created when the Application Processor (AP) does not respond to the Input/Output Processor (lOP) within its allotted time.
The root cause was analyzed and determined to be the fact that at long ranges, a number of single-pixel spots could 9 enter the recognition algorithms and thus require too long to process. The software was fixed and tested on the ground. However, since this error only occurred at longer ranges (greater than 70 meters), it was not deemed significant enough to warrant an AVGS software upload.
One surprising issue was that there were any lighting conditions that adversely affected the AVGS performance. In prior flights, there were no lighting situations that caused any problems for the AVGS. In this case, it was found that two of the four LRT retro-reflectors were smaller than the other two retro-reflectors, and for one of those two this was even more noticeable. Upon analysis, it turned out that the telecentric lens used by the AVGS essentially blurred the image a little bit. This meant that when there was a bright background behind one of the LRT spots, the brightness overlapped the actual spot locations, causing the spots to appear smaller. The two smaller-appearing spots have no spacecraft structure behind them, so when there is a bright background (as there often was due to the solar-inertial pointing of the NEXTSat with the sunlit earth behind it) they suffer the effects. In addition, one of the LRT spots had a shiny white conical antenna placed next to it. Again, due to the solar-inertial pointing, the NEXTSat was usually well lit by the sun. The antenna would shine brightly and would partially block out one of the LRT spots. These problems were only evident at fairly long ranges (100 or more meters), so again they were deemed relatively minor.
Another lesson learned from this sensor development was that there are usually unknown problems with optical systems. In this case, the EDU performed very well at every range, lending hope that the Flight Unit would be comparable. It turned out that there was some coma in the lens of the Flight Unit that caused performance problems in the 25 to 30 meter range for the SRT, when tested on the ground. This was compensated for, but it took time to ensure that the specifications were met. Then, it was discovered that the sensor detected a much larger sensor tilt angle than was accurate. For example, for every degree of motion in the FOV, the sensor would detect 1.05 degrees of motion. Since the specification for Azimuth and Elevation was a total error (bias plus noise) of 0.033 degrees, moving the target even one degree off of the boresight would put the unit out of specifications. An optical model was built that would explain the error, and then the model was tested against reality. The model came close, so a method of correction was developed based on that model and a series of very precise measurements across different portions of the sensor's FOV. The paper [2] goes into much more detail concerning this problem and its solution.
Yet another lesson learned from this project was that the target layout made a big difference in algorithm development, target recognition robustness, and solution accuracy. The LRT layout was somewhat dictated by the physical layout of the spacecraft. Unfortunately, that layout was a nearly-square target with one of the four corners being behind the other three. This out-of-plane location was important in allowing accurate 6-DOF measurements, but it meant that the spot would move more relative to the other spots during any NEXTSat tilt maneuvers. Considerable development and testing was required to ensure that the OE LRT would be properly recognized and measured throughout the required operating range.
The final lesson learned from this project was the extreme importance of extensive ground testing. It could safely be said that every new test uncovered some previously unknown problem with the sensor. The testing, followed by the correction of the problems found, took considerable time, but the time was well spent as evidenced by the excellent performance ofthe sensor on orbit.

CONCLUSIONS
The AVGS played a vital role in supporting the extremely successful Orbital Express mission. The sensor performed significantly better than required in a number of categories, including extending the SRT track range and outstanding repeatability between different mated operations. The solutions from the SRT and the LRT during Dual Track had excellent correlation. The amount of data collected, both static and dynamic, was substantial, providing the basis for more analysis and a better understanding of on-orbit AR&D. Future generations of video-based sensors will benefit from the experiences gained on Orbital Express.