Alignment of Noisy Signals

We study the relative performance of various methods for aligning noisy one-dimensional signals. No knowledge of the shape of the misaligned signals is assumed. We simulate signals corrupted by both additive noise and timing jitter noise which are similar in complexity to nose-to-nose oscilloscope calibration signals collected at NIST. In one method, we estimate the relative shift of two signals as the difference of their estimated centroids. We present a new adaptive algorithm for centroid estimation. We also estimate relative shifts from three different implementations of cross-correlation analysis. In a complete implementation, for signals, relative shifts are estimated from all ( 1) 2 distinct pairs of signals. In a naive implementation, relative shifts are estimated from just ( 1) pairs of signals. In an iterative adaptive implementation, we estimate the relative shift of each signal with respect to a template signal which, at each iteration, is equated to the signal average of the aligned signals. In simulation experiments, 100 misaligned signals are generated. For all noise levels, the complete cross-correlation method yields the most accurate estimates of the relative shifts. The relative performance of the other methods depends on the noise levels.


I. INTRODUCTION
W E STUDY the accuracy of various methods for align- ment of noisy one-dimensional signals.Our work is motivated by a project where we want to characterize impulse response functions of high-speed sampling oscilloscopes [1], [2].In our experiments, we measure noisy signals which drift in time.To improve the signal-to-noise ratio (SNR), we collect many waveforms.We wish to estimate the unknown "true" signal from an ensemble of misaligned noise corrupted signals.
We assume that each noisy signal is shifted with respect to the others.That is, the expected value of the th signal at time is (1) where is a unobserved drift parameter and is the unobserved signal we want to estimate.From a set of signals, we cannot estimate the set of absolute drifts .However, we can estimate the relative drift of the th and th signal .In this work, for all methods, we estimate the relative shift of the th signal with respect to the first signal.Hence, we estimate relative shifts from misaligned signals.Based on the estimated relative shifts, we align each of the signals and then compute the average of the aligned signals.This signal average is our estimate of .The best we can do is estimate a translated version of , that is where depends on the choice of the reference signal.If we pick the first signal as the reference, we must translate the th observed noisy signal by the estimated value of .Summing the first signal with the other translated noisy signals, and dividing by yields our signal average.Given a relative shift estimate, we translate a signal by a Fourier method.
In previous studies, the relative shift of two noisy signals was estimated as the difference of the time centroids of the signals [3].These centroid estimates are sensitive to noise.In this work, we present a new time centroid estimate which is robust against the effects of noise.
If the true signal were known, the relative shifts of a set of signals can be determined by an optimal matched filtering approach.In this approach, cross-correlation analysis of each signal with respect to a template signal [equal to ] would yield shift estimates.Since is unknown, an optimal matched filtering approach is not feasible.However, relative shifts can be estimated by an iterative suboptimal matched filtering approach [3], [4].We denote this iterative approach as the "adaptive" implementation of the cross-correlation method.For comparison, we also estimate the relative shifts from cross-correlation analysis of each signal with respect to the first signal (this is equivalent to halting the adaptive algorithm after the first iteration).We call this the "naive implementation" of cross-correlation analysis.
We introduce a new cross-correlation method for estimating the relative shifts of misaligned signals.Based on cross-correlation analysis of each of the distinct pairs of signals, we estimate the relative shifts of interest by the method of least-squares.In [18], [19] relative shifts were estimated from cross-correlation analysis of all possible pairs of signals.The noise was assumed to be additive and independent of the signal.Assuming knowledge of the power spectrum of both the additive noise and of the signal, a weighted least-squares estimate of the relative shifts was obtained.In this method, the weighting matrix is nondiagonal.In our work, due to jitter, the noise is not independent of the signal.Further, we do not assume knowledge of the power spectrum of the signal nor do we assume knowledge of the power spectrum of the noise.
U.S. Government work not protected by U.S. copyright Hence, the method developed in [18] and [19] is not applicable to our case.
We simulate signals which have complexity similar to experimental data.Near the boundaries, the noise-free simulated signal and noise-free experimental oscilloscope signals are approximately flat.Over a short interval of time, both the simulated and real signals rise and then fall (somewhat like a differentiated Gaussian pulse does).After this rapid rise and fall, the signals follow quasiperiodic oscillations which eventually damp out.We do not claim that the alignment methods presented here will work well for signals which display significant complexity at either boundary.
The paper is organized as follows.In Section II, we define our robust centroid estimate of relative shift.In Section III the three implementations of cross-correlation analysis are presented.In Section IV, the relative performances of the different methods for estimation of relative shifts are studied by means of Monte Carlo simulation.For all cases considered, the complete crosscorrelation method is the most accurate method.In Section V, we estimate relative shifts for a set of 100 measured misaligned oscilloscope signals.

II. CENTROID METHOD
In [3], the time centroid of a signal was computed in two ways.The first centroid estimate was computed from the positive part of the signal (3) The second estimate was (4) These estimates are not robust against the effects of noise.To illustrate, suppose that beyond a certain time , the true (noisefree) signal is 0. Further, assume that additive noise contaminates the signal.In the centroid computation, values of the signal corresponding to , contribute no useful information.Instead, noise in this part of the signal increases the variability of the estimate.
To reduce variability, we estimate the centroid from a subset of the full signal.To belong to this subset, the magnitude of the signal must exceed a selected threshold.We estimate the centroid of a signal as ( 5) where if otherwise (6) where is the threshold.We estimate the relative shift between signal and as .

A. Theoretical Threshold Selection Rule
Given knowledge of the true signal and the actual relative shifts, we can minimize the root-mean-square (RMS) prediction error of the relative shift estimate as a function of .The threshold which minimizes this RMS prediction error is called the "theoretical" estimate of the optimal threshold.
We simulate pairs of misaligned signals as follows.For each pair, the relative drift of the signals is a Gaussian random variable with expected value equal to 0 and standard deviation equal to 5. Each signal has 4096 time samples.The th time sample of the th simulated waveform is (7) where the timing jitter of the th time sample of the th signal is modeled as a realization of a Gaussian process (8) The additive noise realizations are mutually independent Gaussians.Each has expected value of 0 and variance .Jitter realizations at different times are independent; additive noise realizations at different times are independent; and jitter is independent of additive noise at any time.This assumption is consistent with how the high-speed oscilloscope under study operates.The analytic expression for the simulated signal is given in Appendix I.
In  where is selected by the theoretical selection rule method.(Fig. 2).
The noise-free version of is well localized if we threshold the signal at .That is, for .Hence, 0.08 is a localization level.When , the subset of the signal which satisfies , is not well localized; random excursions above 0.08 cause high variability in the centroid estimate.For the example of Fig. 1, the theoretical estimate of the optimal threshold is .The accuracy of the centroid method is dramatically improved by our method (Table I).
For various choices of and , we compute RMS prediction error.For each choice of and , we simulate 12 runs.In each run, we simulate 1000 pairs of noisy signals and select the optimal threshold by minimizing RMS prediction error.(In the study, we vary the threshold from 0 to 1 by increments of 0.01.)In Table II, we list the median value of the 12 estimates for the various noise levels.For all cases, .

B. Empirical Threshold Selection Rule
In real applications, we do not know the actual shifts nor do we know the shape of the "true" signal.Hence, the theoretical selection rule is not practical.As an alternative, we can select the threshold by an empirical threshold selection rule.
In general, the average of misaligned signals will have less total power than the average of properly aligned signals [9].More explicitly, consider a noise-free ideal signal which, at frequency , has Fourier transform .If this signal is translated by , at frequency , its Fourier transform becomes .If the shift is a random variable, the magnitude of the expected value of is less than 1.Hence, at each frequency, on average, misalignment reduces power in an ideal signal.By Parseval's Theorem, the integrated power over all frequencies equals the integrated square of the signal in the time domain.According to our empirical selection rule, the best estimate of the optimal threshold is the one that maximizes the total power of the average of the aligned signals.

= 5
Our statistical measure of the total power of the average of the aligned signals is (9) where is the th time sample of the average of the aligned signals.In practice, for a set of noisy signals, we estimate relative shifts for each of many candidate thresholds.For each threshold, we estimate the relative shifts, the average of the aligned signals and . The threshold which maximizes is our empirical estimate of the optimal threshold.We study the accuracy of our empirical threshold selection rule by a Monte Carlo method.In each run of the Monte Carlo experiment, we simulate a set of 100 misaligned noisy signals.In Figs.3(a) and (b) we illustrate the empirical threshold selection rule by plotting as a function of threshold for a sample run where , and .For this case, the empirical and theoretical selection rules agree very well.In Table III, we list empirical estimates of the optimal threshold for other cases.The statistical correlation between the empirical (Table III) and theoretical (Table II) estimates is .Further all the empirical and theoretical estimates satisfy .Thus, the estimates are plausible because they exceed the localization level of 0.08 (see Section II-A) by or more.In Tables IV and V, we compare the relative performance of the empirical and theoretical threshold selection rules according to a RMS prediction error criterion.The RMS statistic is defined in Appendix II.For the cases studied,the RMS associated with the empirical selection rule is no more than 12% more than the   Comment: Consider the case where jitter realizations at different times are independent; additive noise realizations at different times are independent; and jitter is independent of additive noise at any time.On average, the number of samples above the threshold is inversely proportional to the interval between time samples.For the case where the realizations of additive noise and the realizations of the jitter noise at different times are mutually independent, the accuracy of the centroid estimate should improve if the sampling interval is reduced provided that the total duration of the experiment is fixed.This is so because the centroid estimate is a weighted average.As the number of samples in the weighted average increases, the variability of the weighted average decreases.Hence, the root-mean-square prediction error of the relative shift should scale as RMS (10)  where is the number of samples above the threshold when the interval between samples is .In a Monte Carlo experiment, we sampled realizations of the simulated signal over the interval (1,4096).We varied the sampling interval from 0.1 to 5. In the simulation, , , the RMS relative shift of the two signals in each pair is 5, and the threshold is .For this special case, the RMS prediction error is well approximated by the following formula RMS (11) Over this range of , RMS varied from 0.2 to about 1.4.The root-mean-square difference between the computed and predicted value of RMS was less than 2%.From this study, for mutually independent additive noise and jitter noise processes, we conclude that we can make the centroid estimate as accurate as we like, provided that we can collect an unlimited number of time samples during a fixed time interval.For other signal models, where the jitter and additive noise are mutually independent, we expect that RMS .However, the proportionality constant would not necessarily be the same as the above case.

A. Naive Cross-Correlation
In the simplest implementation of the cross-correlation approach, we estimate the relative shift of the th signal with respect to the first signal.The th sample of the translated version of the th signal is .To estimate the relative shift of the th and first signal, we minimize (12) as a function of .Denote the value of which minimizes the above as .Minimization of the above is equivalent to maximization of the cross-correlation (at lag 0) the first signal and the translated version of the th signal.A reasonable estimate of is .We shift a signal by a Fourier method.We use the fact that the Fourier transform of a translated signal , at frequency is where is the Fourier transform of .Hence, to estimate the translated signal, we adjust the Fourier transform of and then do an inverse Fourier transform to estimate the translated version of .Before computing the Fourier transform of each noisy simulated signal, we taper the signal with a cosine bell data window [21].Each signal has 4096 samples.The taper is 1 for .(Near the boundaries, the expected value of the simulated signal is essentially 0. For these simulated signals, tapering has a negligible effect on the relative shift estimates.)To ensure convergence to the global minimum, we first evaluate (12) over a grid centered on the actual value of the relative shift.About the grid value which minimizes (12), we search for the global minimum using a golden search and parabolic interpolation algorithm [22].
In this naive cross-correlation method, we compute relative shifts from the signals.However, from signals, there are distinct pairs of signals.Hence, is not the most accurate estimate of .For this reason, we call the naive cross-correlation estimate of .

B. Complete Cross-Correlation Method for Alignment
To demonstrate how to estimate the relative shifts more accurately, consider the case where .The three relative shifts we seek to estimate form the vector (13) From the data, we estimate six relative shifts.These estimates form the data vector (14) We have (15) where is a residual vector and (16) We estimate by minimizing the Euclidean norm of .That is, we use the method of least squares.The least-squares estimate of is (17) where denotes the transpose of .Since is a function of all six relative shifts it is expected to be a better estimate than the naive estimate which is based on just three relative shifts.
There are analytic expressions for and .In general, for signals, We derive the inverse of with the Sherman-Morrison formula [23] which states that (19) where (20) We get that if otherwise. ( For the case , we have In general, the estimated relative shift between the th and th signals is (24) We call (24) the complete cross-correlation estimate of .When and (25) Hence, the complete cross-correlation estimate is the weighted mean of different estimates.Each of these estimates has an expected value equal to .The term is weighted twice as much as the other terms .If we assume that the terms are statistically independent VAR VAR (26) When we combine statistically independent estimates which are unbiased but have different variances, the optimal weights are inversely proportional to the variance of each term [24, p. 88].
Given the assumption of independence, the weights in (24) are optimal.Under the assumption that the terms in ( 24) are statistically independent, the relative variance of the complete crosscorrelation and naive cross-correlation estimates is To derive this, we use the fact that the variance of a weighted average of independent random variables is VAR VAR .We do not expect this much variance reduction because the terms in ( 24) are statistically dependent.

C. Adaptive Cross-Correlation
In the adaptive cross-correlation [3], [4] method, during each iteration, we estimate all relative shifts by maximizing the crosscorrelation of each of the signals and a template signal.Initially, the first signal is the template signal.From the relative shifts, we compute the average of the shifted signals.The updated template signal is set equal to this signal average.In [4], the authors remark that the adaptive cross-correlation method approach is a suboptimal matched filter approach.In an optimal matched filter approach, the template is the "true" waveform.
We iterate the adaptive algorithm 30 times.Hence, for set of 100 signals, we perform cross-correlation analyses.For all cases, by 30 iterations, the mean squared difference between each of the shifted signals and the template signal converges to a precision of 0.004% (or less).By 30 iterations, the RMS statistic is accurately determined to 0.1% (or less).

IV. COMPARISON STUDY
For each choice of and , we simulate a set of 100 misaligned signals.The standard deviation of the random shift associated with each signal is 2.5.We estimate relative shifts by naive cross-correlation, complete cross-correlation, adaptive cross-correlation, and the centroid method.In Fig. 4, we plot the RMS prediction error for all the methods versul additive noise RMS.See Appendix II for a definition of RMS prediction error for estimation of relative shifts.If the terms in (24) were independent, the RMS ratio for the complete and naive cross-correlation methods would be according to (27).For the cases studied, the ratio falls in the interval (0.17,0.62).Since the observed variance reduction is less than the theoretical value given in (27) [which is derived under the assumption the terms in (24) are independent], we conclude that the terms in (24) are statistically dependent.For all noise levels, the complete cross-correlation method yields the lowest RMS prediction error.In all cases, the adaptive cross-correlation method yields lower RMS prediction error than does the naive correlation method The relative performance of the centroid and adaptive cross-correlation methods depends on the noise level.
Comment: The performance of the adaptive cross-correlation method should be no better than the performance of an optimal matched filtering approach.In the optimal filtering approach, the template signal is the expected value of an ensemble of perfectly aligned noisy signals.In the adaptive cross-correlation method, the template is the average of a finite number of imperfectly aligned noisy signals.The alignment is imperfect because the relative shift estimates are not exactly equal to the true relative shifts.For high jitter noise, the adaptive cross-correlation method performs poorly compared to the complete cross-correlation method and the centroid method (Fig. 4).
As the number of signals increases, we expect that the RMS prediction error of the adaptive cross-correlation method to decrease, but never to go below, the nonzero RMS prediction error of the optimal matched filtering approach.In contrast, the RMS prediction error of the complete cross-correlation method should tend to an arbitrarily low level provided that the number of signals is increased to an arbitrarily high level.This is expected because the number of distinct pairs of signals is whereas the number of parameter we estimate is only .

V. EXPERIMENTAL DATA
We estimate the relative shifts for a set of 100 experimental oscilloscope signals collected at NIST.In each signal, the interval between samples is 1.953 ps.In the top part of Fig. 5, we show one of the measured oscilloscope waveform and the average of the 100 aligned signals.To compute this signal average, we use the relative shifts provided by the complete crosscorrelation method.In Fig. 6, we show the estimated relative shifts determined by the different methods.In the cross-correlation analyses, signals are not tapered before computing Fourier transforms.We do not taper because as the boundaries are approached, the signal levels off to a nonzero plateau.Hence, near the boundary, tapering would distort the signal from this plateau level toward zero.
We expect that a poor estimation method will yield a rougher relative shift curve compared to a good estimation method.Thus, the centroid method appears to be the least accurate method and the adaptive and complete cross-correlation appear to be the most accurate methods (Fig. 6).The relative shift estimates for the complete and adaptive cross-correlation methods are close; the RMS value of their difference is 0.0125 ps.In the simulation study, for low jitter noise levels, the adaptive and complete cross-correlation method estimates were in good agreement (Fig. 4).Hence, the observed closeness of the adaptive and complete cross-correlation estimates may be attributed to a relative low jitter noise level.For these signals, RMS jitter noise is about 1 ps.

VI. SUMMARY
We simulated signals corrupted by both additive noise and timing jitter noise.In one method, we estimated the relative shift of two signals as the difference of their estimated centroids.We presented a new adaptive algorithm for centroid estimation which is robust against the effects of noise.We also estimated relative shifts by three different implementations of cross-correlation analysis.In the naive implementation of the cross-correlation method, for set of signals, relative shifts are estimated from cross-correlation analysis of pairs of signals.We introduced a complete implementation of the cross-correlation method.In this approach, estimates were determined from cross-correlation analysis of all distinct pairs of signals.In an adaptive implementation of the cross-correlation method, relative shifts were estimated by maximizing the crosscorrelation between the shifted version of each signal and a template signal.After each iteration of the adaptive algorithm, the template was equated to the average of the shifted signals.For all noise levels, the complete implementation of the cross-correlation method was the most accurate method.The accuracies of the adaptive and complete cross-correlation methods were close for low jitter noise.At high jitter noise, the adaptive method was dramatically inferior to the complete cross-correlation method.The relative accuracy of the robust centroid method and the adaptive implementation of the cross-correlation method depended on the choice of noise levels.The relative accuracy of the robust centroid method and the naive implementation of the cross-correlation method depended on the choice of noise levels.In all cases, the adaptive implementation of the cross-correlation method was more accurate than the naive implementation of the cross-correlation method.We also estimated relative shifts for a set of 100 experimental oscilloscope signals.relative shifts from signals, we compute the RMS value of prediction errors which are adjusted so that their mean value is zero.To explain why we do so, consider the ideal case where we estimate the relative shifts exactly.For this case, the true absolute shifts are related to the estimated relative shifts by (28) where for , the constant satisfies .Because we cannot estimate the constant from the data, any measure of performance should be invariant to translation of all relative shift estimates by an arbitrary constant.We select the following translation invariant measure of performance Kevin J. Coakley and Paul Hale Abstract-We study the relative performance of various methods for aligning noisy one-dimensional signals.No knowledge of the shape of the misaligned signals is assumed.We simulate signals corrupted by both additive noise and timing jitter noise which are similar in complexity to nose-to-nose oscilloscope calibration signals collected at NIST.In one method, we estimate the relative shift of two signals as the difference of their estimated centroids.We present a new adaptive algorithm for centroid estimation.We also estimate relative shifts from three different implementations of cross-correlation analysis.In a complete implementation, for signals, relative shifts are estimated from all ( 1) 2 distinct pairs of signals.In a naive implementation, relative shifts are estimated from just ( 1) pairs of signals.In an iterative adaptive implementation, we estimate the relative shift of each signal with respect to a template signal which, at each iteration, is equated to the signal average of the aligned signals.In simulation experiments, 100 misaligned signals are generated.For all noise levels, the complete cross-correlation method yields the most accurate estimates of the relative shifts.The relative performance of the other methods depends on the noise levels.Index Terms-Adaptive estimation, cross-correlation analysis, high-speed sampling oscilloscopes, least-squares estimation, robust estimation, signal alignment, timing jitter noise.

Fig. 1 (
a), we plot the noise-free signal versus time.In Fig. 1(b), we plot a noisy realization of the signal versus time where and .As a dashed line, we plot

= 5 RMS
associated with the theoretical selection rule.Hence, the empirical selection rule performs almost as well as the theoretical selection rule.

Fig. 4 .
Fig. 4. RMS prediction error versus RMS additive noise when 100 misaligned simulated signals are aligned by different methods.

Fig. 6 .
Fig. 6.Relative shift estimates for experimental data due to different methods.
the following damping factors: Define a sigmoid function as follows: where Define the following arguments: Define the following functions: Our function is .APPENDIX II PERFORMANCE CRITERION FOR ESTIMATING RELATIVE SHIFTS FROM SIGNALS When we estimate Paul Hale received the Ph.D. degree in applied physics from the Colorado School of Mines, Golden, CO, in 1989.He has been with the Optoelectronics Division of NIST, Boulder, CO, since 1984.He has conducted research in birefringent devices, mode-locked fiber lasers, fiber chromatic dispersion, broadband lasers, interferometry, polarization standards, and photodiode frequency response.He is presently Leader of the High-Speed Measurements Project in the Sources and Detectors Group.His present interests are in precision optoelectronic frequency response measurement and related areas.Dr. Hale, along with a team of four scientists, received the Department of Commerce Gold Medal in 1994 for measuring fiber cladding diameter with an uncertainty of 30 nm.In 1998, he received a Department of Commerce Bronze Medal, along with Dr. C. M. (Jack) Wang and three other scientists for developing measurement techniques and standards to determine optical polarization parameters.

TABLE I THEORETICAL
ESTIMATES OF OPTIMAL THRESHOLD AND RMS PERDICTION ERROR FOR CENTROID METHOD FOR A PAIR OF MISALIGNED SIMULATED SIGNALS.THE SIMULATED SIGNAL IS THE SUM OF A DIFFERENTIATED GAUSSIAN, THREE GAUSSIANS AND A DAMPED SINUSOID.= 2:5

TABLE II THEORETICAL
ESTIMATES OF OPTIMAL THRESHOLDS FOR CENTROID METHOD OF ESTIMATION OF RELATIVE SHIFTS OF 100 MISALIGNED SIMULATED SIGNALS.

TABLE III EMPIRICAL
ESTIMATE OF OPTIMAL THRESHOLD FOR CENTROID ESTIMATE OF RELATIVE SHIFTS OF 100 MISALIGNED SIMULATED SIGNALS.

TABLE IV RMS
FOR ESTIMATED SHIFTS OF A SET OF 100 SIGNALS.THRESHOLDS ARE SELECTED EMPIRICALLY (TABLE III)

TABLE V RATIO
OF RMS PREDICTION ERRORS OF RELATIVE SHIFT FOR EMPIRICAL THRESHOLD SELECTION RULE AND THEORETICAL THRESHOLD SELECTION RULE IMPLEMENTATIONS OF THE CENTROID METHOD