A novel adaptive moving average method for signal denoising in strong noise background

The moving average (MA) method has been widely used in signal processing, but it has problems of the dead zone and the fixed window. In this paper, an adaptive moving average (AMA) filtering method is proposed, which can sniff the inherent characteristics of the signal and assign time-varying optimal parameters to signal processing, hence solve dead zone and the fixed window problems. Firstly, this paper builds the theoretical framework of AMA, including the trial steps and optimization of the necessary parameters. To verify the effectiveness of the AMA, three signal processing methods are taken as comparison methods to process the noisy simulation and experimental signals. Comparison methods includes the MA, variational mode decomposition (VMD), and wavelet threshold denoising (WTD). Signals processed include linear frequency modulation (LFM) simulation signal, aperiodic square wave (ASW) simulation signal, LFM experimental signal produced by a signal generator, and nondestructive test signal of wire rope. Also, the output is analyzed qualitatively and quantitatively with signal-to-noise ratio (SNR), cross-correlation coefficient, amplitude error, and a newly defined local coincidence index. Compared to MA, VMD and WTD, the proposed AMA can solve the dead zone problem, recover noisy signal with higher SNR, cross-correlation coefficient, and lower amplitude error. These results indicate that AMA is a promising method in signal processing.


Introduction
In signal processing, noise exists in signal acquisition and is unavoidable. Although noise can be beneficial in a few cases, such as enhancing a weak signal with stochastic resonance induced by noise [1][2][3][4] or noise reduction [5], noise is harmful to signal transmission and analysis for most cases [6,7]. So far, various methods have been proposed in signal processing [8][9][10][11], which are mainly divided into infinite impulse filter and finite impulse filter [12,13]. Among all kinds of infinite impulse filtering methods, Kalman filtering is the most widely used method [14,15]. And it has been improved greatly, such as extended Kalman filter [16], a e-mail: jianhuayang@cumt.edu.cn (corresponding author) unscented Kalman filter [17]. However, a finite impulse filter has more significant advantages in unbiasedness and aperiodicity than the infinite impulse filter. The finite impulse filter technology supports bounded input and bounded output [12,18], which are not available in infinite impulse filter [19]. In view of the advantages of finite impulse filter, it has been applied in various fields [20][21][22].
The moving average (MA) method, a typical finite impulse filter, can obviously weaken the noise and extract the signal trend. Owning to its trend extracting ability and simple calculation, MA is widely used in energy, stock market, and other fields. In the field of energy, Amir [23] assessed the effects of various intensities of the MA filter duration and turbulence on recording wind speeds. Yuan [24] combined the MA with least square support vector machine to forecast the wind power. Amjad [25] proposed a simple moving voltage average technique with a fixed step size direct control incremental conductance method, which improves the dynamic and steady-state performance of the photovoltaic system and the ability of the power generation system to adapt to changes in environmental conditions. In the stock market, Nguyen [26] applied the MA to the prediction of the stock market in Vietnam, and the result shows that this method can generate 19.8% excess return every year. Stefan [27] combined the MA with new classification based on artificial neural networks to improve the response ability of the stock trend prediction. Gustave [28] quantified the relationship between growth and nutrition intake through 7-day MA to help clinical students identify the growth mode caused by nutrition intervention. Also, many improvements have been made to overcome the shortcomings of the MA. Svetunkov [29] allowed selecting automatically length of the MA model and made the MA a more powerful instrument that can be used in practice for different purposes. Nakano [30] proposed a generalized exponential MA model in financial markets and attained high performances. Li [31] presented a maximum likelihood least squares based iterative algorithm for the bilinear systems with an autoregressive MA. However, to the best of our knowledge of the related research progress, two shortcomings of the MA have not been solved despite all the efforts. One is the dead zone problem, and the other one is the fixed window width problem [32]. The problem of the dead zone causes the distortion near the terminals of the output, and the fixed window width limits the adaptive performance of the MA. These problems prohibit it from getting a good recovery when the signal changes sharply.
To solve the above problems of the MA, this paper proposes an adaptive moving average (AMA) filter. In AMA, the window width for each data point changes according to the gradient of each data point. So, more noise can be removed, and more signal details can be retained. Also, to verify the effectiveness of the AMA, some signal processing methods including MA, variational mode decomposition (VMD) [33], and wavelet threshold denoising (WTD) [34] will be employed as the comparison methods. Based on some indicators, such as signal-to-noise ratio (SNR), cross-correlation coefficient, or other newly defined indicators, quantitative analysis of processing simulation and experimental signals is necessary.
The paper is organized as follows. Section 2 introduces the MA and its shortcomings. Section 3 builds the theoretical framework of AMA, including the implementation steps and necessary optimization of parameters. In Sect. 4, the effectiveness of the AMA is evaluated with MA, WTD, VMD as the comparison methods, signals processed here includes noisy simulation signal, experimental signal produced by a signal generator, nondestructive test signal of wire rope. Sections 5 and 6 give the discussions and conclusions of this paper, respectively.

Model of MA
The MA is often defined in previous studies as [35] x(k) where s is the input signal series, x is the output signal series, k is the time index, w is the window width. The number of one side data points m includes the left-side m and the rightside m, and the sum of left-side m and the right-side m equals w. Here, k 1, 2, 3, …, N, N is the total number of the signal series. In Fig. 1, the MA output signal at the point k is equal to the mean value of all the data points of the input signal in the interval [k − m, k + m]. This method is always effective for an infinite time index.
The time index is always finite in the actual sampling. There are not enough m points left in the left-side signal sequence when k is less than m. Similarly, there is not enough m points left in the right-side signal sequence when k is greater than N − m. This problem is called the dead zone problem [21].
To solve the dead zone problem, the number of the data points used for calculation in the dead zone needs to be further modified according to actual sampling. As is shown in Fig. 2, the left-side m equals k − 1 or the right-side m equals N − k when k is less than m or greater than N − m, respectively. Also, the left-side m should be equal to that of the right-side [21]. Hence the corresponding intervals of the data points used for the calculation are [1, 2k − 1] and [k − (N − k), N]. It is easy to find that the window width decreases when k is close to 1 or N, and the noise reduction ability of the MA decreases rapidly. Therefore, the MA cannot achieve a good recovery in the dead zone.
Furthermore, there is a fact that a large window width provides better noise reduction, but a small window width allows more transient information to be maintained. Unfortunately, the window width of the MA is constant. The fixed window width becomes too large when the gradient increases, and vice versa. Therefore, the above-mentioned problems prohibit MA impossible from achieving a best recovery at each point.

Disadvantages of MA
In Fig. 3a, u(t) is a harmonic pure signal with a sampling frequency of 10,000 Hz, a frequency of 10 Hz and an amplitude of 0.5. We add a Gaussian white noise to u(t) to obtain the noisy input signal s(t) with a SNR of − 10 dB. Letting m 200, x(t) is the MA output of s(t).    Also, x(t) has an obvious signal distortion in the dead zone. So far, we draw a conclusion that there is an obvious distortion at the endpoints and jumps when MA is used to process noisy signals, and the dead zone problem remains unsolved.

The theoretical framework of the AMA
Step 1 In the given discrete noisy time series s(k), k 1, 2, 3, …, N, N is the total number of s(k). To get a prior knowledge of the gradient at each point of the signal, employ the MA and set m 1 to default, such as m 1 50, which is the number of one side data points in calculation. However, the number of data points available on one side of the signal in the dead zone may be less than m 1 . To avoid this, all data points available in window should be calculated. So, we generate a matrix M a with dimension 2 × N according to Eq. 2, which saves the number of data points available for calculation at each data point of the signal. Specifically speaking, M a (1, k) and M a (2, k) represent the number of the left-side and right-side data points of s(k). M a is defined as Here, m 1 denotes the number of one side data points used in the calculation, M a expresses the available data number matrix.
As are shown in Fig. 5, M a (1, k) and M a (2, k) are not necessarily equal in the AMA, which is different from that of the MA.
According to M a , we calculate the MA output x a (k) for prior knowledge Step 2 According to Eq. 4, calculate the left and right gradients of each point in x a to obtain the 2 × N-dimensional gradient matrix G. Here, G (1, k) and G (2, k) represent the left and right gradient at point k, respectively, which are shown in Fig. 6. Step 3 Construct the m-adjustment function H As is shown in Fig. 7, H is a function regarding the m-adjustment base m 2 , the adjustment rate γ , and the gradient matrix G, e (•) is the exponential function.
For signals with similar waveforms and frequencies, m 2 and γ have similar optimal values. The optimizations of m 2 and γ are systematically explained in the following parts. Based on the optimal m 2 and γ , the gradient matrix G is the only variable of the function H, whose output is the best-m matrix M b .
Step 4 For a data point, an optimal m may be greater than the available m. So, it is necessary to compare the value of the corresponding elements in M a and M b , and the smaller value is chosen as the number of data points for calculation, thus all these values make up the chosen Here, M a , M b , M c is the available data number matrix, best-m matrix, and chosen m matrix, respectively. The symbol min(·) means the minimum.
Step 5 Calculate the output x 3.2 Parameter optimization of AMA

Optimization of m 2
To find an optimal value for m 2 , we generate an LFM signal with an amplitude of 0.5, and the sampling frequency of 10,000 Hz. Also, we add a Gaussian white noise into u(t) and get s(t) with the SNR of − 10 dB. To optimize the m 2 , set m 1 and γ to 200 and 0.02, respectively. Besides, optimization interval of m 2 ranges from 10 to 600, and the optimization step size is 10. Here, the indicator SNR is defined as SNR 10 log 10 P u P n (9) where P u and P n are signal power and noise power, respectively. Figure 8 shows the curve of SNR versus m 2 . The SNR reaches the maximum when m 2 125 and remains at a high level when m 2 is less than 215. Also, the AMA has window width attenuation function H, which guarantees an optimal neighborhood matrix M b when m 2 is greater than 125. Therefore, it is advisable to select a number larger than 125 as the neighborhood attenuation base m 2 , such as m 2 200.

Optimization of γ
Add Gaussian white noise into LFM signal u(t) to get s(t) with SNR of − 20 dB. m 1 , m 2 are both 200. γ increases from 0.01 to 0.5 with the step size of 0.01. Figure 9 shows the curve of the SNR versus γ . A relatively large SNR can be achieved when γ is between 0.03 and 0.11. Especially when γ 0.05, the SNR reaches the maximum 44.4 s. Of course, running time can be reduced when global computing takes larger step size, but the optimization precision will also be reduced.
The above results show that the parameter optimization method used in Sect. 3.2 may not guarantee the global optimal parameters, but the running time is greatly reduced. Also, the final output SNR is close to the optimal value. So, considering efficiency and accuracy, method used in Sect. 3.2 is better than global computing.

Validation of AMA
This section includes three parts. Part one is a comparison between MA and AMA on noisy LFM, ASW signal processing. Part two is a comparison between the effect of AMA and VMD on processing noisy ASW and noisy LFM signal produced by a signal generator. Part three is comparison among AMA, WTD, and VMD on processing nondestructive test signal of wire rope.

Compare with MA on processing noisy LFM and ASW signal
This section studies the improvement of the AMA compared to the MA in the aspect of signal de-noising, what is worth studying here is the ability of the AMA to solve the dead zone problem of the MA. Therefore, this section processes the LFM simulation signal and the ASW simulation signal with the above two methods under strong noise.
Add Gaussian white noise to an LFM and ASW simulation signals u(t) to get s(t) with a SNR of − 20 dB and − 10 dB, respectively. Plot u(t) and s(t) in Fig. 12. It can be seen from the figure that both the pure LFM simulation signal and the pure ASW simulation signal are completely submerged in strong noise, and the effective information of the original waveform cannot be obtained. The denoising results of the noisy LFM signal based on the MA and the AMA are presented in Fig. 13a and b, respectively. In Fig. 13a, the MA output has an obvious signal distortion in the dead zone at the end of the signal, where the output is obviously deviated from the pure signal. Differently and luckily, as is shown in Fig. 13b, the distortion mentioned above does not appear in the AMA output. Also, in this LFM signal with a positive frequency modulation rate of β 10 Hz/s, the transient frequency will increase with the increase of the pure signal duration, causing an obvious amplitude attenuation in the MA output. Especially after 0.5 s, the amplitude of the MA output attenuates to half of amplitude in pure signal. Even worse, this attenuation does not disappear at all and lasts to the end. However, the AMA output amplitude shows a relatively high stability and stable around 1, which is consistent with the theoretical amplitude of a pure signal. Figure 14a and b show the denoising results of the noisy ASW simulation signal with the MA and the AMA, respectively. Obviously, the AMA solves the dead zone problem, and its output has a steeper jump in jump zone. Further observation shows that the MA output in jump zone is linear but that of the AMA is nonlinear, which is more fitting to the pure signal. So, the AMA has a better signal recovery ability, higher adaptive ability, and can better realize the noise reduction under strong noise.
The above conclusion is the qualitative description of the MA and the AMA. To make a further quantitative analysis, the SNR and cross-correlation coefficient C ux are calculated, respectively. The C ux is defined as follows where u and x denote the mean value of u and x, respectively. Also, to compare the recovery ability of the MA and the AMA for the signal jump, a new local coincidence index S ux is defined as where p and q are the starting and ending indexes of the jump area, respectively. p, q 1, 2, 3…N. Symbols max(·) and min(·) denote the maximum and minimum of the function, respectively. x(p: q) contains the elements from the p-th one to the q-th one in the time series of x. Table 1 lists the values of SNR, C ux and S ux of the MA output and the AMA output. As is shown in the Table 1, AMA can obtain higher SNR and C ux when process the LFM signal and the ASW signal. Which indicates that the AMA has stronger adaptability and noise reduction capability. Also, the AMA can also obtain a better output with S ux of 0.8821 when process the ASW signal, which is greater than that of the MA. It indicates that the AMA has better ability to recover signal jump than the MA, and the AMA can track signal waveform better when the signal changes.

Compare to VMD on processing noisy simulation and experimental signals
To further prove the noise reduction ability of the AMA, comparison with other noise reduction algorithms is necessary. Among these noise reduction methods, the EMD has gained a lot of research. However, to our research work, the EMD fails to do the signal recovery test under strong noise, but the VMD can make it. Therefore, the VMD is employed as a comparison method in this paper. Here, the simulation signal used for testing includes the LFM signal, the ASW signal, and an experimental signal collected from an actual circuit. Figure 15 shows the VMD and the AMA recovery of the simulation signals under strong noise. In Fig. 15, the VMD output contains more noise while the AMA output contains less noise, which is closer to the pure signal.
In view of the output of the VMD that too much noise is contained in the recovery, it is considered that the noise is too strong, hence the research on the performance of the VMD Fig. 16 The curves of the C ux versus the SNR and the AMA under different noise intensity. First, add different intensity of noise into pure signals and get the simulation signals with SNR of − 20 dB to 20 dB. Then, process each group of simulation signals with the AMA and the VMD, respectively, and calculate C ux of output and pure signal. Figure 16 illustrates the curves of C ux versus SNR. In Fig. 16, the VMD and the AMA can get good outputs when the SNR is higher than 0 dB. As the noise intensity increases and the SNR decreases, the C ux of the VMD and the AMA declines. When SNR reaches − 20 dB, the C ux of the VMD output drops to 0.35, while the AMA can still get a better output with C ux 0.7. This means that AMA can recover the pure signal better than the VMD when processing the actual circuit signal under strong noise.
In Fig. 17, an LFM signal produced by an actual circuit is used for comparison between the AMA and the VMD. As is shown, the signal is generated by a Tektronix AFG3022C signal generator and collected by a RIGOL DS1072U oscilloscope. The sampling frequency is 21,850 Hz, the signal duration is 1 s, the signal amplitude A 0.25, the initial frequency f 10 Hz, the frequency modulation rate β 10 Hz/s, and the SNR − 20 dB. Figure 18a shows the experimental LFM signal under strong noise and Fig. 18b and c show the outputs processed by the VMD and the AMA, respectively. In the figure, the VMD output contains too much noise, while the AMA output contains less noise. And the amplitude of AMA output is close to the true amplitude of 0.25. So, the AMA is more suitable for the experimental LFM signal processing, and it can achieve a better signal recovery under strong noise.

Compare with WTD and VMD on processing nondestructive test signal of wire rope
To verify the capability of the proposed AMA in processing engineering signals, this section takes VMD and WTD as the comparison methods to process wire rope signal. The signal is collected with Y66 portable steel wire rope flaw detector from model 6 × 37S-FC steel wire rope, which has served in Anshan iron mine of Angang mining company. Figure 19 shows the test bench.
WTD is widely used in processing nondestructive test signal of wire rope [36][37][38], but different wavelet basis has different denoising effects [39]. Here, db5 wavelet basis is chosen for its excellent noise reduction effect in wire rope signal processing [40].  Figure 20 shows the noisy signal, WTD output, VMD output and AMA output. In (a), noisy signal contains strong noise with the SNR of − 10 dB. In (c), VMD fails to eliminate most noise. Compared to WTD output in (b), AMA in (d) has smaller waveform disturbance at the peak and purer output at the stable.
To quantify the difference of AMA, WTD, and VMD in wire rope signal processing, the amplitude error (AE) index is defined in Eq. 12.
Here, |·| denotes the absolute value. Nondestructive test signal of wire rope with SNR from − 20 to 0 dB are processed by WTD, VMD, and AMA. Figure 21 shows the indexes versus the SNR of input. Figure 21a indicates that AMA output has the highest SNR when processing the signal with SNR lower than 10 dB. Figure 21b shows that AMA output can always keep C > 70% when SNR is lower than 0 dB, which is higher than that of WTD and VMD. Figure 21c shows that compared with WTD and VMD, AMA has lower AE when SNR is lower than − 5 dB, which is always kept less than 10%. In brief, compared with WTD and VMD, the AMA output can obtain higher SNR, C, and lower AE, especially under strong noise.

Discussions
The m-adjustment function is constructed in Eq. 5, where exponential model is directly used. The purpose of which is to adaptively adjust the window width according to the signal gradient everywhere, and greatly reduce the noise and retain the signal. Given the above consideration, use smaller window width when the signal gradient is large and vice versa. In fact, the m-adjustment function is not limited to the exponential model proposed in Eq. 5, all other models that can achieve the above purpose can be adopted. Different models have different noise reduction effects. So, it is also a meaningful work to propose new models and compare the noise reduction effects, more better models are believed to be proposed in future research.
Illustrated in Figs. 8 and 9, the parameter optimization part shows the existence of optimal m 2 , γ . Based on which, AMA can achieve the best noise reduction effect. However, when processing other signals, these optimal parameters may not ensure optimal output. Specially, these optimized parameters are universal for the same type of signal. Specifically, for signals with similar scales and waveforms, such as signals collected by the same equipment at different times, AMA with same optimal parameters can provide similar relatively optimal outputs. That means such parameter optimization is not always necessary for similar data, and optimal parameters can be shared when dealing with similar data sequences. Therefore, much time can be saved.
In Sect. 3.3, parameter optimization used in Sect. 3.2 is proved to miss the global optimum, and the reasons for this are complex. Including artificially increasing m 2 from 125 to 200 for a larger optimal neighborhood matrix M b . Also, optimization of m 2 and γ are performed only once in Sect. 3.2. Final optimization result is believed to be closer to the global optimum if more iterative optimization could be performed, but the running time will increase. So, when considering the efficiency and accuracy, method used in Sect. 3.2 is more balanced than global computing.

Conclusions
In this paper, a new algorithm named AMA is proposed, which adjusts the window width to optimal according to the gradient of each data point in signal sequence. With SNR as the index, parameters optimization improves the noise reduction effect of the AMA. In validation part, several methods including MA, WTD, and VMD are taken as comparison methods to process simulation signal, experimental signal produced by a signal generator, and nondestructive test signal of wire rope. Some analysis results are summarized as follows: (1) The AMA can effectively solve the dead zone problem that outliers exist at signal endpoints, which is unavoidable for the MA. (2) AMA can adapt to signal frequency fluctuation better than MA, and recovery signal with lower amplitude attenuation. (3) Compared to the VMD and WTD, AMA can provide better output with higher SNR, C, and lower AE, especially under strong noise background.