On Predictive Coding for Erasure Channels Using a Kalman Framework

We present a new design method for robust low-delay coding of autoregressive (AR) sources for transmission across erasure channels. It is a fundamental rethinking of existing concepts. It considers the encoder a mechanism that produces signal measurements from which the decoder estimates the original signal. The method is based on linear predictive coding and Kalman estimation at the decoder. We employ a novel encoder state-space representation with a linear quantization noise model. The encoder is represented by the Kalman measurement at the decoder. The presented method designs the encoder and decoder offline through an iterative algorithm based on closed-form minimization of the trace of the decoder state error covariance. The design method is shown to provide considerable performance gains, when the transmitted quantized prediction errors are subject to loss, in terms of signal-to-noise ratio (SNR) compared to the same coding framework optimized for no loss. The design method applies to stationary auto-regressive sources of any order. We demonstrate the method in a framework based on a generalized differential pulse code modulation (DPCM) encoder. The presented principles can be applied to more complicated coding systems that incorporate predictive coding as well.


I. INTRODUCTION
I N transmission of real-time signals data losses are typically an unavoidable impairment.The real-time constraint makes it necessary to consider data with a high transmission delay lost.This delay can for example occur as a result of network congestion.On other types of lossy channels such as wireless links, the real-time constraint makes it impractical to retransmit lost data.Transmission can be protected against losses by, e.g., error correcting codes or Multiple Description Coding (MDC), or the effects of losses on the transmitted signal may be mitigated through various loss concealment techniques at the receiver [1]- [3].For low-delay coding applications, errorcorrecting codes are impractical due to the delay they impose.In such cases, another possibility is to modify the source coding itself to increase robustness against losses.
Linear Predictive Coding (LPC) has been widely used for source coding, especially speech coding, for a long time.
It is one of several source coding techniques in standards used in Voice over IP (VoIP) and is widely used in several mobile phone standards [4]- [10].LPC works well for signals with temporal correlation where it exploits this correlation to compress the source signal.In a typical LPC source coding system, the predictor in the encoder is an all-zero filter that ideally assumes that the source signal is the outcome of an auto-regressive (AR) process, in which case the predictor can perfectly whiten the source signal.In the decoder, the source signal is reconstructed from the whitened prediction residual.
The predictor in an LPC source coding system is typically determined by modeling the source signal as the outcome of an AR process for which the coefficients are estimated.The predictor can be chosen to match these estimated AR coefficients, i.e., the coefficients of the prediction filter are equal to the coefficients of the source AR process.This is in general not optimal when the prediction residual is affected by noise, e.g., quantization or channel noise, and better performance can be achieved with a mis-matched predictor [11], [12].
Differential Pulse Code Modulation (DPCM) is an example of a predictive source coding scheme which includes feedback of quantization noise in the coding of the source signal [13].Kalman filtering can be applied in predictive coding to provide Minimum Mean-Squared Error (MMSE) estimation of the source signal.Previous applications of Kalman filtering to predictive coding employ Kalman filters at both the encoder and decoder and transmit quantized Kalman innovations from encoder to decoder [14]- [18].
The effect of channel errors on DPCM performance has been investigated for transmission across ATM networks in [19].The authors investigate optimization of the predictor for channel losses in a first order DPCM system, but provide no optimization results for higher order coding systems.
When considering a Kalman filter-based decoder, the work in [16], [20] applies to optimizing the Kalman filter for given noise statistics by selecting the optimal measurement vector that minimizes some measure on the a posteriori state error covariance.However, this approach does not take channel losses into account.
The handling of lost measurements in a Kalman estimator is investigated thoroughly in [21], [22], but this work does not consider optimization of a coding system for such losses.
An approach for optimization of a predictive quantization scheme employing Kalman-like filters at encoder and decoder is presented in [23], [24] where channel losses are modeled by a Markov model.[24] is contemporaneous work with a different philosophy; it presents an optimization method based on Jump Linear System (JLS) modeling and Linear Matrix Inequality (LMI)-constrained convex optimization to design fixed gains for the encoder and decoder filters for each channel state.This approach reduces computational complexity by restricting the decoder to account only for present channel loss through a JLS-based decoder that switches between the states of the channel loss model.
In this paper, we present a novel optimization method for the design of low-delay predictive coding systems, demonstrating a method for designing a robust encoder and decoder for given loss statistics.In particular, we examine DPCM, which is a canonical method of predictive coding which captures the basic problems of real-time transmission over channels with packet loss.In contrast to other efforts to design robust DPCM methods (e.g., [19], [24]), we consider a generalized DPCM encoder structure with separate prediction and noise feedback filters, an encoding structure commonly employed in speech coding.Moreover, we consider the case where these encoder filters are fixed time-invariant filters, leading to lowcomplexity quantization of signal samples.This encoder transmits quantization information (related to quantized prediction errors) that is subject to packet loss/erasure.The decoder views the received information from the encoder as noisy signal measurements, and utilizes Kalman filtering principles to perform MMSE estimation of the signal.This approach of viewing the encoder as producing noisy measurements is in contrast to previous approaches in [14]- [18], in which the encoder's transmitted quantized prediction error is viewed as the innovation, with both the encoder and decoder running synchronized Kalman filters.
Our predictive coding scheme consists of both offline and online stages.In the offline stage, the fixed encoder filters, and the initial Kalman measurement filter at the decoder are jointly designed, taking into account both the quantization noise and packet loss statistics.In the online operation, the decoder's Kalman filter parameters are updated with each received or lost packet, taking into account the particular loss outcome sequence in the MMSE estimation.Since the encoder remains fixed while the decoder is time-varying, synchronization between encoder and decoder is not assumed.Simulation results demonstrate the efficacy of the proposed method.This low-delay predictive coding design approach can be extended beyond DPCM and to the robust transmission of vector data, such as Line Spectral Frequencies.Therefore, this paper presents a re-thinking of fundamental concepts, and presents a new design method that can be employed in different coding application contexts.
The system model used to illustrate the application of our method and the actual design method are presented in Section II.Section III contains descriptions of simulations conducted to evaluate the performance of the method and results of the simulations showing substantial improvements of the presented method over coding without optimization for loss.Finally, Section IV discusses the implications of the proposed method and the simulation results. is treated in Section II-B4.The coding framework is summarized in Section II-D.We provide an overview comparison to the method from [24] in Section II-E.

A. Source Encoder
The source encoder chosen to illustrate the application of our design method is based on generalized DPCM coding.We consider an encoder with the noise shaping structure illustrated in Fig. 1 as given in [25], [26].Fig. 1 includes the quantization noise model described in Section II-A1.
The encoder codes the source signal s.The source signal is modeled as outcomes of a stationary AR process of order N , driven by zero-mean stationary white Gaussian noise r; the α i are the source AR coefficients, defining the source process together with N .The encoder has prediction filter P independent from the quantization noise feedback filter F .The structure depicted in Fig. 1 is equivalent to a classic DPCM encoder, as described in e.g.[13], when P (z) = F (z) and ρ = 1.
The input to the quantizer, d n , is given by Note that e n is the prediction error, qn is the filtered quantization noise feedback, d n is the input to the quantizer, and qn is the quantization error; p is the predictor order; a i , i = 1, . . ., p are the predictor coefficients; f is the noise feedback filter order and b i , i = 1, . . ., f are the noise feedback filter coefficients.In this work, p = f = N .1) Linear Quantizer Model: As depicted in Fig. 1, the output transmitted to the decoder is quantization indices, j, for the quantized prediction error, z n , in (5).As explained in Section II-B, z n is seen as the Kalman measurement by the decoder, We use scalar quantization Q(•) with a gain-plus-additivenoise model [13].The model accommodates correlation between quantizer input and quantization noise.In fact, where ρ ∈ [0, 1], and q n is a stationary zero-mean white Gaussian noise, independent of d n , with variance where the quantization noise is modeled with a variance proportional, by a constant k, to the variance of the input to the quantizer, d n .The assumption of white Gaussian q n is a simplifying assumption in the sense that quantization noise is generally not Gaussian and only approximately white under high-rate assumptions [27], [28].In order to be able to model the quantization noise as measurement noise in the Kalman filter in the decoder, this noise must be white Gaussian.
There are techniques for handling non-Gaussian measurement noise in a Kalman filter, see [29]- [31], alternatively, a nonlinear measurement model could be accommodated by the Extended Kalman Filter (EKF) [32].Nevertheless, we retain the Gaussian assumption in order to keep the Kalman filter of the standard form.This is to avoid non-linear modifications to the covariance updates of the Kalman filter and facilitate the inclusion of the noise model in the optimization approach presented in Section II-B4.Note that ρ and k are given by the coding loss, β, of the quantizer: Note that β is the inverse of the quantizer coding gain [28].
The noise incurred by quantization is In order to simplify the calculation of quantizer input variance in the optimization of the encoder, we wish to feed back a white noise component.Therefore, d n is scaled by ρ in the quantization noise feedback to de-correlate the noise feedback from d n .Thus, we only feed back the uncorrelated part of the quantization noise q n = z n −ρd n .This allows us to model the input to the quantizer, d n , as white Gaussian which simplifies the optimization of the encoder.The design of quantizers for use in the encoder is treated in Section II-C.

B. Kalman Filter-Based Decoder
The decoder is based on Kalman filtering, i.e., MMSE estimation of the source signal s.The Kalman filter at the decoder estimates the source signal based on measurements, z, reconstructed from the received quantization indices, j, which may be subject to losses.In order to derive the Kalman estimator ŝn of s n , the source process and encoder equations are modeled by a state space model of the form given in, e.g., [32].The state transition equation is chosen to represent the evolution of the source signal s n as well as the states of the encoder filters P and F .The measurement equation represents the filtering and quantization operations of the encoder.So, the measurements become the quantized prediction error outputs from the encoder.Note that where our formulation leads to the quantized prediction error being seen as a Kalman measurement, previous formulations of Kalman predictive coding have mapped this quantity to the Kalman innovation [14]- [18].This difference is instrumental for obtaining the robustness to packet loss which we will demonstrate in this paper.
The decoder is derived from the state-space model described below.The process equation is given by ( 10) and the measurement equation by (11).
The state x n corresponds to the joint states of the signal, predictor and noise feedback filter.
The state transition matrix F is defined as follows (the subscripts in (13) denote the dimensions of the individual components): where are the coefficients of the source AR process, I x is an x × x identity matrix, and 0 is an all-zero matrix with the specified dimensions.Thus, the top-left part of F represents the AR filtering of the process noise, given by ( 1), generating the source signal, and shifts past source signal samples through the state.The bottom-right part of F delays previous quantization noise samples through the state.
The process noise w n is defined as which is a stationary zero-mean white Gaussian process noise with Notice that the first component of the process noise, r n , models the source signal excitation and the second component, q n , models the quantization noise fed back to the filter F .Clearly, the definition of the process noise (14) introduces correlation between the process noise w n and the measurement noise in the form of q n in (11).The connection between quantization noise in the state originating from w n , and q n added to the measurement is captured by including correlation between process and measurement noise as follows: where R = σ 2 q .As shown later, we use a formulation of the Kalman filter which takes the covariance S into account.Let G be a transform to allow the process noise w n to be defined in a compact form with G given by The measurement vector h represents the filtering operations of the encoder as well as the scaling in the model of the quantizer where h contains the coefficients of the prediction error and noise feedback filters such that by (2), whereby (11) follows from (6).To summarize, h represents the filtering in the encoder before quantization.Due to the quantization noise model presented in Section II-A1, h represents the filtering after quantization and produces the measurements seen by the decoder when these are not lost.
The state-space model ( 10) and ( 11) represents the production of the source signal as well as the encoding of it.This state space model forms the basis of the decoders described in the following sections.First we describe the decoder and the design algorithm for the lossless case.Subsequently, we extend the principles to losses.
1) Lossless Transmission: The decoder receives information (quantization indices j) to build measurements z n from the encoder.In the case of lossless transmission, all measurements are received by the decoder.The decoder in this case is given by the Kalman filter with correlated process and measurement noise for the described state space model, (10) and ( 11), given in for example [32] x− n+1 = F xn + GSR −1 z n (23) in which the following shorthand notation is used: The decoded source signal ŝn is given as the first element of xn according to (12).Since the source signal, s , is stationary and the encoder fixed (constant h), the Kalman filter statistics will converge to fixed values as n → ∞: Correspondingly, we may write the fixed Kalman filter decoder as in ( 26)- (28).
where P − is the solution to the Riccati equation 2) Coder Design for Lossless Transmission: Our optimization of the coding framework is somewhat similar to the approach in [16], but [16] does not consider optimization for sample erasures and the optimization objective has a different structure.The design of the encoder and decoder consists of offline selection of the measurement vector h, as this defines the encoder through (18) and (19) and defines the decoder through ( 26)- (28).The method is based on choosing the measurement vector h * to minimize the mean squared error of the state estimate xn at the decoder, at time n given the a priori state estimate x− n and corresponding state error covariance P − .If we first look at the situation for the lossless case, described in Section II-B1, the objective is which can be written as with xn , x− n , and P − given by ( 26)-( 28), and Note that h * and P − , and thereby P, will depend on each other through ( 28) and (30): having selected a h * (1) according 1 to (30) for some P − (1) , this will yield a new P − (2) by ( 28) for h = h * (1) , again resulting in a new h * (2) by (30).Therefore, we use an iterative approach, iterating over (28), (30) and (31) starting from some initial h (0) and P − (1) (to be explained in the following), iterating until convergence.In the following, the index i identifies the iteration number.
Similar to [16], we express the measurement noise covariance-or equivalently, quantization noise variance-R (i) as a function of h (i) .Because our framework models the encoding as the Kalman measurement and quantization noise as the measurement noise, R (i) is expressed as follows, cf.Fig. 1 and (2): where R xx,(i) is the state correlation matrix, which has the following structure Equations ( 32) and ( 33) allow reformulation of (31) as The minimization stated in (30) of the trace of (34) can be attained by maximizing the trace of its right-most term.
where P − (i) Quantization noise is now taken into account in (35), through (32).Equation ( 35) may be rewritten as a Rayleigh quotient through a Cholesky factorization of the matrix in the denominator where L is a lower triangular matrix.We define y = L T x such that The vector y * (i) maximizing the right-hand side of (36), i.e., is given as the eigenvector of L −1 P − (i) 2 L −T corresponding to its largest eigenvalue [16].Clearly, the fractions in (35) and (37) are invariant to scaling of x or y, equivalently.As a result, we may take the measurement vector as given by (37) with a normalization by the first element of the vector in order to keep h (i) as formulated in (19), with its first element equal to 1. Then where c is the first element of the vector L −T y * (i) .Having selected h * (i) , P − is updated according to (28): where iteration indices have been omitted on the quantities F and Q to simplify the equation.These quantities are however dependent on h * (i) .The encoder and decoder are designed by iteratively performing the steps given by ( 37)- (39).The algorithm is initiated with initial measurement vector h (0) set to match the source and P (0) set to the unique stabilizing solution to (28) for h = h (0) .The algorithm is outlined in Table I. 3) Lossy Transmission: Considering the situation where measurements z n may be lost, we have a time-varying Kalman filter.The measurement vector h n and measurement noise covariance R n are time-varying.This models the possible loss of measurements at the decoder.Specifically, we substitute h and R in ( 21)-( 24) by where γ n are outcomes of a stationary Bernoulli random process modeling measurement arrival with arrival probability Pr{γ n = 1} = γ and loss probability Note that R is the measurement noise covariance in the case of no loss and σ 2 I is the measurement noise covariance in the case of loss.We let σ 2 → ∞ in (41), representing infinite uncertainty about the measurement z n at the decoder when it is lost in transmission.See [21], [33] for other examples of this approach.Replacing R and h by ( 40) and ( 41) in ( 21)- (24) and taking lim , we obtain the equations defining the online filtering operation where As in the lossless case, the decoded source signal ŝn is given as the first element of xn according to (12).The important difference between the lossless case and the lossy case is that the decoder equations now depend on sample arrival γ n .Furthermore, one cannot rely on fixed P − and P in the lossy case since these become stochastic through their dependence on γ n .4) Coder Design for Lossy Transmission: Extending the design method from Section II-B2 to the decoder for the lossy case described in Section II-B3, we could consider the objective to obtain a h * n at each time step n optimized for all arrivals γ 0 . . .γ n .Hereby we would minimize the trace of (43) rewritten via (32) and (33) as Since this optimization would minimize the trace of (47), we can see that the optimization is not defined at loss events, i.e., at n for which γ n = 0. P n is independent of h in the event of a loss, since P n = P − n in this case.Furthermore, since h defines both the encoder and decoder, (46) would require the encoder to know γ n which in turn requires instantaneous loss-less feedback of this information from decoder to encoder.
Instead, we seek a method that allows offline calculation of a constant h * , given the statistics of loss.So, the goal is a method that improves decoding performance under average loss conditions rather than the specific loss outcomes.In contrast to the usual Kalman filter, P n is stochastic due to measurement losses γ n .We propose the following offline method for designing measurement vectors for improved performance under sample losses.Ideally, it would be desirable to obtain a h * that at each n minimizes the expectation of P n with respect to all γ k , k = 0, . . .n, i.e., E γ0...γn {P n }.However, it is not possible to directly calculate this expectation, a fact which is also pointed out in [21].We use a simplified approach where the philosophy is to obtain a h * that minimizes the ensemble average of P n over γ.
The method is a modification of the design for lossless transmission presented in Section II-B2.At each iteration i, h (i) is selected to minimize the trace of E γ P (i) , the ensemble average of P (i) , (43), with respect to γ (the measurement loss process).This requires the arrival probability γ to be known in order to design the encoder and decoder.P − (i) is updated according to the discrete-time Riccati equation, [32, p. 108], of the decoder Kalman filter, adapted for measurement losses We take the ensemble average of (48) with respect to γ . (49) The measurement vector is selected at each iteration according which now takes both quantization noise and loss of measurements into account.Equation (50) may be rewritten as a Rayleigh quotient by the same approach as in Section II-B2, cf.(36).We define y = L T x similar to Section II-B2, replacing P − (i) by is given as the eigenvector of 2 L −T corresponding to its largest eigenvalue.As in Section II-B2, the measurement vector is calculated as where c is the first element of the vector L −T y * (i) .Equations ( 49), ( 51) and ( 52) are iterated until convergence of (49), upon which the resulting h * (i) is chosen as fixed measurement vector h * for the decoder given by ( 42)-( 45) and the corresponding h * for the encoder.The optimization method is summarized in Table II.

C. Quantizer Design
In general, it is not a trivial matter to design a quantizer for a predictive quantization system.The optimal quantizer depends on the encoder filters, and the encoder filters depend on the quantization noise.Therefore, existing approaches proceed by iteratively optimizing the filters and the quantizer in turns.The optimum design of quantizers for predictive quantization schemes has been treated in the literature, e.g., in [34].In this paper, we concentrate on the optimization of the encoder and decoder filters to improve decoding performance with respect to sample losses.The impact of quantization in the decoder plays a secondary role compared to the loss of transmitted data.Therefore, we choose a simpler suboptimal approach to the design of quantizers.
As the encoder (and decoder) is designed for a specific loss probability by changing the encoder filters, P and F , accordingly, the statistics of the input to the quantizer, d n , generally vary with the loss probability.Therefore, the quantizer should also be adapted for the specific loss probability in order to be appropriately loaded.
The source process is an AR process driven by zero-mean white Gaussian noise.As seen from ( 1), the source signal is a sum of Gaussian random variables and so, is Gaussian.The prediction filter output e n is zero-mean Gaussian by the same argument.Since the prediction filter in general does not match the source (generally a i = α i , ∀i), the prediction residual is not white.According to the quantization noise model presented in Section II-A1, the noise, qn , fed back to the quantizer input is also zero-mean Gaussian.Under the model assumptions, the input, d n , to the quantizer is thus zero-mean Gaussian.
The quantizer in the encoder is designed based on the statistics of the input in this case, the Gaussian p.d.f. with zero mean and variance calculated as follows.Equation (32) in Section II-B2 states the quantization noise variance for the time-varying case used in the optimization algorithm.In the coding framework, for a fixed measurement vector h, all signals in the encoder are stationary and so, (32) reduces to (53), i.e., where From ( 18), ( 19), ( 53) and (54) we can calculate the quantizer input variance as follows: We have considered both Lloyd-Max and uniform quantization for the coding framework presented in this paper.
Lloyd-Max quantizers can be designed to match a specific input p.d.f.using the "Lloyd II" algorithm [13].Lloyd's and Max's original quantizers for Gaussian input can be found in [35], [36] and scaled according to input variance.
Uniform quantizers can be designed to match a Gaussian input p.d.f.using the expression for the step size in [37].
The coding loss, β, and corresponding parameters, ρ and k, are estimated empirically for the quantizer.These parameters are independent of quantizer scaling, provided that the quantizer is optimally loaded for the given input, and only depend on the quantizer type, uniform or Lloyd-Max, and resolution.So β is estimated as follows: 1) Design a quantizer, Q(x), (Lloyd-Max or uniform) with given precision for a unit-variance zero-mean Gaussian distribution, f X (x).2) Generate a random sequence of data, x, according to the distribution f X (x).
The estimate β for a particular quantizer (type and resolution) is used as β in the calculation of quantization model parameters k and ρ in the encoder and decoder presented in Sections II-A and II-B.

D. Summary of Coding Framework
For transmission across erasure channels, the framework presented in Sections II-A and II-B operates as follows: • It is assumed that both the encoder and the decoder know the source signal model {α i , i = 1, . . ., N }, var {r}, and channel arrival probability γ.
• The encoder and decoder parameters in the form of h are designed according to the method in Sections II-B2 and II-B4 in case of lossless transmission.This is done offline in both encoder and decoder, respectively.
• The encoder filter parameters a i and b i , i = 1, . . ., N are obtained from the designed h * by ( 18) and ( 19).
• The quantizer Q(•) is designed as outlined in Section II-C.
• The encoder codes the source signal according to ( 2)-( 4) and transmits quantization indices, j, for reconstruction of z n at the decoder, which requires the decoder to know the quantizer codebook.
• The decoder receives quantization indices from the encoder with a probability of γ (equal to 1 in case of lossless transmission) and decodes the source signal depending on whether the current index was lost or not (γ n = 0 / γ n = 1), using h * in (42)-( 45) (( 26)-( 28) in case of lossless transmission.)

E. Comparison to Related Method
As mentioned in the introduction, [24] presents a method for robust predictive quantization.This section presents an overview comparison illustrating important differences between [24] and our proposed method.We shall refer to our method as Iterative Measurement Vector Improvement (IMVI) and the method in [24] as Gain Vector Search (GVS).
The encoders are Linear Time-Invariant (LTI) systems, both in the case of GVS and IMVI, whereas the decoders are generally time-varying.The decoder in GVS varies as a JLS according to the state of the Markov loss model, with a fixed set of encoder/decoder gains for each state.Thus, the decoding in GVS only depends on the current state of loss.Our decoder in IMVI varies both according to sample loss, γ n , as well as the time-varying Kalman filter statistics, P − n and P n .The Kalman filter statistics encompass the effects of all previous losses, so the decoding in IMVI at time n depends on loss at time n as well as all previous losses.
The GVS method works by optimizing two different Kalman-like filters, at the encoder and decoder respectively.Both the encoder and decoder filters are identical in structure to a Kalman filter, but the filter gains are not calculated in the same manner as in Kalman filters.Our IMVI method employs an actual Kalman filter, but only at the decoder.The encoder relies on fixed Finite Impulse Response (FIR) filters.The IMVI method applied to the coding framework in this paper is based on an encoder structure in which the filtering of quantized prediction errors has been split into separate prediction error and quantization noise feedback parts.This offers a higher degree of freedom in encoder design.
The GVS framework accommodates auto-regressive moving average (ARMA) source models, while our model is restricted to accommodate AR source models in its current form.
The GVS framework uses a general Markov loss model, whereas our model only explicitly accommodates i.i.d.losses.Other types of (non-i.i.d.) losses such as Gilbert-Elliot loss models can be handled in our IMVI framework in terms of overall loss probability.

A. Simulations
Simulations have been conducted to evaluate the performance of the optimization method proposed in Section II-B4.
This paper has supplementary downloadable material available at the web page [38], provided by the authors.This includes all Matlab code necessary to fully reproduce the simulation results in this section.
• For testing IMVI, stationary random source signals were generated from AR processes of different orders.Sample arrivals γ n were simulated as outcomes of a Bernoulli random process over a series of loss probabilities γ ∈ [0, 1] and applied to the transmitted encoder quantization indices j.The generated source signals were encoded with encoder and decoder designed for each specified loss probability (γ in (49)).The quantization indices with losses were decoded using the Kalman decoder given by ( 42)-(45).
• As a baseline for comparison for IMVI, source signals were generated in the same manner as for IMVI above.
The generated source signals were encoded with encoder and decoder designed for no loss (γ = 0).The quantization indices, j, subject to the same losses as above for IMVI were decoded using the Kalman decoder given by ( 42)-(45).We shall denote this baseline method "Baseline".The simulations have been conducted for both uniform and Lloyd-Max quantizers at 2, 3, and 4 bits/sample, respectively.
Test data were generated from statistical models estimated from signals encountered in speech: AR coefficients were estimated from 20ms sub-sequences selected from voice-active regions of speech found in [39].The coefficients represent sequences with both low-pass, band-pass, and high-pass spectral shapes.For each of the examples we have plotted the power spectrum of the source AR process in Fig. 2.
Decoded signal Signal-to-Noise Ratio (SNR) is compared for IMVI and Baseline.In the following, we present results of the simulations described in Section III-A.We present data   The decoded signal SNRs are plotted in Fig. 3a, 3b, 4a and 4b for the simulated range of loss probabilities at quantization rates of 2, 3, and 4 bits/sample.Baseline-{2,3,4} and IMVI-{2,3,4} respectively.Examples I and II have been produced with Lloyd-Max quantization.Examples III and IV have been produced with uniform quantization.The source processes of the examples are of third, fifth, ninth, and tenth order, respectively.

B. Numerical Examples
The source AR process for Example I is a third-order process with the following parameters: α 1 = 1.6898, α 2 = −0.7865,α 3 = −0.0035.The parameters correspond to the matched prediction parameters shown in Table III (with F = P ).The filter parameters for Baseline-2 are shown in IV.The filter parameters designed for the specific loss rates at 2 bits/sample quantization (IMVI-2) are shown in V.The accompanying quantizer parameters are listed in Table VI; the quantizer parameters for Baseline-2 are the parameters for γ = 0 at all loss probabilities.Parameters for the remaining cases of Example I (Baseline-3, -4 and IMVI-3, -4) as well as for Examples II-IV have been omitted to save space.For the remaining examples, we show the decoded signal SNRs in Fig. 3a, 3b, 4a and 4b.

C. Summary
The examples show substantial improvements in decoded signal SNR under sample erasure conditions.For all examples, the improvement is rather modest at low loss probability, especially at the lowest quantization rate (2 bits/sample), improving for higher quantization rates (3 and 4 bits/sample).For higher loss rates, the improvement in decoded signal SNR is substantial.The maximum decoded signal SNR observed in examples I-IV are shown in Table VII.
At 2 bits/sample, IMVI demonstrates a maximum improvement in decoded signal SNR in the range 1.7 to 3.7 dB, at 3 The examples I-IV demonstrate that IMVI is capable of substantially improving the coding performance of the coding framework presented in Sections II-A and II-B.The method consistently improves performance for different AR source signal models and quantizers.

IV. CONCLUDING REMARKS
We have presented a novel method for optimization of predictive quantization of AR signals for transmission over channels with sample erasures.An important contribution of the presented method is a coding framework "design philosophy" that considers the encoding a process that produces noisy measurements of the source signal, in Kalman estimation's understanding of the term.The decoding is viewed as optimal estimation of the source signal based on these measurements.
The proposed method, IMVI, provides offline design of the encoder and decoder for optimal estimation by a Kalman filter at the decoder.By taking channel erasures into account in minimizing the trace of the Kalman state error covariance, we have obtained a design method that allows selection of encoder and decoder parameters which improve robustness to losses and provides MMSE estimation given the actual channel losses at the decoder.
As mentioned in the introduction, earlier applications of Kalman filtering in source coding, [14]- [18], have employed Kalman filtering at both the encoder and decoder and have not specifically considered transmission loss.Our method employs a Kalman filter at the decoder.The encoder relies on fixed FIR filters.This has the advantage of keeping the encoder simple, which could be a simple sensor node limited in power consumption and/or computation power, while providing MMSE estimation at the decoder, which could be a centralized controller or monitoring node without such restrictions in power consumption or lacking computational power.
In this paper, IMVI is demonstrated by application to a generalized DPCM encoder structure in which the filtering of quantized prediction errors has been split into separate prediction error and quantization noise feedback parts.This offers a higher degree of freedom in encoder design than an encoder more along the lines of classic DPCM with only a single filter.This higher degree of freedom may provide additional gains over single-filter encoders.
IMVI is limited to AR source signal models in the current framework.We believe it is feasible to extend the current model to more general ARMA source signal models, making the framework more versatile.This is a topic of future investigation.
IMVI has been demonstrated to improve decoded signal SNR substantially under sample erasure conditions for a diverse selection of source signal models.Furthermore, the improvements are demonstrated consistently for several different model orders and quantization parameters.

Fig. 1 .
Fig. 1.Generalized DPCM source encoder model with AWN quantizer model and de-correlated quantization noise feedback.

Fig. 2 .
Fig. 2. Power spectra of the source signals used in examples I-IV.

Fig. 3 .
Fig. 3. Examples of coding: (a) a third-order AR source, (b) a fifth-order AR source.Decoded signal SNRs are plotted against i.i.d.channel loss probability for Baseline and IMVI with Lloyd-Max quantization at 2-4 bits/sample.

Fig. 4 .
Fig. 4. Examples of coding: (a) a ninth-order AR source, (b) a tenth-order AR source.Decoded signal SNRs are plotted against i.i.d.channel loss probability for Baseline and IMVI with uniform quantization at 2-4 bits/sample.

TABLE I DESIGN
(28)RITHM FOR LOSSLESS TRANSMISSION Initialize h (0) : a l = b l = α l , ∀l Initialize P −(1) to unique stabilizing solution to(28)for h = h 0 Set to desired precision and i = 0 Set stop difference = ∞ while stop difference > do Set i = i + 1

TABLE II DESIGN
ALGORITHM FOR LOSSY TRANSMISSION Initialize h (0) : a l = b l = α l , ∀l

TABLE III MATCHED
PREDICTION PARAMETERS FOR EXAMPLE I.

TABLE IV FILTER
PARAMETERS FOR BASELINE-2, EXAMPLE I.

TABLE V FILTER
PARAMETERS DESIGNED FOR THE SPECIFIC LOSS RATES IN EXAMPLE I (IMVI-2).

TABLE VI QUANTIZER
PARAMETERS USED IN EXAMPLE I, IMVI-2.

TABLE VII MAXIMUM
OBSERVED SNR IMPROVEMENT IN EXAMPLES I-IV.