Lateral inhibition neural networks for classification of simulated radar imagery

The use of neural networks for the classification of simulated inverse synthetic aperture radar (ISAR) imagery is investigated. Certain symmetries of the artificial imagery make the use of localized moments a convenient preprocessing tool for the inputs to a neural network. A database of simulated targets is obtained by warping dynamical models to representative angles and generating images with different target motions. Ordinary backward propagation (BP) and some variants of BP which incorporate lateral inhibition obtain a generalization rate of up to approximately 78% for novel data not used during training, a rate which is comparable to the level of classification accuracy that trained human observers obtained from the unprocessed simulated imagery.<<ETX>>


A Simulated ISAR Database
Our database consisted of simdated inverse synthetic aperature radar (ISAR) images of ships. Real imagery is a Doppler vs. range profile produced by the ship's motion. Mathematically, the image shape depends on the cross-product of the line of sight vector with the instantaneous angular velocity vector of the ship. The resulting image exhibits the combined effects of the independent motions of the ship due to its roll, pitch, and yaw. Our artificial database contains target silhouettes which simulate the rudimentary shape of a ship in an image at different aspect angles and with varying degrees of roll, pitch, and yaw.
Although real imagery also has spectral characteristics, for the purposes of identifying the Perceptual Class of a ship, general shape information of a silhouette is usually sufficient. By "Perceptual Class" we mean the categories "Commercial/Auxilliary", "Combatant", "Landing Platform" , "Submarine", and "Small Craft." For the purposes of this pilot study, we were interested in the determination of Perceptual Class. Therefore, we chose to simplify the problem by creating simulated images which all had binary image intensity levels.
We actually investigated a subset of the Perceptual Class problem: the ability of backward propagation and variants of bp incorporating lateral inhibition to distinguish the difference between commercial/auxilliary ships and combatants in the artificial database. For training, simulated images were presented to represent a variety of different target aspect angles and motion parameters. Each of these frames was presented once with the bow left and right. In figure 1, we show some examples of the kinds of image.frames which we used. The actual input to the network was the weighted rms image variation about the image center of mass in each range bin: where pij is the intensity of the image in range-doppler cell i j, and & ( j ) = cj jpij is the image center-ofmass in range-bin i. Figure 2 shows a sample image frame and the corresponding range-bin moments used as input. Global 2-D invariant moments are of somewhat limited utility for recognition tasks such as this, which require more detailed shape information (Park and Sklansky, 1990). Therefore, we use local range-bin U.S. Government work not protected by U.S. copyright.

11-1 16
T moments to give more details about the structural shape. We also make this choice because of symmetry: while features may change their scale and si n in the Doppler domain, they will remain constant in range.
Our database consisted of 4896 simulate! ISAR image frames drawn from 21 different ships. We divided the database into three subsets: one for training, a test set for monitoring generalization during the training procedure (cross-validation), and finally a second test set for evaluating generalization after training. Details of the composition of these data subsets are summarieed in the following table. 1

I 2 Lateral Inhibition Neural Networks
Lateral inhibition has been studied in a variety of contexts, for example in unsupervised learning networks (Cooper and Scofield, 1988;Intrator, 1990;Seabach, 1990), in reinforcement learning (Sutton, personal communication) and dynamical models (Horn and Usher, 1990). Also, recent work (Giraud, Liu, bernard, and Axelrad, 1991) has looked at networks with excitatory-inhibitory pairs of neurons.
In the context of backward propagation (Rumelhart, Hinton, and Williams, 1986; Werbos, 1974), Sandon (1987) developed an ad hoc method for incorporating a linear competition between the error signals. At a given level in the network, the error signal for a particular neuron was modified by subtracting out an amount proportional to the sum of the error signals at the other nodes: Heuristically, it is appealing to allow competition between the error signals; however, such an approach does not guarantee that the network will implement a gradient descent minimization procedure. In contrast, we have taken the approach of imbedding the lateral inhibition in the energy functional and carrying out gradient descent on the modified energy functional. We accomplish this by introducing fixed lateral inhibition in the forward pass of the data through the network. With this approach, we have the advantage of being certain that we are actually carrying out a minimization procedure by gradient descent.
In our network, during the forward propagation of the pattern, the linear net input is computed for each cell, then the lateral inhibition of these inputs is computed, after which the nonlinear sigmoid is applied to the result to compute the firing rate of each cell. Therefore, the feedforward equations for pattern s are: where il'(n) is the net feedforward input, ~3 ; '~~) is the cell activity, and .(.;A) = is the nonlinear sigmoid input-output function. Here, p(n) E & is the fixed parametric inhibition in layer n, and N n is the number of cells in that layer. Figure 3 compares the lateral inhibition architecture with that of ordinary backprop. With these modifications, the energy function to be minimized is defined in terms of the inhibited outputs of the final layer of the network. When we take the gradients of the modified energy functional with respect to the synaptic weights, we find that the ordinary backprop error signals, 6!7), must be replaced by inhibited error signals, g! : ) , of the form:  6~~o n e c k r r ) is the Kronecker delta, or identity matrix). This fixed architectural constraint allows competition between the cells in a layer to represent particular features in the data.
Notice that the last layer error signals are equivalent to the ad hoc formula used by (Sandon, 1987); this layer is looking at the spatial difference in how close the output layer cells are to their respective targets.

Benchmarks for Backprop and Lateral-Inhibition Variants
We considered both three-and four-layer architectfires (one and two layers of hidden units respectively). In the end, we settled on a three-layer architecture as the best configuration. Experiments with five hidden units revealed poor generalization to novel data. However, we obtained good generalization with seven or eight hidden units. More cells led to overfitting. Amplitudes of some sample first layer synaptic vectors are shown in figure 4.
We chose a small step-constant 77 = 0.1 with moderate momentum IC = 0.6 for the experiments.
We trained the networks to distinguish between artificial ima es of combatants and those of commercial/auxiliaries as described above. Synapses were saved in a iuffer whenever the generalization on the monitoring data set improved. The synapses in this buffer, representing the best level of generalization 11-1 18 T w . .

11-1 19
Some of the variation can be accounted for by degree of experience. It is interesting, however, that the most experienced human observor, DD in the table, did not obtain the best score. Nevertheless, this experiment provides a baseline for the degree to which the perceptual class of the simulated image silhouettes can be identified.
The results which we have obtained for the neural networks for a range of randomly chosen initial starting conditions appear to have a similar mean classification accuracy, although we can not make a strict comparison because we have not allowed for an "other category" in the neural net experiments. An expanded database of "other" simulated targets will be useful in future work. We also plan to develop an on-line version of the simulation which will present randomly chosen views of the ships in the database to the neural network. Work on hybrid networks combining unsupervised and supervised learning algorithms is also planned.

Acknowledgement
We wish to acknowledge Dave Drake for assisting us in the creation of our artificial radar database and to thank the other