UAV Classiﬁcation With Deep Learning Using Surveillance Radar Data (cid:63)

. The Unmanned Aerial Vehicle (UAV) proliferation has raised many concerns, since their potentially malicious usage renders them as a detrimental tool for a number of illegal activities. Radar based counter-UAV applications provide a robust solution for UAV detection and classi-ﬁcation. Most of the existing research addresses the problem of UAV clas-siﬁcation by extracting features from the time variations of the Fourier spectra. Yet, these solutions require that the UAV is illuminated by the radar for a longer time which can be only met by a tracking radar architecture. On the other hand, surveillance radar architectures don’t have such a cumbersome requirement and are generally superior in maintaining situational awareness, due their ability for constantly searching on a 360 ◦ area for targets. Nevertheless, the available automatic UAV classi-ﬁcation methods for this type of radar sensors are relatively ineﬃcient. This work proposes the incorporation of the deep learning paradigm in the classiﬁcation pipeline, to provide an alternative UAV classiﬁcation method that can handle data from a surveillance radar. Therefore, a Deep Neural Network (DNN) model is employed to discern between UAVs and negative examples (e.g. birds, noise, etc.). The conducted experiments demonstrate the validity of the proposed method, where the overall classiﬁcation accuracy can reach up to 95 . 0%.


Introduction
Considering their increasing popularity, UAVs have been devised to assist people in their daily activities because of their ease of use and robustness across various environments. They have been rendered as an efficient solution to a number of useful applications ranging from advanced cinematography to wildlife surveillance and package delivery. On the other hand, it is not uncommon for those technological advancements to be also used for malicious purposes. In particular, This work has received funding from the European Unions Horizon 2020 Research and Innovation Programme under grant agreement N • 740859, ALADDIN UAVs have been reported in many illegal activities, such as espionage, carrying explosives for terrorist attacks or provocative messages in crowded events. Finding an effective solution to this potential threat, that can reliably detect and classify incoming UAVs, around a protected area has been an active research endeavor. Different modalities have been explored both in scientific literature and industry for UAV detection and classification, including optical, thermal, acoustic and radar data. The latter have been proved to be very effective in detecting flying vehicles and objects, and is being employed for decades in pertinent applications. Yet, typical radar sensors designed for detecting manned aircraft, with large Radar Cross Section (RCS) measurement [1] and high velocity, are not suitable for detecting very small and slow moving objects, flying at low altitude such as UAVs [2]. Moreover, UAVs share similar cues with birds and thus classifying between the two targets is another major challenge. Towards this end, specifically designed radar architectures have been devised for this demanding application that overcome such burdens.
The available radar architectures employed in literature for target detection can be divided in two broad categories: surveillance and tracking radars [3]. Surveillance radars provide 360 • continuous coverage as they operate with a rotating antenna to detect and track multiple targets at the same time [4]. On the other hand, tracking radars retain their antenna to a specific direction of a designated target, illuminating it for sufficient time which enables for target classification through the analysis of time variations of the Fourier spectra (e.g. target velocity, Doppler and micro-Doppler signatures [5]). In particular, the micro-Doppler (m-D) signature of UAVs has been widely employed within literature for classification purposes [2,[6][7][8][9]. The typical detection and classification pipeline is to perform radar signal processing algorithms for detecting targets (e.g. CFAR detection algorithm) [4] and extract intrinsic features from the processed signal (m-D signature) [8,10] for automatic classification.
Despite the promising classification capabilities that tracking radars have proved to withhold, the volatile nature of UAVs can make this type of solution ineffective in a real world application. Extracting the m-D signature from a non cooperative target with high maneuverability can prove to be a challenging task and most of the research has been conducted in ideal scenarios, in close flight range. On the other side, a surveillance radar seems like a more appropriate choice for the task at hand because it is designed to constantly search the protected space for new targets, hence providing 360 • coverage and protection at all times. Nevertheless, the time for which a surveillance radar illuminates the target is not long enough to allow for deeper analysis of the intrinsic characteristics of the target through the Fourier spectra which creates a drawback in the classification capabilities of the radar. This has driven UAV classification methods with surveillance radars in the literature to utilize mostly target motion information by creating handcrafted kinematic features (e.g. motion tracks, velocity) [11] or building probabilistic motion estimation models [12] and less often through radar signature features derived from Range Profiles and Range Doppler matrices (e.g. RCS, Signal to Noise Ratio (SNR), mean amplitudes, etc) [13]. However, those methods heavily rely on specific handcrafted features omitting the rich information cues available on the Range Profiles Matrix.
Deep Neural Networks (DNNs) have been proved very efficient in discovering high-level and abstract features directly from data [14]. This work proposes a novel UAV classification method based on a custom Convolutional Neural Network (CNN) [15] architecture that classifies detected targets between UAVs and negative examples (i.e. birds, noise, etc.). The proposed network learns directly from the Range Profiles Matrix signature of the detected target alongside with radar signature features derived from both Range Profiles and Range Doppler matrices, such as Radar Cross Section (RCS), Signal to Noise Ratio (SNR) and radial velocity. We train and evaluate our proposed network with real radar measurements performed from a X-band Linear Frequency Modulated Continuous Wave (LFMCW) surveillance radar operating at 9.35 GHz. To the best of authors knowledge, this is the first attempt to utilize the available radar signature information, as an input to a DNN model, so as to learn the intrinsic characteristics of the targets directly from data. The main contributions of this work can be summarized as: • Demonstrate how the Range Profiles Matrix data can be utilized for automatic UAV classification. • Propose a custom CNN network architecture that discriminates detected targets between UAVs and negative examples.

Related Work
The brief literature review is focused on UAV detection and classification methods for tracking and surveillance radars.

Tracking Radars
In recent years, the m-D signature has been the most commonly employed feature for UAV classification in the field of counter-UAV radar based applications. It is mainly utilized as a signal pre-processing step to extract features combining them with state of the art machine learning and deep learning algorithms for automatic target classification. Among the first who utilized the m-D signature for UAV classification were [7]. The authors extracted the m-D signature with the Short Time Fourier Transform (STFT) and proposed to utilize key characteristics such as rotation rate, blade tip velocity, rotor diameter and number of rotors to classify between four different rotary wing UAVs. In a similar work, Harmanny et al. [9] proposed to extract m-D features from spectrograms and cepstrograms with a low power Continuous Wave (CW) tracking radar operating at X-band for a potential two step classification process that initially classifies UAVs versus birds and subsequently classifies the type of UAVs based on the number of rotors. Molchanov et al. [2] studied the problem of UAV classification for a CW tracking radar operating in X-band at radio frequency of 9.5 GHz.
Eigenpairs were extracted from the correlation matrix of the m-D signature and were employed as intrinsic features to train three classifiers, a linear and a nonlinear Support Vector Machine (SVM) and a Naive Bayes Classifier (NBC) to classify between different UAV wing types and birds. De Wit et al. [6] considered a m-D feature extraction method based on singular value decomposition (SVD) to classify UAVs and birds with a X-band CW tracking radar sampled at 96 kHz. The authors proposed three main features to allow for quick classification: target velocity, spectrum periodicity, and spectrum width.
In an effort to extract the m-D signature with a non-Fourier algorithm, Oh et al. [8] proposed an empirical-mode decomposition (EMD) based method for automatic UAV classification. A nonlinear SVM was trained for target class label prediction after feature normalization and fusion. The authors validated their method on the same dataset as [2] outperforming common Fourier based m-D extraction methods.
The first work that utilized CNNs to learn directly from the m-D signature spectrograms and classify UAVs was [10]. The authors employed GoogleNet [16] and trained it with spectrograms from real UAV measurements from a FMCW radar. In addition, they proposed a method to improve the m-D signature results by merging it with its frequency domain representation, namely the cadence velocity diagram (CVD). Mendis et al. [17] utilized a deep belief network (DBN) [18] for UAV classification using a S-band CW tracking radar. The authors experimented with three different UAV types two rotary and one fixed wing. The spectral correlation function (SCF) was employed to identify unique modulations caused by the many dynamic components of the target of interest.

Surveillance Radars
UAV classification methods that utilize a surveillance radar are not as common in literature. A similar work to ours is [13] where the authors proposed a binary classification method to distinguish between UAV and bird tracks with data captured under a surveillance radar. The authors adopted a set of twenty features based on movement, velocity and target RCS extending the works of [11] and [19] that initially proposed a similar approach to classify aircraft and bird tracks. The handcrafted features are combined with a Multi Layer Perceptron (MLP) classifier demonstrating high classification accuracy. We utilize a deep learning network to learn directly from the data combining some typical radar features instead of relying on handcrafted features entirely. Furthermore, we work on each detection separately instead of utilizing tracks, which are groups of multiple detections across time.
A different approach based on motion information only is [12]. The authors proposed a probabilistic motion model estimation method based on calculating the time-domain variance of the model occurrence probability in order to classify between UAVs and birds with data originating from a surveillance radar. They validated their approach on simulated and real data showing promising results.

Radar Parameters
The radar sensor that is utilized to acquire data, constantly scans within the range and azimuth domain. Thus, this 2D radar omits altitude information from the localization of the detected target. In particular, it is a X-band LFMCW surveillance radar that operates at 9.35 GHz, with a transmitted power of 4 W and a Pulse Repetition Frequency (PRF) of 3.3 kHz. The radar antenna elevation angle is usually set to around 20 • and the radar antenna performs 20 full revolutions per minute. The max detected range is configurable, ranging from 600m up to 4km, with different range resolutions. Both short range and long range configurations were picked during the data creation process.

Signal Processing
The raw data acquired by the 2D radar are a matrix of complex values, referring to In-phase and Quadrature receiver channels. The two dimensions of the matrix represent fast time and slow time signal information [4]. Each column represents a sweep, and rows represent the sampling of the LFMCW signal. The number of rows is fixed by the sampling frequency of the received signal (around 600 samples), the number of columns depends on the PRF and on the duration of the acquisition.
Raw data are processed by calibration, radar equation correction, and Fast Fourier Transform (FFT) along the direction of fast time, in order to obtain the Range Profiles Matrix of RCS values [4]. The Range Profiles Matrix is also a matrix of complex values; although, its two dimensions now represent range and slow time values. Slow time instances can be also interpreted to identify azimuth values. To detect targets, Range Profiles are divided in samples, each sample refers to a specific range cell (rows of the matrix) and the slow time or azimuth instance correspondents (columns of the matrix). These samples are processed by FFT along slow time direction, also referenced as Doppler direction in literature [4], and the result is a list of detections. This process produces the Range Doppler Matrix of RCS values in range and radial velocity dimensions. The local maximum points of the Range Doppler Matrix represent potential detected targets. These points are processed by clustering and produce a list of detections in range and radial velocity domain (rows and columns of Range Doppler Matrix). The same rows and columns correspondents can be used to extract the Range Profile Matrix signature of each detection. This is the first input to the proposed DNN model.
A set of radar signature features can be extracted from the Range Doppler Matrix and are used as the second input to our DNN model. Those would be RCS of the target, which is the detection amplitude. The Signal to Noise Ratio (SNR), which is the ratio between the detection amplitude versus the mean amplitude in the same range of the detection. And finally, the radial velocity of the detected target.

UAV Classification Pipeline and DNN Model
UAV Classification Pipeline. The input to our proposed UAV classification method is a list of detections and radar features derived from classic radar signal processing algorithm [4]. Our DNN model yields an inference whether a detected target is a UAV or a negative example (e.g. bird, noise, etc.) based on the signature of the target, provided by the radar. The network is trained in a supervised manner, by utilizing a dataset consisting of multiple UAVs and negative example detections.
Network Architecture. The basic building block of our proposed model are the Convolutional Neural Networks (CNNs) [15]. Driven by their unprecedented success in tasks, such as image recognition [20] and object detection [21], we designed our model from scratch, following a similar architectural design, as it is depicted in Figure 1. To the best of our knowledge, there are no recommended DNN architectures based on literature that solve the problem at hand. We opt for a two branch architecture to handle the analysis of Range Profiles Matrix signature, separately from the radar features, and concatenate the output feature maps of both branches before the fully connected layer for joint training. The Range Profiles Matrix branch is consisted of two 1D CNN blocks with 128 and 64 output convolution filters each followed by a ReLU activation layer, an average pooling layer and a dropout layer. The radar features branch consists of a single 1D CNN layer with 64 filters followed by a ReLU activation and a dropout layer. The output of the fully connected layer is the probability of the input sample under test, to be one of the two existing classes (UAVs and negative examples) assigned by a Softmax activation function.

Training and Testing Settings
The proposed DNN is trained from scratch and we use the Xavier initialization for the initial weight values. The dropout layers probability is p = 0.5 and is applied to prevent overfitting effects. The objective function is set to be the categorical cross entropy. We utilize the Adam optimizer with initial learning rate of 0.001. To achieve the best UAV recall possible, we keep every prediction with a probability above 0.5 for UAV class as a UAV prediction. The Range Profiles matrix input samples have size of 126 × 4 and we utilize the magnitude of each complex number before feeding it as input to the DNN. The three additional radar signature features are concatenated beforehand and then are being fed to the model. The network was trained for 100 epochs with a batch size of 128. All experiments were performed on a NVIDIA GeForce 1070 with 8GB memory.

Dataset description
To train and evaluate our proposed model, we performed real radar data acquisitions with multiple UAVs flying under different flight plans, both close to the radar (50 to 600m), as well as far away (up to 2km). External GPS trackers were attached on top of the deployed UAVs, to provide ground truth trajectories, thereby enabling easier annotation for the training and evaluation of the proposed method. Each radar detection has a ground truth label assigned to it, for measuring how close is the detection to the GPS ground truth trajectory. A total of 4.5 hours of recorded data were collected to form a dataset with sufficient amount of data for a DNN model. By this dataset, two acquisitions were utilized for evaluating the DNN model, which include multiple UAVs (up to 5 drones) that soar in the sky with distinct patterns. The samples that were utilized for the training of the classifier were evenly selected, with respect to a balanced distribution between the available classes. The selected samples utilized for training purposes, include 2536 UAV and 2594 negative samples. Furthermore, the two test acquisitions utilized for testing the method include 2038 UAV and 1371 negative samples. Table 1 illustrates the confusion matrix with detailed classification results of our DNN model, for both classes and all the detection samples under testing examination. The metrics that are utilized are the per class precision, recall and F-1 score, as well as the overall classification accuracy. Per class metrics are depicted in Table 2. It can be observed that the model yields a substantial performance in all metrics, achieving a remarkably high score, where the average precision and recall are 94.48 % and 95.55 % respectively. It turns out, the model can reliably distinguish between radar signatures, which belong to UAVs and those that can be whatever object, such as a bird or noise. Another factor that indicates the superior classification capabilities of the model, is the near perfect UAV precision score of 98.79 %. By the total number of samples, only 23 negative ones are falsely classified as UAVs. The model's overall classification accuracy (correctly classified samples to total samples) is 95.0 %. The two distinct flights employed for testing the model's performance, were carried out under close range configuration, flying in distances up to 450m from the radar. In Figure 2 we adduce the per distance UAV precision and recall metrics, in an attempt to examine how the range factor affects the performance of the classifier. It can be observed that relatively closer to the radar (up to 200m) the proposed model seems to classify some of the UAV samples as negative, thus the UAV recall is below 90 %. However, at mid-range (≥ 200m) both UAV precision and recall are above 90 %. Finally, we present qualitative results of our model on Figure 3, where we have included the model predictions for one of the testing acquisitions. On the left part, the flight plan GPS trajectory followed by 5 UAVs flying next to each other is depicted. On the right part, we present the predicted results of our model on the radar detections of this flight. We show UAV predictions with blue color and negative example ones with red. Our model can reliably recognize the UAVs that are flying within the area covered by the surveillance radar.

Conclusion
This work presents a deep learning based UAV classification pipeline that utilizes data from a surveillance radar. The proposed method has been evaluated on a dataset of real radar data acquisitions with UAVs. This study provides an alternative pathway to the radar research community, for automatic target classification algorithms. As future work, we intend to extend the model's architecture, so as to recognize the wing type of UAV and also to incorporate earlier samples from the sequence, thus taking into account, besides the appearance, the relative position of the samples, as an additional cue for classification purposes.