Hidden Markov model technique for dynamic spectrum access

Dynamic spectrum access is a paradigm used to access the spectrum dynamically. A hidden Markov model (HMM) is one in which you observe a sequence of emissions, but do not know the sequence of states the model went through to generate the emissions. Analysis of hidden Markov models seeks to recover the sequence of states from the observed data. In this paper, we estimate the occupancy state of channels using hidden Markov process. Using Viterbi algorithm, we generate the most likely states and compare it with the channel states. We generated two HMMs, one slowly changing and another more dynamic and compare their performance. Using the Baum-Welch algorithm and maximum likelihood algorithm we calculated the estimated transition and emission matrix, and then we compare the estimated states prediction performance of both the methods using stationary distribution of average estimated transition matrix calculated by both the methods.


INTRODUCTION
Spectrum sensing is usually done by measuring the power spectral density of the channel of interest in cognitive radio [1] based system to perform the dynamic spectrum access [2] operation. This method doesn't give estimated value or doesn't help in prediction in a noisy channel. Estimating the status of channel as being free or busy is important aspect of cognitive radio system. A hidden Markov process (HMP) [3][4][5][6] is useful in this prediction. A hidden Markov process (HMP) is a discrete-time finite state homogeneous Markov chain (MC) [7] observed through a discrete-time memory less invariant channel. HMPs are more commonly referred to as hidden Markov models [3]. The parameter of HMP is estimated offline from real measurements using Baum-Welch (B-W) algorithm [8,9] and maximum likelihood (ML) algorithm [10]. Giventhisparameters the state of the primary user at a given time and frequency band is determined. Hidden Markov model (HMM) [11][12][13][14][15][16][17][18][19][20][21][22][23][24] is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobservable (i.e. hidden) states. Here we avoided mathematical expressions and formulas and provided only experimental data and results.
The rest of the paper is organized as follows. In section 2, system model is explained. In section 3, we find the most likely states using Viterbi algorithm [25] and compare it with original states of the channel. In section 4, we compare the Baum-Welch and maximum likelihood algorithm and concluded in section 5. We used @MATLAB for this experimentation.

SYSTEM MODEL
Let us consider HMPθ = (π,A,B), parameterized by the initial state matrix π, the state transition matrix A, and the emission matrix B.; Here we assume a discrete-time model, where the time variable n takes values in {1,2,…}. The input to the system at time n consists of an estimate of the power spectral density. Each channel is modeled as a two-state HMP. The output of the system consists of an estimated transition and emission probability matrices of each channel from which we can predict the current states as well as the future states of the channel using predictor.
Channel parameter estimation using HMP, associate with each of the channels is based on offline training data. HMP may be view as a discrete time bivariate random process {(Sn,Yn), n=1,2,…}, where {Sn} is the hidden process i.e. states of the channel and {Yn} is the observable process i.e. sequences. The hidden process {Sn} is a discrete time finite state Markov chain. The random variables {Yn} are conditionally independent given {Sn}. Furthermore, the distribution of Yn is time invariant and depends on {Sn} only through Sn. The hidden process {Sn} takes values in a finite set S = { 0,1, … M-1}. In this system we have focused on the case M=2, such that 1 represents the channel is free whereas state 2 represents a primary user existence on the channel. Figure 1 shows the state-transition diagram of the hidden process.

FINDING MOST LIKELY STATES
Using transition and emission matrices of the channel, we generated the sequences and states of the channel using @MATLAB. Using Viterbi algorithm, we generate the most likely states and compare it with channel states. Figure 2 shows the block diagram we used to check Viterbi algorithm. We create the Hidden Markov Model (HMM) using transition probability matrix and emission probability matrix. We create two HMM, HMM1 and HMM2 having same emission probability matrix considering the same communication system.HMM1 is monotonous while HMM2 is dynamic where states changes more fast. Here Trans1 and Trans2 are transition matrices for HMM1 and HMM2 respectively. Emis is the common emission probability matrix. We Here Viterbi algorithm is used to find the hidden states from the given sequence. We created ten sequences and its states from same HMM and using Viterbi algorithm found out estimated states. On an average estimated state sequences are 90.66 % correct for HMM1 and 84.8 % correct for HMM2. Table 1 shows the sample data for estimating states using Viterbi algorithm. Figure 3 and Figure 4 shows the eigenvalues of transition probability matrix of HMM1 and HMM2 respectively. An eigenvalue plot indicates whether the Markov chain is periodic, and the plot reveals the period of the chain. All eigenvalues at roots of unity indicate the periodicity. The spectral gap is the area between the circumference of the unit circle and the circumference of the circle with a radius of the second largest eigenvalue magnitude (SLEM). The size of the spectral gap determines the mixing rate of the Markov chain. The mixing time of a Markov chain is the time until the Markov chain is "close" to its steady state distribution. In general, the spectrum determines structural properties of the chain. Here we simply exchange the occurrence probabilities of both states so spectral gap for both the HMMs are same but eigenvalues are different. A stationary distribution of a MC is a probability distribution that remains unchanged in a MC as it progresses. It give us the important confirmations like transient state and recurrent stat. It represents the limiting time-independent distribution of the state for Markov process as the number of steps on transition increases.

COMPARISON OF BAUM-WELCH AND MAXIMUM LIKELIHOOD ALGORITHM
Here we used the following block diagram ( Figure 5) to conduct the comparison. For Baum-Welch algorithm, transition probability matrix and emission probability matrix along with sequences generated by them have been used to find out the estimated matrices. While for Maximum Likelihood algoritham, sequences generated and states generated by these matrices have been used.

Estimation using Baum-Welch algorithm
Here we estimate the transition and emission probability matrixes using Baum-Welch algorithm. Using MATLAB, we estimated the matrixes using the sequences and transition probability matrix and emission probability matrix of the system. By running the experiments six times we found that averages of the matrixes gives better results than single iteration. Table 2 shows the result we obtain for HMM1. Here estTR is the sample estimated transition matrix and estEmis is the sample estimated emission matrix, while AvEstimatedTran is the average estimated transition matrix and AvEstimatedEmis is the average estimated emission matrix. Table 3 shows the result we obtain for HMM2.

Estimation using maximum likelihood algorithm
We also obtained the estimated transition probability and emission probability matrix by using the maximum likelihood (ML) estimate. In this process, MATLAB function uses the sequences and states to calculate the estimated matrixes. Table 4 shows the variation in data using maximum likelihood method for HMM1. If all the states are same then estimated transition matrix is single row and single column matrix and estimated emission matrix shows one row only. For example, for HMM1, if all the states are showing value "1" (channel is free) then Estimated transition matrix = 1. And estimated emission matrix is a row vector [0.24, 0.32, 0.12, 0.20, and 0.12]. HMM2 is more dynamic, where states are shifting fast. In HMM2 case the maximum likelihood estimate is giving appropriate result. For example, in Table 4 estimateTR1 is one of the estimated transition matrix and EstimateE3 is one of the estimated emission matrix. Table 5 shows the average of estimated transition matrices for both the HMM i.e. HMM1 and HMM2 using maximum likelihood method along with its stationary distribution. Figure 6 and Figure 7 shows the comparison of stationary distribution for HMM1 and HMM2 using both methods.  Figure 6. Comparison of stationary distribution for HMM1 using both methods Figure 7. Comparison of stationary distribution for HMM2 using both methods

CONCLUSION
Here we have used Viterbi algorithm to find the hidden states from the given sequence. As HMM1 changes slowly, state prediction percentage is higher as compare to HMM2. Still for hundred percentages result, we suggest "K out of N rule" so that predicted states would be correct. Spectrum occupancy detection in noisy channel is important task in cognitive network. Using Baum-Welch algorithm and maximum likelihood we have estimated the transition and emission probability matrices. If all the states are same then estimated transition matrix using maximum likelihood is single row and single column matrix and estimated emission matrix shows one row only. So for slowly changing channel this method gives confusing result. In terms of stationary distribution of the channel, maximum likelihood method gives good result than Baum-Welch algorithm, it is plotted in Figure 6 and Figure 7. Baum-Welch algorithm uses only sequences along with transition and emission matrices finds it comparatively difficult to estimates the transition and emission matrices. While ML method uses sequences and states to estimates both the matrices give comparatively good result in terms of stationary distribution if channel is not monotonous. HMM2 is more dynamic, where states are shifting fast, ML method gives comparatively good result as shown in Figure 7. In cognitive radio system, channel detection time is quite large (for IEEE 802.22 it is 2 sec.), also channel move time is there, so finding the averages is not a problem. Therefore in the estimation process, we suggest average of