An Optimum Approach for Preprocessing of Web User Query

ABSTRACT


INTRODUCTION
With the large amount of information available on the Web, it is much tiresome task to locate and retrieve the desired information. However the user needs to invest the time in hours to make himself satisfy for his needs. In such cases, Web Search engines play a great role in retrieving meaningful results for the user query. A search engine is a software system that is designed to search for information on the World Wide Web correspond to keywords or characters specified by the user [1][2][3]. These search engines built on various Information Retrieval techniques have thoroughly changed the ways that people search and acquire information. Nevertheless, in many situations search engines have difficulties in retrieving relevant and quality information due to noisy nature of user's query [4]. This problem occurs in many ways, for instance when a user searches the same query in different ways or enters ambiguous queries. Moreover, the user query is often much ambiguous to be easily understood by the system. So in due course, the results retrieved by the search engine are deficient. Therefore, it is important to preprocess the query by exploiting imperative algorithms before passing it to the search engine [5]. In query processing phase a set of keywords is given as an input for preprocessing, which describes the user information needs. We have created an optimum system for preprocessing of Web user query.
A set of keywords is given as an input which describes the user information needs. In contrast to usual search engines that retrieve only the results on the basis of the likely search while ignoring the semantics of the user requirements, our scheme uniquely contributes a special and novel algorithm that focus on finding the relevant meaning that describes the user's desires. The algorithm tries to find the user behavior and prevents from the repetition of accessed data. Thus it is an intelligent module as it improves the probability of success by finding the appropriate results. It performs several tasks to achieve the preprocessing phase: Stop words removal, Stemming, Spelling Correction, Tokenization and part of speech. Tokenization is the process of splitting up a query string into a set of tokens or words. It usually splits words by blank, punctuation and quotation marks at both sides of a sentence. The tokens not only considered as words but also numbers, punctuation marks, parentheses and quotation marks. Parsing is the process of analyzing a string of symbols, either in natural language or in computer languages, conforming to the rules of a formal grammar. Our mean to focus on finding the relevant meaning that describes the user's behavior and prevents from the repetition of accessed data [6]. It improves the probability of success by finding the appropriate and concerned results. It performs several tasks to achieve the preprocessing phase such as stop words removal, stemming, spelling correction, tokenization. Stopwords are words which are removed after processing of natural language. The process of reducing words to their stem known as Stemming and whereas tokenization is the method of splitting the sentence into terms.

QUERY PREPROCESSING TECHNIQUES 2.1. Stopwords Removal:
Stopwords [7] are the superfluous terms which are detached after processing of natural language. The motivation is automating the process of identifying and removing the stop words and produces the list of meaningful words. Stop words are like (the, of, and, or, etc.). These types of words don't carry any weight, hence need to be removed.
If the texts are based on a template, it might be useful to remove the words that make up the template to reduce these words' impact on the similarity measure [8].

Stemming:
It is the process of reducing the terms to their stem. Its aim at identify the basic forms of word. During this phase affixes and other lexical components are removed from each token, and only the stem remains [8]. For example, played and playing are both stemmed into play.

Spelling correction:
The proposal is to use the correct spelling [9] of the word that is most regular among queries typed in by other web users. It is more probable that the user who typed lov intended to type the query love. There is list of corrected words in the dictionary. The system finds the distance between misspelled words and corrected word and returns a corrected word, where there is smallest distance between misspelled word and corrected word.
The edit distance between two strings (or words) X and Y of lengths m and n respectively, can be defined [10] as follows.

Tokenization:
Tokenization is the scheme of splitting the sentence into terms. Different policies can be chosen regarding how to split into words, and the choice depends on the type of data to tokenize [8]. The main characteristic of tokenization is to remove noisy words, symbols and numbers that can affect the performance of the search query.

EFFECTIVE APPROACH FOR PREPROCESSING
In the supervised approach, which is also known as the corpus-based approach, machine learning classifiers such as Support Vector Machine (SVM), Naïve Bayes (NB), Decision Tree (D-Tree), K-Nearest Neighbor (KNN), etc., are applied to a manually annotated dataset [11]. After building the dataset, some preprocessing techniques are automatically applied to it.

IMPLEMENTATION
The implementation has done on Eclipse IDE using Java language with jdk (java development kit). Eclipse is an fundamental tool for any Java developer, including a Java IDE, a Git client, XML Editor, Maven and Gradle integration. Figure 1 illustrates the results obtained on the query "I want to purchase a phonee of high qualities from the markeetes of Mumbai". Step-wise Illustration Step 1.Input user query: I want to purchase a phonee of high qualities from the markeetes of mumbai Step 2. Execute Stopwords removal module: Output: want purchase phonee high qualities markeetes mumbai (query without any stopword) 3. Execute Stemming: Output: 3 want purchase phonee high quality markeete mumbai (query after performing stemming) Step 4. Execute Spelling Correction module: Output:4 want purchase phone high quality market mumbai (query with corrected word) 5. Tokenization: :'want' 'purchase' 'phone' 'high' 'quality' 'market' 'mumbai' (Tokens)