User stories collection via interactive chatbot to support requirements gathering

Nowadays, software products have become an essential part of human life. To build software, developers must have a good understanding of the requirements of the software. However, software developers tend to jumpstart system construction without having a clear and detailed understanding of the requirements. The user story concept is one of the practices of the requirements elicitation. This paper aims to present the work conducted to develop an Android chatbot application to support the requirements elicitation activity in software engineering, making the work less time-consuming and structured even for users not accustomed to requirements engineering. The chatbot uses Nazief & Adriani stemming algorithm to pre-process the natural language it receives from the users and artificial mark-up language (AIML) as the knowledge base to process the bot’s responses. A preliminary acceptance test based on the technology acceptance model results in an 83.03% score for users’ behavioral intention to use.


INTRODUCTION
Software is an important aspect that has an impact on human life, such as work, daily activity, finance, and other essential fields. There are some steps to build software, and developers must have a clear understanding of the requirements of the software. However, the developer tends to build software without having a clear understanding of the requirements in detail because for them having understood requirements in detail is a waste of time, in which requirements are constantly changing [1]. These things lead to software failure because when the developer does not have a clear understanding of the system, an error can occur in system design and the next steps [2].
Requirements engineering is an activity that has the aim to know, understand, analyze, and documenting what are the requirements that needed by stakeholder [1], so involvement of stakeholders is crucial in software project especially in requirements engineering process [1,3]. One of the steps requirements engineering is requirements elicitation which user story is one of its practice. User stories introduced the first time to extreme programing in agile development methods and started used in other methods, such as Scrum, [4]. User stories focus on interaction with the user, so users can participate and describe the functions that can be useful for stakeholders. A chatbot is a computer agent that can interact with the user and make human-bot conversation feels like they were a human-human conversation. Based on TELKOMNIKA Telecommun Comput El Control  User stories collection via interactive chatbot to support requirements gathering (Ferliana Dwitam) 891 previous research, natural language processing used for chatbot can interact and understand like a human [5][6][7]. Based on a survey by Oracle, more than 70% of the respondents said that people or businesspeople already used chatbot or have a plan to use a chatbot in 2020 [8]. When using a chatbot, the customer does not need to waste their time to meet face to face to get specific service which makes users save the cost; these things make chatbot has brought positive impact for both customers and the company [9].
Sending messages is the core of the mobile experience [10], and the increase in internet and mobile devices in the present makes the interaction between humans easier [11]. Based on a survey in 2017, mobile device users in Indonesia already reach 341,4 million [12]. Artificial mark-up language (AIML) is an XML-based mark-up language that used for brain or knowledge of the chatbot which used the first time in ALICE bot [13,14]. In other researches, using AIML for the brain of the chatbot is a success in creating interaction between bot and user.
The Nazief & Adriani algorithm is a stemming algorithm commonly used for Bahasa Indonesia, which is considered as the qualifief one, compared to other stemming algorithms [15]. That algorithm used because chatbot using Bahasa Indonesia and Nazief & Adriani algorithm have higher accuracy for the Indonesian language stemmer than other algorithms such as porter [16]. Other works have also shown the potential of Nazief & Adriani algorithm in stemming natural languages in Bahasa Indonesia [17][18][19]. Furthermore, chatbot for related works of literature concerning the development chatbot in Bahasa Indonesia has also shown the use of chatbot in various fields [20][21][22]; however, the use of chatbot and Nazief & Adriani algorithm in the requirements engineering field is still rare to be found.
This research aims to bridge the gap between the stakeholders and the software engineers in terms of requirements gathering by developing a chatbot for the stakeholders to chat with when submitting preliminary requirements and/or user feedbacks regarding a software product. The chatbot uses AIML as its knowledge base, Nazief & Adriani stemming algorithm, and follows the User Stories format which will require the users to provide information regarding his/her role, needs, and business value in a natural conversation. The bot's performance was measured using several chatbot metrics, and a preliminary user acceptance test following the technology acceptance model was conducted for evaluation.

RESEARCH METHOD
In this research, the chatbot application built using the Nazief & Adriani stemming algorithm for the stemming process and AIML is used for knowledge of the chatbot. Chatbot used for gathering software requirements using user stories.

Chatbot design
In building the Android application with the chatbot included, the object-oriented programming approach is used. UML diagrammatic notations are used to design the application. Figure 1 shows the use case diagram of the chatbot. There are two kinds of users, the generic user, and the software developer. The generic users could perform two main activities which are to view the list of chat rooms in his/her application, and also to give user stories by communicating with the chatbot. The developers could do anything the generic user can do, and also, he/she could also view the submitted user stories as the requirements gathered for the corresponding application.

AIML
Our research uses AIML (Artificial Intelligence Markup Language) used for the knowledge (brain) of the chatbot. In AIML, for one tag category usually, it consists of tag pattern and tag template. Tag pattern is used for matching user input and tag template used for output from the bot. Figure 2 shows an example of the AIML of the chatbot application. That example shows the knowledge of the chatbot for users to input their role. After the user inputs their role, the submitted data will be caught by the wildcard symbol in tag pattern and will be set in variable temp. After that, in order to get a response from the chatbot, the value of the variable temp will be checked.

Text pre-processing
Text preprocessing process is done before parsing the message to the brain file. Firstly, normalization is done, which message will be converted to lower case. After normalizing the message, tokenizing will be done. Tokenizing is a process that split the message into tokens. Each token will be checked if there is a mistyped word. Process stop word removal will be done after checking the typo word after that stemming will be done with Nazief & Adriani stemming algorithm. Stemming starts by checking the combination prefix-suffix process. If a common combination detected, the suffix is then checked and stemmed, and after that, the prefix will be checked and stemmed, and finally, a new word will be received. If a unique combination is detected, the prefix will be checked, and stem and after that suffix will be checked and stemmed, then again, a new word will be received. Figure 3 shows the activity diagram for the preprocessing process.

User story
The user story concept is a part of the agile development method that was first proposed in extreme programming for gathering user requirements that are needed by the system [4]. It then becomes key in the software development process [23]. User stories focus on the interaction between the user or customer [3], and they will describe functions that are needed for the system in the form of a story [4]. Example form of the user story can be seen in Figure 4 [24]: a. <role> represents the role of the person that provide the story. b. <activity> represents the requirement. c. <business value> represents the reason or value of the above requirement.

RESULTS AND ANALYSIS
The result of the research is a chatbot application that used for gathering software requirements using user stories and Nazief & Adriani stemming algorithm. In this chatbot AIML used for the chatbot knowledge. A user acceptance test is applied in this research to evaluate the chatbot's performance, perceived by the users. Figure 5 (a) display the splash screen of the chatbot applications. Splash screen will be show when user open the applications. Figure 5 (b) shows the chat room list menu. This menu show list chat room that user created. In this menu user can create or delete chat room. Chat room is represented by a card. Each card of display index of card, the name of the application and role of user.  Figure 6 shows the user story menu. The user story menu is for the developer so they can see a list of user stories that have been obtained from the user. Each user story represented by a card that every card shows the story, reason, application name, and the date that the user story obtained. In each card also have a delete button so the developer can delete the user story. At the top of the user story, there is a field that can be used by the developer to search the user story by application name, story, and reason. This list of the user story is already sorted by descending so the latest data will always be displayed in the first row, Figure 7 shows the chat rooms and an example of giving a user story. In this chat room, users can chat with the bot and give the user story. The first-time bot will ask the role of the user. After the user tells their bot, the role and the bot will then store the role information, and the user can start to give the user story. Every time the user gives the user the reason the bot will ask confirmation whether or not the user has given by the user correctly. If the story and the reason already correct, the user story will be accepted and saved.

Chatbot metrics
Several metrics were used to evaluate the bot's performance quantitatively. The metrics used include total elapsed time, total number of user turn, total number system turn, the total number of turns per task, dan total elapsed time per turn. Total elapsed time calculates the total elapsed time from when the user starts the conversation until when the user finishes submit a user story. The total number of user turn calculates the number of user chats from when the user starts the conversation until when the user finishes submit a user story. The total number of system's turn calculates the number of bot responses from when the user starts the conversation until when the user finishes submit a user story. Total elapsed time per turn calculates the elapsed time for a user to finish his/her turn; this is calculated by dividing the total elapsed time by the total number of turns. The metrics evaluate the provision of 30 user stories given by three users in which the time is measured manually using a timer. Table 1 shows the sample metric of a user when testing the chatbot application, performing a total of 8 tasks. Each task in Table 1 represents an activity in which a user is communicating using the chatbot to provide user stories, with the following numbers are calculated for the metrics.

Preliminary user acceptance test
A preliminary user acceptance test was conducted to evaluate the perceived usefulness (PU), user attitude towards (A), behavioral intention to use (BI), and perceived ease of use (PEOU), following the technology acceptance model proposed by Davis [25]. The questionnaire uses a five-point Likert scale used in order to determine the level of user acceptance, which is comprised of strongly agree (5), agree (4), neutral (3), disagree (2), and strongly disagree (1). Thirty-three respondents participated in this questionnaire. In order to test the capabilities of the chatbot, we use a university information system, as the case study for which the users will provide feedback.
User acceptance test is done to 33 users using a web-based university information system, called MyUMN, as a test case for which the users will provide user stories on. The respondent is MyUMN's users which are student, lecturer or university staffs. The respondent's age is above 20 years old, both male and female respondents. Firstly, respondents were asked to download the APK file for the chatbot application. Then, users experimented with the app, based on the application description explaining about user story and what the chatbot aims to perform. After the users tried and experimented with giving feedbacks regarding the MyUMN web application and communicating with the chatbot, questionnaires were distributed, and interviews were conducted to get feedback from the users, mainly focusing on the four aspects of the technology acceptance model mentioned previously. The questionnaires' answer sheet is formed based on the intervals using the Likert Scale, with interpretation, as shown in Table 2 below.
Based on the questionnaires filled by the users after using the chatbot application, we calculated the average score from the questionnaire for each factor in the technology acceptance model. Results are as follow, 83.79% for user attitude towards the chatbot application, 85.45% for perceived usefulness, 84.55% for perceived ease of use, and 83.03% for users' behavioral intention to use. The graphical result of user acceptance test can be seen in Figure 8. Interiews also shows that the users' as stakeholders of the MyUMN web application. The overall results seems not to differ from one aspect to another, however if we look closely, perceived usefulness achieved the highest score, this aspect measures the users' acceptance regarding how well the chatbot is perceived to be useful to support the requirements gathering activity (from the developers point of view) and the feedback provision activity (from the users' perspective) by providing a possibility of a natural communication to structure the feedbacks following the user story format, eliciting the feedback providers' role, their needs, and the business value regarding the needs/requirements provided.  The behavioral intention to use scored the lowest of the four aspects, however, looking at the interpretation score shown in Table 2, it still managed to be able to be interpreted as Strongly Agree. Interviews conducted after the questionnaires-filling activity shows the users' motivation and intention to use the chatbot again in the future.

CONCLUSION
The conducted research has developed a chatbot application based on the Android mobile platform, to support the requirements gathering activity by providing ways to bridge the communication gap between the developers of a software product and their stakeholders. The requirements that are gathered are then naturally enforced by the chatbot to follow the user story format containing the users' role, requirements/needs, and its business value. The chatbot also implemented the Nazief & Adriani stemming algorithm for the specific need of stemming words written in Bahasa Indonesia. Furthermore, it utilizes the Artificial Intelligence Markup Language for the bot's knowledge base in which the bot response is decided, given a user chat has been submitted. A preliminary user acceptance test based on the technology acceptance model shows promising results. The test focused on evaluating four aspects of user acceptance which are the user attitude toward the chatbot application, perceived usefulness, perceived ease of use, and the users' behavioral intention to use. The chatbot received scores for the four aspects as follow, 83.79% for user attitude towards the chatbot application, 85.45% for perceived usefulness, 84.55% for perceived ease of use, and 83.03% for users' behavioral intention to use. Future works include employing a more advanced natural language processing method for feedbacks written in Bahasa Indonesia, developing the web dashboard for the superuser (developers) to manage the AIML brain file and manage submitted feedback by filtering and classifying them into meaningful information for the software engineers. Furthermore, a more thorough user acceptance test could also be conducted to further evaluate the chatbot's performance and its users' perspective toward the chatbot.