Application Development Risk Assessment Model Based on Bayesian Network

This paper describes a new risk assessment model for application development and its implementation. The model is developed using a Bayesian network and Boehm’s software risk principles. The Bayesian network is created after mapping top twenty risks in software projects with interrelationship digraph of risk area category. The probability of risk on the network is analyzed and validated using both numerical simulation and subjective probability from several experts in the field and a team of application developers. After obtaining the Bayesian network model, risk exposure is calculated using Boehm's risk principles. Finally, the implementation of the proposed model in a government institution is shown as a real case illustration.


Introduction
To develop a high quality application on time and within budget, one usually has to deal with various risks [1].It is widely known that the success rate of successful IT project is very low.This fact is supported by a research from Standish Group on IT project 1994IT project -2004 where it is shown that the completely failure rate was 18% in all projects, 53% of them were completed with unsatisfactory time, costs or effect, and only 29% of them had successfully accomplished the project target [2].These facts show us that it is necessary to consider risk assessment as a way to systematically identify whether the occurrence of risk may affect the objectives of organization.Risk assessment method, as part of risk management, can be used as a tool to analyze both opportunities and consequences of risk prior to decide the next strategic action [3][4].Risks in application development are related not only to the resources and functional problems encountered in the process of developing application but also to the impact of such problems.
Risk assessment in application development is a process to identify risk factors such as lack of clarity of project requirements, delivery not according to schedule and time, and failure in achieving the main objective of application development project [5].Risk assessment procedure generally consists of risk identification, risk analysis and risk evaluation; and it provides an understanding of risks, their causes, consequences , and their probabilities [3][4].According to Tao, risk management in software projects should focus on prevention and reduction of risks, assessing the likelihood of problems, determining risk potential that could become a major concern [6].In other development, Sonchan et.al.have introduced top 20 software risks based on the frequency of their citations on highly referred and recent literature in the area of risk management of application development projects.They have succeeded in extracting and classifying top risks from thirty most frequently cited and recently published literatures on software project risks.They used Delphi method to propose potential impacts and probabilities of all classified risk [7].They categorize these risks based on risk taxonomy described in [8].However, their study did not investigate the interrelationship and probability of risks.
Other study from Gallagher has identified risk areas that are interconnected and he used interrelationship digraph to identify the cause and effect between risks [9].He describes risk areas involving: (i) schedule pressure and veracity, (ii) suppression of information, (iii) The study in [7] does not cover the possibility of relation between risks, where the study in [8] indicates that it is possible to find the relationship between risks.Based on these research result, we enhance the work of [7] to show the possibility of relation between the top 20 risks.Earlier effort on this topic can be seen in [10] where the theoretical background, context diagram, mapping table, and preliminary model were elaborated.Our paper investigates further on how to determine the probability and interrelationship of such risk in application development using Bayesian network concept.Moreover, this paper also presents the impact of such risk using Boehm's application risk principles and risk exposure.Finally the implementation of the proposed model as a real case illustration in a government agency will be elaborated.

Research Method
This section will describe the step-by-step procedure to develop the proposed model.Initially, we adopt the research result on top twenty risks in software project as our basic risk classification [7].In order to find the relationship among risks, those mentioned risks are mapped to the SEI Taxonomy-Base Risk Identification to find the characteristic of related risks [8].Then based on the characteristic of the risks, the Category of Risk Area identified by Gallagher is used to find the interrelationship among risk [9].The obtained grouping is shown in Table 1 to explain the relationship between risk areas.This result can also be seen in our previous research [10].With this grouping, one can create a dependency model based on interrelation of risk category area shown in Figure 1.For clarity and consistency in further discussion, we assign each relevan risk with specific codes as seen in Table 2. Using the works described in [9], a direct relation between risk items can be constructed as an initial model shown in Figure 2. As we can see, the network contains many node dependencies.The validity of this initial dependency needs to be verified for implementation in a real environment.To evaluate node dependencies, we turn to expert judgment through discussion and survey.The experts involved in this step are application developers that we selected from application developer team in Statistics Indonesia.Based on their experience, we are able to eliminate the relation among risks that are considered irrelevant.The result of model refinement can be seen in Figure 3 and Table 3.In the next section we will show that this model is valid to be implemented in a real application based on judgment from international experts in the field and application developers.In this paper, following model derivation, we work further for implementation.We expand the model to cover risk exposure and its impact on organization.We begin with exposure concept from Tan to find the risk factor [11].Based on this risk factor we calculate the risk exposure using the result from Boehm [12].The risk exposure is defined as the relation between probability of unsatisfactory outcome and los s as a consequence of those unsatisfactory outcomes.In this paper we use risk factor as an input of loss as a consequence of unsatisfactory outcome.We will use the following equation. Where: = Probability of unsatisfactory outcome Loss(UO) = Loss as an impact of unsatisfactory outcome Finally we use the Risk-Level matrix from Stoneburner to classify the risk.The risk calssification consists of three levels, i.e.: High, Medium, and Low [13].With these levels, it will be easier for the management to comprehend the risk as a whole and to take action as a response to the risk.

Evaluation
To evaluate the model, first we perform a numerical simulation on several nodes.Since the model uses the basic concept of probability, then the following conditions described in equation ( 2) must be fulfilled.
Where: CPT probability in Conditional Probability Table total probability at a specified node P(R) probability of the risk at nodes using the proposed model For illustration purpose we calculate the probability of node (R1|R2), which means the probability of "Unclear Customer Requirements" with evidence of risk "Requirement Creep".Assumed at random that the probability of node R1 is shown in Table 4, and assumed that the prior probability of (R1|R2) is shown in Table 5.Using simple probability theory we can calculate: 1. P(R2|R1=T)=0.9 The risk's probability of "Requirement Creep" if risk of "Unclear Customer Requirements" occurs is 0.9. 2. P(R1)=0.7 The risk's probability of "Unclear Customer Requirements" is 0.7.Based on CPT in equation (2).We conclude that the risk's probability of "Unclear Customer Requirements" with evidence of risk of "Requirement Creep" is 0.75.We can calculate the probability of each node in the network with the same procedure and will obtain the consistency that each node fulfills the condition in equation (2).
In addition to evaluation by simulation, we also evaluate the model through judgment from international experts and application developers.We involve 20 application developers from the government agencies we mentioned before and 10 international researchers/expert s in the field of application development.The result of relationship evaluation from 20 application developers and 10 researchers point of view can be seen in Table 6.Based on numerical simulation and subjective judgment in

Figure 1 .
Figure 1.Proposed relationship model based on Category of Risk Area

Table 1 .
Proposed mapping table of risk assessment model

Table 3 .
Relationship among top twenty risk

Table 6 ,
we conclude that our proposed model has consistency and can be used as an assessment tools.

Table 6 .
Evaluation of relationship among risks based on point of view from 20 application developers and 10 researchers