Assessment of Crowdsourcing Task Multidimensional Relationship Model through Application Prototype

ABSTRACT


INTRODUCTION
People around the world had been exposed to the cyber world where they are currently connected to the Internet. Aligned with the Internet evolution, crowdsourcing has been introduced by Jeff Howe (2006). Crowdsourcing is a method where the job provider offers the voluntary task to the digital worker with having different skills, knowledge, experiences and education level to solve a problem and give the idea via a flexible open call.
Many job providers in the world start to apply the crowdsourcing as a new business model where organization depend on the power of the digital worker to obtain job to be completed [2]. Malaysia has also implemented this business model since the past few years. Locally, there are three main components that play major roles. They are job providers, platforms and digital workers. The job providers are the companies or individual that outsource their tasks to the crowdsourcing platform. Meanwhile, platform is a system that advertise the outsourced tasks to the digital worker. The platforms act as a moderator between the job provider and digital worker. Finally, the digital worker is anyone with any kind of skills who intend to perform or complete the task [3][4]. According to Arolas, & Guevara (2012) and Arolas et al. (2015), crowdsourcing definition can be divided into eight elements [5][6]. From the eight elements, this study focuses on the digital worker or crowd which classification consists of three elements: (i) who forms the digital worker, (ii) what the digital worker has to do and (iii) what the job provider get in return. The first element is who forms the digital worker means that the digital worker must have skill and ability to perform the task. The skill and ability are used to define the competency level of digital worker. The second element is what the crowd has to do means that the digital has to solve the problem based on the type of tasks. The type of tasks refers to a simple and complex task. The third element is what the job provider gets in return means that the job provider will obtain outcome variation from the crowd based on the type of tasks. Based on the explanation above, it can be concluded that the competency level of digital workers, the type of tasks and the outcome variation are the most important part that will be used in this study as the dimensions.
The relationship within each dimension representation of multi-dimensional structures. Multidimensional describes different dimensions consist of different parts or different process [7] and [8]. According to Sue (2001), multidimensional must have at least three dimensions and each dimension have many categories that are related to each other. Through our theoretical study, this dimension also comprises of multidimensional characteristic. It consists of three dimensions' character in the type of tasks (simple, complex), outcome variety (high, low) and competency level of digital worker (knowledge, comprehension, application, analysis, synthesis, evaluation).
Basically, the aim of the platform is to ensure that the tasks are taken immediately and completed within a given time by the right skill of the digital worker. However, the platform does not have a structured mechanism to classify the type of task that would confirm the task match to the digital worker. Task is given based on digital worker skills and knowledge. To solve that issues, this study develop and test the relationships between the types of tasks, outcome variation and the digital workers in alluvial chart apps prototype. An alluvial chart app has been developed and tested to visualize each relationship between the three dimensions. Based on the understanding relationship of each dimension and it is significant eventually minimize issues related to select the digital worker to complete the task. This makes the process of the platform to select the digital worker with suitable task based on competency level become easier.

RESEARCH METHOD
This section highlights few methods of process apps prototype and assessment the level of respondent's satisfaction toward the apps prototype depends on how well the visualization, the interaction between the apps and the information provided to present the relationships between the types of tasks, outcome variation and the digital workers.

Flow Application Prototype
The apps start with specific to customize the visualization diagram box. Specifications allow respondents to set width, height and color of the user preferred. After inserting the data store, the computations begin. It starts with the calculation of the node padding. This node padding is calculated based on data stored. The data are the type of tasks, outcome variation, competency level and scale of competency level. Node padding is used to develop space or area around the node. The node padding is used to control and set the nodes at the top, bottom, left and right nodes. The padding node has two values of px and %. The px value determined the length, while the value of % determines by the width.
Two processes involved in calculating the node padding which it computes the node nested properties and populate source and target link for each node. The nested node was calculated to obtain the block of the node. The second process is populating source and target link for each node. The link is connections between two nodes that are connected to each other. The size of each link was measured by the value. The height of the link represents the size of the dimensions that are contained in the two nodes connected with the link. For each link contains target and source. The depths of each link refer to the amount of flow from source to target.
There have three steps to measure the size of each link for each node. There were computed by summing the association links, iteratively assigned the breadth for each node and push any overlapping nodes. The size of each node was calculated according to the total incoming link, no incoming link and no outgoing link. To get the size of nodes, the iteration step was executed to assign the breadth for each node so no further data is coming. Nodes were assigned with the maximum breadth of incoming neighbors plus one, nodes with no incoming links are assigned breadth zero, while nodes with no outgoing links were assigned the maximum breadth. Lastly, push nodes were executed any overlapping nodes to avoid the collision.
After obtaining the total size of each link for each node, the nodes resorted and links in ascending order were arranged. Finally, the alluvial chart to visualize the relationship among of dimensions were displayed. Respondents can choose or change the other dimension, as they like to view the clearer chart.

Process Application Prototype
The process of apps prototype start when respondent select data from provided data set by using a text box control. The data can be entered manually by copy and paste in the text box provided. Respondent can specify the presentation of charts for chart customization. The purpose of data entry is to store selected data that will be used to generate the alluvial graph. After that, respondent will be able to select the dimensions that are related to the other dimensions that he or she interested into. Finally, the alluvial chart is generated and visualized. Alluvial chart enable to present the flows in order to find out the relationship between different dimensions. It visually connect to the number of components that are related in groups. It is useful to see the number of scale belonging to the parameters in each block.

Structure for Measuring the Quality of Apps Prototype
There are three process assessments involved during the quality of apps prototype. This assessment is conducted to measure the quality of using app prototype in order to assess the respondent satisfaction.  Figure 1, respondents use existing knowledge to measure the quality of apps prototype. This research will concern all related aspects of the apps prototype: (i) visualization, (ii) interaction with the apps prototype, and (iii) information given. Respondents used apps prototype to visualize the visualization. The respondents access the data through the data store contained in the app prototype. Then apps prototype will display the visualization in the form of alluvial charts. The respondents need to study visualization, interact with the apps, and finally get some information based on their knowledge. Therefore, the success and satisfaction of respondents using this apps prototype depends on how well the visualization, the interaction between the apps and the information provided.

Relationship between Attributes
The relationships between the attributes corresponding to the three levels of assessment are described in Figure 2. There are three assessments that need respondents to answer are: (i) quality of visualization, (ii) quality of interaction with the apps prototype, and (iii) quality of information.

1) Quality of visualization
Quality of visualization is assessed to identify the capability of the visualization apps to transform the input data and make them accessible to the respondent. The issues to be measured are initial settings, data display and reporting function.
a. Initial setting refers to the requirement of input data format and parameter setting for visualization. b. Data display refers to visualizing data structures, data variations, data contents, data relationships, data descriptions, data tabulations and data decoration. c. Reporting functionality represents an apps prototype function that allows respondents to assess whether the respondent is satisfied with the benefits derived from the visualization.

2) Quality of interaction with apps prototype
Quality of interaction is assessed to identify whether the user considers this app to be easy to use and learning, accurate, effective and efficient. This research classifies the attribute of the interaction into four categories. There is ease of use, learnability, efficiency and accuracy. a. Ease of use means apps prototype is easy to control by respondents. b. Learning ability affects how respondents master the app's prototype easily and quickly in performing the desired tasks. c. Efficiency measures the extent to which respondents feel that apps prototype can assist them in performing work such as improving work performance and short time needed to get a graph. d. Accuracy indicates that these apps are free from mistakes or failure of apps prototype working.

3) Quality of information
Quality of information is assessed to identify whether the users are satisfied with the output information provided by the app's prototype. There are three attributes that the user needs to evaluate information. Among these attributes are richness, accuracy and clarity.
a. Richness of information means that the output information provided by the app's prototype is complete, interesting, needed and useful to the respondent. It also must correspond to respondents' requirements and expectations. b. Accuracy of the output information provided by the apps prototype must be accurate, correct and consistent with respondent knowledge. c. Clarity of information means that the information is presented clearly, easily understood and can be show the relationship by the respondent.

RESULTS AND ANALYSIS
In order to test the satisfaction level of respondents on the quality of prototype applications, this study relies on three assessments, namely visualization, interaction between application and information provided. For this study, the test was conducted by 12 respondents.

Quality of Visualization
To identify the quality of visualization, 14 questions under this aspect were asked. The question is utilizing 5 Likert scale (strongly agree, agree, neutral, disagree and strongly disagree). The details of the result from the testing were recognized according to the percentage value for each attribute. Figure 4 illustrates the idea of visualization quality.  Figure 4, this study found that 75% of respondents agreed with the way the alluvial chart displays input data meets the requirements of input data format. While 92% and 84% of respondents agree with the way setting the parameters (nodes, node size, label) in the alluvial chart easy to use and easy the respondent to understand. Therefore, the initial setting does not reveal any problems. On the data display feature, it is found that more than 50% of the respondents stated that the data display has a good description, labels and design. More than 58% of the respondent agree that the data on display had a relationship between dimensions. 80% of the respondents support that the size alluvial chart used is suitable. On reporting functionality, most respondents agree and satisfied with the data display and parameters used in the alluvial chart.
To know the extent of the design features presented in the alluvial chart, this study presents some questions about the properties of the alluvial chart. There are two assessments related to alluvial chart properties: (i) helpful and (ii) adequate. The answers are stated in Table 1 and Table 2. Based on Table 1, data labels have the highest readings with 42% of the respondent strongly agreed and 58% respondent agree. Data label will provide facilities and help the respondent to read the data. Other than that, the node size has the second highest after the data label. The node size can make easier for the respondent to know the amount of data. This is followed by the node and links. While the color shows the lowest readings among the five aspects. Based on Table 2, the data labels have the highest reading with 34% of the respondent strongly agreed and 58% respondent agree about data label. Respondents satisfied with data labels because the labels to indicate the data is sufficient. Third highest is node and is followed by a link. While the color shows the lowest readings among the five aspects of this.

Quality of Interaction with the Apps Prototype
To measure the quality of the interaction, eight questions were asked to the respondents. The answers from respondents have been displayed in Figure 5. Most respondent easily using these apps and quickly learn how to use the apps. 50% respondents are satisfied with the time required to obtain the alluvial chart. While 42% of respondents are strongly agree that to display an alluvial chart in an application prototype has minimal steps. Most respondents agree that the apps prototype is accurate. Respondents are satisfied with the overall quality of interaction against the app's prototype because every time a respondent interacts with the app's prototype, the user wants the process to be easy, accurate, effective and efficient.

Quality of Information
To measure the quality of the interaction, eight questions were asked to the respondents. The answers from respondents have been displayed in Figure 5.  Figure 5 shows that information obtained from apps prototype is very helpful and useful in data analysis. The output information received by the respondent must be complete and interesting so that respondents can understand the output given. Based on this statement, respondents agreed with the output information provided by apps prototype are complete (76%), useful (58%) and interesting (66%). However, respondents are not quite sure about the accuracy of the output information (42%) were provided by the prototype of this application. 76% of respondents agreed that the alluvial chart shows a comprehensive relationship. The quality information delivers overall satisfaction to the respondent in determining the output information provided.

CONCLUSION
The purpose of this paper is to develop, test and measure the relationships between the types of tasks and the digital workers in alluvial chart apps prototype. Based on the current literature, this study identified three dimensions that are related to each other's. The dimensions consist of type of tasks (simple and complex), outcome variation (high and low), and competency level of digital worker (knowledge, comprehension, application, analysis, synthesis and evaluation). The measurement consists of three main aspects which are visualization, interaction with the app's prototype and information. Based on the result and finding, this study found that 76% of respondents agreed that the alluvial chart shows a comprehensive relationship between the types of tasks and the digital workers. Majority of the respondent is satisfaction toward the app prototype in term of three aspects. As a conclusion, this study defined the comprehensive relationships among the dimensions. These will facilitate a platform to match between digital workers to the tasks and the understanding on relationships of each dimension and its significance eventually minimize process related to the selection through characteristic and skills of the digital workers.