A Fast and Incremental Development Life Cycle for Big Data Computations

Big Data does not only refer to a huge amount of diverse and heterogeneous data. It also points to the management of procedures, technologies, and competencies associated with the analysis of such data, with the aim of supporting high-quality decision making. There are, however, several obstacles to the effective management of a Big Data computation, such as data velocity, variety, and veracity, and technological complexity, which represent the main barriers towards the full adoption of the Big Data paradigm. The goal of this work is to define a new software Development Life Cycle for the design and implementation of a Big Data computation. Our proposal integrates two model-driven methods: a first method based on pre-configured services that reduces the cost of deployment and a second method based on custom component development that provides an incremental process of refinement and customization. The proposal is experimentally evaluated by clustering a data set of the distribution of the population in the United States based on contextual criteria.