Published March 29, 2026 | Version v1

Big Data Analytics for Predictive Modelling Using Distributed Data Processing Frameworks

  • 1. Department of Computer Science, Rayat Shikshan Sanstha's Dada Patil Mahavidyalaya Karjat Dist.- Ahilyanagar

Description

Nowadays in the digital world, an enormous amount of data is generated in one second in different forms such as text, images, audio and videos. The current data processing system is not capable of processing such huge and dynamic data. The digital transformation sees the organizations produce huge volumes of structured and unstructured data, which comes in the form of social media platforms, financial transactions, healthcare systems, IoT devices, and online services among others. The use of traditional database system to manage and analyse this fast-growing data has been a significant challenge owing to the constraints in the level of scalability, storage capacity, and processing speed. In order to eliminate these problems, Big Data technologies offer distributed storage and parallel processing.

In this paper, a scalable big data analysis system with big data technologies and machine learning algorithms is provided. Its distributed platform such as Apache Hadoop and Apache Spark is used to process big data fast and efficiently. An artificial intelligence model is created to identify trends in the data and make valuable predictive decisions. The outcome demonstrates that such a system is quicker, performs better with large data and provides more precise predictions.

Files

24. Raut Nikita Navanath.pdf

Files (287.2 kB)

Name Size
md5:7facd6533d86bd9697f60cc0414129bd
287.2 kB Preview Download

Additional details

References

  • 1. Jeffrey Dean and Sanjay Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Communications of the ACM, vol. 51, no. 1, pp. 107–113, 2008. 2. Tom White, Hadoop: The Definitive Guide, 4th ed., Sebastopol, CA: O'Reilly Media, 2015. 3. Matei Zaharia et al., "Apache Spark: A Unified Engine for Big Data Processing," Communications of the ACM, vol. 59, no. 11, pp. 56–65, 2016. 4. Ian Goodfellow, Yoshua Bengio, and Aaron Courville, Deep Learning, Cambridge, MA: MIT Press, 2016.