COMPUTER DOMAIN
Published March 21, 2024 | Version v1
Journal article Open

Cloud Canvas: Orchestrating Distributed Image Processing

Creators

Description

In this paper, we introduce a Distributed Image Processing Pipeline, leveraging deep learning and distributed computing for real-time image analysis. The pipeline aims to optimize resource utilization, implement advanced deep learning models, ensure data integrity, and minimize communication overhead. By distributing image processing tasks across a cluster of computing nodes, it enables faster processing and scalability. Integration of Apache Kafka and Apache Zookeeper enhances real-time stability and data distribution. The system's flexibility allows customization with diverse deep learning models and processing tasks. Applications span medical imaging, autonomous vehicles, satellite imagery, and industrial quality control, addressing complex image analysis challenges with accuracy and efficiency.

Files

Cloud Canvas -Formatted Paper.pdf

Files (454.4 kB)

Name Size Download all
md5:5f139ec4f37d331a184108681bae7216
454.4 kB Preview Download

Additional details

References

  • 1. Yang, T., Xu, Q., Meng, F., & Zhang, S. (2022). Distributed real-time image processing of formation flying SAR based on embedded GPUs. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 15, 6495-6505.
  • 2. Dong, L., Lin, Z., Liang, Y., He, L., Zhang, N., Chen, Q., ... & Izquierdo, E. (2016). A hierarchical distributed processing framework for big image data. IEEE Transactions on Big Data, 2(4), 297-309.
  • 3. Han, S. S., Kim, Y. K., Jeon, Y. B., Park, J., Park, D. S., Hwang, D., & Jeong, C. S. (2020). Distributed deep learning platform for pedestrian detection on IT convergence environment. The Journal of Supercomputing, 76, 5460-5485.
  • 4. Zhu, M., & Chen, Q. (2020). Big data image classification based on distributed deep representation learning model. IEEE Access, 8, 133890-133904.
  • 5. Kim, Y. K., Kim, Y., & Jeong, C. S. (2018). RIDE: real-time massive image processing platform on distributed environment. EURASIP Journal on Image and Video Processing, 2018, 1-13.
  • 6. Kim, Y. K., & Jeong, C. S. (2017, January). Large scale image processing in real-time environments with Kafka. In Proceedings of the 6th AIRCC International Conference on Parallel, Distributed Computing Technologies and Applications (PDCTA) (pp. 207-215).
  • 7. Rostanski, M., Grochla, K., & Seman, A. (2014, September). Evaluation of highly available and fault-tolerant middleware clustered architectures using RabbitMQ. In 2014 federated conference on computer science and information systems (pp. 879-884). IEEE.
  • 8. Hunt, P., Konar, M., Junqueira, F. P., & Reed, B. (2010). {ZooKeeper}: Wait-free coordination for internet-scale systems. In 2010 USENIX Annual Technical Conference (USENIX ATC 10).
  • 9. Kreps, J., Narkhede, N., & Rao, J. (2011, June). Kafka: A distributed messaging system for log processing. In Proceedings of the NetDB (Vol. 11, No. 2011, pp. 1-7).
  • 10. Hintjens, P. (2013). ZeroMQ: messaging for many applications. " O'Reilly Media, Inc.".