Towards a New Model of Real-Time Graph Computation on Social Media
Authors/Creators
Description
Real-time analysis of social media data is increasingly constrained by the scalability, memory efficiency, and latency limitations of existing graph-processing frameworks, which exhibit different trade-offs in meeting real-time requirements. This study adopts a conceptual, literature-based comparative approach to examine four representative frameworks: Apache Hadoop with Giraph, Apache Spark with GraphX, Neo4j, and Apache Flink with Gelly, focusing on scalability, real-time processing, and memory management. Based on the identified limitations, a conceptual architecture is proposed that integrates dynamic graph partitioning, in-memory processing, and distributed memory management to support real-time graph computation in Social Media Networks (SMNs). The primary contribution lies in coordinating these established techniques within a unified processing pipeline designed to address the scalability and latency demands of social media graphs. The proposed architecture remains at the conceptual design stage and has not undergone direct distributed benchmarking. Nevertheless, implementation patterns from two related social-media graph studies provide an indicative, prototype-informed evaluation envelope, suggesting approximately 300–400 nodes, 1,100–1,400 edges, 150–220 KB of graph-memory context, sub-second average API response times, and graph recomputation times of approximately 50–330 seconds under study-scale conditions. These figures are estimates for evaluation planning rather than measured results or controlled comparisons among the four frameworks. Accordingly, the study establishes a structured comparative foundation and identifies a staged validation process involving prototype development and performance evaluation using public social-network datasets. Its principal contribution is a technically motivated architectural design that informs future empirical investigation while explicitly acknowledging the absence of direct experimental validation and controlled cross-framework performance measurements.
Files
65.pdf
Additional details
References
- 1. Alghamdi, A.M.; Pileggi, S.F.; Sohaib, O. Social Media Analysis to Enhance Sustainable Knowledge Management: A Concise Literature Review. Sustainability 2023, 15, 9957. https://doi.org/10.3390/su15139957 DOI: https://doi.org/10.3390/su15139957 2. Martín-Rojas, R.; García-Morales, V.J.; Garrido-Moreno, A.; Salmador-Sánchez, M.P. Social Media Use and the Challenge of Complexity: Evidence from the Technology Sector. J. Bus. Res. 2021, 129, 621–640. https://doi.org/10.1016/j.jbusres.2019.12.026 DOI: https://doi.org/10.1016/j.jbusres.2019.12.026 3. Madi, S.A.; Pirrò, G. Community Deception in Directed Influence Networks. Soc. Netw. Anal. Min. 2023, 13, 122. https://doi.org/10.1007/s13278-023-01122-8 DOI: https://doi.org/10.1007/s13278-023-01122-8 4. Bello, A.; Olufemi, K. Artificial Intelligence in Fraud Prevention: Exploring Techniques and Applications, Challenges, and Opportunities. Comput. Sci. IT Res. J. 2024, 5, 1505–1520. https://doi.org/10.51594/csitrj.v5i6.1252 DOI: https://doi.org/10.51594/csitrj.v5i6.1252 5. Hlaoua, L. An Overview of Aggregation Methods for Social Networks Analysis. Knowl. Inf. Syst. 2024, 67, 1–28. https://doi.org/10.1007/s10115-024-02296-z DOI: https://doi.org/10.1007/s10115-024-02296-z 6. Chen, W.; Liu, S.; Yang, Y.; Hu, W.; Yu, J. Dynamic Edge Loading Balancing with Edge Node Activity Prediction and Accelerating the Model Convergence. Sensors 2025, 25, 1491. https://doi.org/10.3390/s25051491 DOI: https://doi.org/10.3390/s25051491 7. Alhaj Ali, A.; Logofătu, D. An Analysis on Graph-Processing Frameworks: Neo4j and Spark GraphX. In IFIP Advances in Information and Communication Technology; Springer International Publishing: Cham, Switzerland, 2022; pp. 461–470. https://doi.org/10.1007/978-3-031-08333-4_37 DOI: https://doi.org/10.1007/978-3-031-08333-4_37 8. Jabeen, H. Big Data Outlook, Tools, and Architectures. In Knowledge Graphs and Big Data Processing; Janev, V., Graux, D., Jabeen, H., Sallinger, E., Eds.; Springer: Cham, Switzerland, 2020; Volume 12072, pp. 35–55. https://doi.org/10.1007/978-3-030-53199-7_3 DOI: https://doi.org/10.1007/978-3-030-53199-7_3 9. Manocha, A. Optimizing Data Supply and Memory Management for Graph Applications in Post-Moore Hardware-Software Systems. Ph.D. Thesis, Princeton University, Princeton, NJ, USA, 2025. 10. Gogri, D. Advanced and Scalable Real-Time Data Analysis Techniques for Enhancing Operational Efficiency, Fault Tolerance, and Performance Optimization in Distributed Computing Systems and Architectures. 2023. https://doi.org/10.13140/RG.2.2.23876.23681 11. Nooribakhsh, M.; Fernández-Diego, M.; González-Ladrón-De-Guevara, F.; Mollamotalebi, M. Community Detection in Social Networks Using Machine Learning: A Systematic Mapping Study. Knowl. Inf. Syst. 2024, 66, 7205–7259. https://doi.org/10.1007/s10115-024-02201-8 DOI: https://doi.org/10.1007/s10115-024-02201-8 12. Shakhovska, N.; Boyko, N.; Zasoba, Y.; Benova, E. Big Data Processing Technologies in Distributed Information Systems. Procedia Comput. Sci. 2019, 160, 561–566. https://doi.org/10.1016/j.procs.2019.11.047 DOI: https://doi.org/10.1016/j.procs.2019.11.047 13. Moon, S.; Iacobucci, D. Social Media Analytics and Its Applications in Marketing. Found. Trends Mark. 2022, 15, 213–292. https://doi.org/10.1561/1700000073 DOI: https://doi.org/10.1561/1700000073 14. Song, S.; Choi, J.; Cha, D.; Lee, H.; Choi, D.; Lim, J.; Bok, K.; Yoo, J. Large-Scale Dynamic Graph Processing with Graphic Processing Unit-Accelerated Priority-Driven Differential Scheduling and Operation Reduction. Appl. Sci. 2025, 15, 3172. https://doi.org/10.3390/app15063172 DOI: https://doi.org/10.3390/app15063172 15. Marx, J.; Mirbabaie, M. The Investigator's Dilemma—A Review of Social Media Analytics Research Ethics in Information Systems. Australas. J. Inf. Syst. 2022, 26. https://doi.org/10.3127/ajis.v26i0.3287 DOI: https://doi.org/10.3127/ajis.v26i0.3287 16. Bas, S.; Kaya, K.; Tugay, R.; Oguducu, S.G. Data Augmentation in Graph Neural Networks: The Role of Generated Synthetic Graphs. arXiv 2024. https://doi.org/10.48550/arXiv.2407.14765 17. Goyal, A.; Singh, S.; Sharma, S. Fraud Detection on Social Media Using Data Analytics. Int. J. Eng. Res. Technol. 2020, 9. https://doi.org/10.17577/IJERTV9IS010204 DOI: https://doi.org/10.17577/IJERTV9IS010204 18. Sadri, A.M.; Hasan, S.; Ukkusuri, S.V.; Cebrian, M. Exploring Network Properties of Social Media Interactions and Activities during Hurricane Sandy. Transp. Res. Interdiscip. Perspect. 2020, 6, 100143. https://doi.org/10.1016/j.trip.2020.100143 DOI: https://doi.org/10.1016/j.trip.2020.100143 19. Li, J.; Lai, S.; Shuai, Z.; Tan, Y.; Jia, Y.; Yu, M.; Song, Z.; Peng, X.; Xu, Z.; Ni, Y.; et al. A Comprehensive Review of Community Detection in Graphs. Neurocomputing 2024, 600, 128169. https://doi.org/10.1016/j.neucom.2024.128169 DOI: https://doi.org/10.1016/j.neucom.2024.128169 20. Vidal-Silva, C.L.; Madariaga, E.; Pham, T.; Rubio, J.M.; Urzua, L.A.; Carter, L.; Johnson, F. Advantages of Giraph over Hadoop in Graph Processing. Eng. Technol. Appl. Sci. Res. 2019, 9, 4112–4115. https://doi.org/10.48084/etasr.2715 DOI: https://doi.org/10.48084/etasr.2715 21. Theodorakopoulos, L.; Karras, A.; Theodoropoulou, A.; Kampiotis, G. Benchmarking Big Data Systems: Performance and Decision-Making Implications in Emerging Technologies. Technologies 2024, 12, 217. https://doi.org/10.3390/technologies12110217 DOI: https://doi.org/10.3390/technologies12110217 22. Arora, A.; Dumbrava, S. Eighth Joint Workshop on Graph Data Management Experiences & Systems (GRADES) and Network Data Analytics (NDA). In Proceedings of the International Conference on Management of Data, 2025; pp. 864–865. https://doi.org/10.1145/3722212.3724484 DOI: https://doi.org/10.1145/3722212.3724484 23. Saad, M.; Zhang, Y.; Tian, J.; Jia, J. A Graph Database for Life Cycle Inventory Using Neo4j. J. Clean. Prod. 2023, 393, 136344. https://doi.org/10.1016/j.jclepro.2023.136344 DOI: https://doi.org/10.1016/j.jclepro.2023.136344 24. Lopes, A.; Rodrigues, D.; Saraiva, J.; Abbasi, M.; Martins, P.; Wanzeller, C. Scalability and Performance Evaluation of Graph Database Systems: A Comparative Study of Neo4j, JanusGraph, Memgraph, NebulaGraph, and TigerGraph. In Proceedings of the 2023 Second International Conference on Smart Technologies for Smart Nation (SmartTechCon), 2023; pp. 537–542. https://doi.org/10.1109/SmartTechCon57526.2023.10391694 DOI: https://doi.org/10.1109/SmartTechCon57526.2023.10391694 25. Fan, W.; He, T.; Lai, L.; Li, X.; Li, Y.; Li, Z.; Qian, Z.; Tian, C.; Wang, L.; Xu, J.; et al. GraphScope: A Unified Engine for Big Graph Processing. Proc. VLDB Endow. 2021, 14, 2879–2892. https://doi.org/10.14778/3476311.3476369 DOI: https://doi.org/10.14778/3476311.3476369 26. Rampášek, L.; Galkin, M.; Dwivedi, V.P.; Luu, A.T.; Wolf, G.; Beaini, D. Recipe for a General, Powerful, Scalable Graph Transformer. arXiv 2022. https://arxiv.org/abs/2205.12454 DOI: https://doi.org/10.52202/068431-1054 27. Vatter, J.; Mayer, R.; Jacobsen, H. The Evolution of Distributed Systems for Graph Neural Networks and Their Origin in Graph Processing and Deep Learning: A Survey. ACM Comput. Surv. 2024, 56, 6. https://doi.org/10.1145/3597428 DOI: https://doi.org/10.1145/3597428 28. Sakr, S.; Bonifati, A.; Voigt, H.; Iosup, A.; Ammar, K.; Angles, R.; Aref, W.G.; Arenas, M.; Besta, M.; Boncz, P.A.; et al. The Future Is Big Graphs: A CommunityView on Graph Processing Systems. Commun. ACM 2021, 64, 62–71. https://doi.org/10.1145/3434642 DOI: https://doi.org/10.1145/3434642 29. Khan, H.M.S.; Hayat, C.M.A.; Tayyab, H.; Ali, K. An Enhanced Cost Effective and Scalable Network Architecture for Data Centers. Spectr. Eng. Sci. 2024, 2, 1–32. https://www.sesjournal.com/index.php/1/article/view/61 30. Mittal, M.; Kobielnik, M.; Gupta, S.; Cheng, X.; Wozniak, M. An Efficient Quality of Services Based Wireless Sensor Network for Anomaly Detection Using Soft Computing Approaches. J. Cloud Comput. 2022, 11, 70. https://doi.org/10.1186/s13677-022-00344-z DOI: https://doi.org/10.1186/s13677-022-00344-z 31. Chen, Z.; Liang, K.; Yuan, L.; Zhang, W.; Yang, Z. Recent Advances in Efficient Dynamic Graph Processing. Appl. Sci. 2025, 15, 6003. https://doi.org/10.3390/app15116003 DOI: https://doi.org/10.3390/app15116003 32. Zhang, H.; Zang, Z.; Zhu, H.; Uddin, M.I.; Amin, M.A. Big Data-Assisted Social Media Analytics for Business Model for Business Decision Making System Competitive Analysis. Inf. Process. Manag. 2022, 59, 102762. https://doi.org/10.1016/j.ipm.2021.102762 DOI: https://doi.org/10.1016/j.ipm.2021.102762 33. Oliveira, E.A.R.d.; Rocha, A.R.d.; Mattoso, M.; Delicato, F.C. Latency and Energy-Awareness in Data Stream Processing for Edge Based IoT Systems. J. Grid Comput. 2022, 20, 27. https://doi.org/10.1007/s10723-022-09611-4 DOI: https://doi.org/10.1007/s10723-022-09611-4 34. Liu, Y.; Yu, T.; Meng, Q.; Liu, Q. Flow Optimization Strategies in Data Center Networks: A Survey. J. Netw. Comput. Appl. 2024, 226, 103883. https://doi.org/10.1016/j.jnca.2024.103883 DOI: https://doi.org/10.1016/j.jnca.2024.103883 35. Çatalyürek, Ü.V.; Devine, K.D.; Faraj, M.F.; Gottesbüren, L.; Heuer, T.; Meyerhenke, H.; Sanders, P.; Schlag, S.; Schulz, C.; Seemaier, D.; et al. More Recent Advances in (Hyper)Graph Partitioning. ACM Comput. Surv. 2023, 55, 253. https://doi.org/10.1145/3571808 DOI: https://doi.org/10.1145/3571808 36. Jung, S.; Lee, H.; Myung, S.; Kim, H.; Yoon, S.K.; Kwon, S.-W.; Ju, Y.; Kim, M.; Yi, W.; Han, S.; et al. A Crossbar Array of Magnetoresistive Memory Devices for In-Memory Computing. Nature 2022, 601, 211–216. https://doi.org/10.1038/s41586-021-04196-6 DOI: https://doi.org/10.1038/s41586-021-04196-6 37. Ariyoshi, R.; Li, A.; Hasegawa, M.; Ohtsuki, T. Energy-Efficient Resource Allocation Scheme Based on Reinforcement Learning in Distributed LoRa Networks. Sensors 2025, 25, 4996. https://doi.org/10.3390/s25164996 DOI: https://doi.org/10.3390/s25164996 38. Moon, S.; Lim, Y. Task Migration with Partitioning for Load Balancing in Collaborative Edge Computing. Appl. Sci. 2022, 12, 1168. https://doi.org/10.3390/app12031168 DOI: https://doi.org/10.3390/app12031168 39. Alwaisi, S.S.A.; Abbood, M.N.; Jalil, L.F.; Kasim, S.; Mohd Fudzee, M.F.; Hadi, R.; Ismail, M.A. A Review on Big Data Stream Processing Applications: Contributions, Benefits, and Limitations. JOIV Int. J. Inform. Vis. 2021, 5, 456–460. https://doi.org/10.30630/joiv.5.4.737 DOI: https://doi.org/10.30630/joiv.5.4.737 40. Mercado, R.; Bjerrum, E.J.; Engkvist, O. Exploring Graph Traversal Algorithms in Graph-Based Molecular Generation. J. Chem. Inf. Model. 2022, 62, 2093–2100. https://doi.org/10.1021/acs.jcim.1c00777 DOI: https://doi.org/10.1021/acs.jcim.1c00777 41. Zhang, T.; Gao, Y.; Zhao, J.; Chen, L.; Jin, L.; Yang, Z.; Cao, B.; Fan, J. Efficient Exact and Approximate Betweenness Centrality Computation for Temporal Graphs. In Proceedings of the ACM Web Conference 2024; ACM: New York, NY, USA, 2024; pp. 2395–2406. https://doi.org/10.1145/3589334.3645438 DOI: https://doi.org/10.1145/3589334.3645438 42. Khan, M.N. Mastering Real-Time Data Processing Applications: Optimization Strategies for Peak Performance. Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol. 2024, 10, 895–904. https://doi.org/10.32628/CSEIT241051078 DOI: https://doi.org/10.32628/CSEIT241051078 43. Wang, G.; Chen, L.; Dikshit, A.; Gustafson, J.; Chen, B.; Sax, M.J.; Roesler, J.; Blee-Goldman, S.; Cadonna, B.; Mehta, A.; et al. Consistency and Completeness: Rethinking Distributed Stream Processing in Apache Kafka. In Proceedings of the 2021 International Conference on Management of Data (SIGMOD '21), Virtual Event, China, 2021; pp. 2602–2613. https://doi.org/10.1145/3448016.3457556 DOI: https://doi.org/10.1145/3448016.3457556 44. Al-Okaily, M.; Al-Okaily, A. Financial Data Modeling: An Analysis of Factors Influencing Big Data Analytics-Driven Financial Decision Quality. J. Model. Manag. 2024, 19, 301–321. https://doi.org/10.1108/JM2-08-2023-0183 DOI: https://doi.org/10.1108/JM2-08-2023-0183 45. Hooda, A.; Hooda, A.; Yadav, D. Adaptive Real-Time Big Data Processing Framework: A Machine Learning and Reinforcement Learning Approach Using Random Forest and Q-Learning for Dynamic Resource Management. Res. Sq. 2024. https://doi.org/10.21203/rs.3.rs-4962286/v2 DOI: https://doi.org/10.21203/rs.3.rs-4962286/v2 46. Li, M.; Lu, S.; Zhang, L.; Zhang, Y.; Zhang, B. A Community Detection Method for Social Network Based on Community Embedding. IEEE Trans. Comput. Soc. Syst. 2021, 8, 308–318. https://doi.org/10.1109/TCSS.2021.3050397 DOI: https://doi.org/10.1109/TCSS.2021.3050397 47. Zhou, K.; Dong, Y.; Wang, K.; Lee, W.S.; Hooi, B.; Xu, H.; Feng, J. Understanding and Resolving Performance Degradation in Deep Graph Convolutional Networks. In Proceedings of the 30th ACM International Conference on Information and Knowledge Management (CIKM 2021), Virtual Event, 2021; pp. 2728–2737. https://doi.org/10.1145/3459637.3482488 DOI: https://doi.org/10.1145/3459637.3482488 48. Shukla, S.; Hassan, M.F.; Tran, D.C.; Akbar, R.; Paputungan, I.V.; Khan, M.K. Improving Latency in Internet-of-Things and Cloud Computing for Real-Time Data Transmission: A Systematic Literature Review (SLR). Clust. Comput. 2021, 26, 1. https://doi.org/10.1007/s10586-021-03279-3 DOI: https://doi.org/10.1007/s10586-021-03279-3 49. Lin, Y.; Li, H.; Li, A.; Shi, Y.; Zhuang, S. Application of AI-Driven Cloud Services in Intelligent Agriculture Pest and Disease Prediction. Appl. Comput. Eng. 2024, 67, 61–67. https://doi.org/10.54254/2755-2721/67/2024MA0063 DOI: https://doi.org/10.54254/2755-2721/67/2024MA0063 50. Albach, M.; Wright, J.R. The Role of Accuracy in Algorithmic Process Fairness across Multiple Domains. In Proceedings of the 22nd ACM Conference on Economics and Computation (EC '21), 2021. https://doi.org/10.1145/3465456.3467620 DOI: https://doi.org/10.1145/3465456.3467620 Downloads PDF Published 2026-10-12 How to Cite AL-Atroshi, C., & Sadiq Ahmed, N. (2026). Towards a New Model of Real-Time Graph Computation on Social Media: Comparative Analysis, Challenges, and Future Directions. Dasinya Journal for Engineering and Informatics, 2(4). https://doi.org/10.65542/djei.v2i4.65 More Citation Formats Issue Vol. 2 No. 4 (2026): Edition Five Section Articles License Copyright (c) 2026 Chiai AL-Atroshi, Nawzat Sadiq Ahmed Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License. Crossref 0 Scopus 0 Google Scholar Europe PMC 0 Similar Articles Ziyad Hazim Abduljabbar , AI-Enhanced AES Encryption for Kurdish Unicode Texts , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 2 (2026): Edition Three Ribwar Rashid Aziz, Dana Lattef Hussein, Rebwar Khalid Mohammed, Advanced MySQL Binary Log-Based Data Recovery , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 4 (2026): Edition Five Omar Mohammed Amin, Rebin Abdulkareem Hamaamin, Araz Ibrahim Mustafa, Aso M. Aladdin, Enhanced Diabetes Prediction , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 4 (2026): Edition Five Nizar Faqishafyee, Emad Sadiq, Harwan Taha, Adel Al-zebari, AI-Based Adaptive Power Factor Correction for Reactive Power Reduction in Distribution Systems , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 4 (2026): Edition Five Dawlat Abdulkarim Ali, Haval Tariq Sadeeq, Deep Learning Frameworks for Dental Anomaly and Disease Classification , Dasinya Journal for Engineering and Informatics: Vol. 1 No. 1 (2025): Edition One Sirwan Younis Abdullah, Mayyadah Ramiz Mahmood, Comparative Analysis of Deep Neural Networks for Online Facial Expression Recognition and Facial Shape Classification , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 3 (2026): Edition Four Yahya Salim Ahmed, Huda Dham Bader, Aymen Dhafer Abdul-nafa, Afraa Alameen, Remote Vital Sign Monitoring System Utilizing LoRa , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 2 (2026): Edition Three Rebaz Hamza Jahwar, Lokman Aziz Abdulkareem, Analysing Gas-Liquid Flow Effects on the Performance of ESP by Nodal Analysis , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 3 (2026): Edition Four Iman Youssif Ibrahim, Dindar M. Ahmed, Adaptive Multi-Task Learning for Urban Air Quality Assessment , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 3 (2026): Edition Four Ronak Hassan, Jihan Ahmed, Bitcoin Price Prediction Using Blockchain Transaction Data and Machine Learning Models , Dasinya Journal for Engineering and Informatics: Vol. 2 No. 1 (2026): Edition Two