Published July 30, 2024 | Version CC-BY-NC-ND 4.0
Journal article Open

Transformer-Based Methods for Water Level Prediction: A Case Study of the Kien Giang River, Quang Binh Province

  • 1. Department of Artificial Intelligence, Thuyloi University, 175 Tay Son, Dong Da, 100000, Hanoi, Vietnam.
  • 1. Hanoi University of Science and Technology, No. 1 Dai Co Viet, Hai Ba Trung, 100000, Hanoi, Vietnam.
  • 2. Department of Artificial Intelligence, Thuyloi University, 175 Tay Son, Dong Da, 100000, Hanoi, Vietnam.

Description

Abstract: Accurate and timely water level prediction is of paramount importance in various applications, including flood forecasting, hydroelectric power management, and environmental monitoring. Traditional recurrent neural network (RNN)-based methods have been widely used for this task. However, recent advancements in long-term time-series forecasting have introduced transformer-based models that have significantly improved the performance in time-series prediction tasks. In this research, we investigate the application of transformer-based models to the task of water level prediction, specifically focusing on the Nhat Le River Basin. We conducted multiple experiments with different test cases and various model architectures, providing specific analyses of the model’s prediction capabilities. The transformer-based models consistently outperformed conventional RNN-based methods across a range of evaluation metrics, including root mean square error (RMSE), mean absolute error (MAE), and coefficient of determination (R2). Moreover, these models exhibited excellent flood peak prediction accuracy, with errors consistently below 0.02 meters. The robustness and scalability of transformer-based models make them promising for accurate water-level predictions in real-world applications.

Files

H993613080724.pdf

Files (626.2 kB)

Name Size Download all
md5:a30024337d33cd6561bae9728d030902
626.2 kB Preview Download

Additional details

Identifiers

Dates

Accepted
2024-07-15
Manuscript received on 30 June 2024 | Revised Manuscript received on 08 July 2024 | Manuscript Accepted on 15 July 2024 | Manuscript published on 30 July 2024.

References

  • Ly N, Duong N, and Dai N, "Khi hau va thuy van tinh Quang Binh", Science and Technics Publishing House Science and Technics Publishing House, 2013
  • Vaswani, Ashish, et al. "Attention is all you need." Advances in neural information processing systems 30 (2017).
  • H. Wu, J. Xu, J. Wang, and M. Long, "Autoformer: Decomposition Transformers with Auto-Correlation for Long-Term Series Forecasting," Adv. Neural Inf. Process. Syst., vol. 27, no. NeurIPS, pp. 22419-22430, 2021.
  • Haoyi Zhou and Shanghang Zhang and Jieqi Peng and Shuai Zhang and Jianxin Li and Hui Xiong and Wancai Zhang, "Informer: Beyond Efficient Transformer for Long Sequence TimeSeries Forecasting," The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, vol. 35, 11106-11115, 2021. https://doi.org/10.1609/aaai.v35i12.17325
  • Montgomery, Douglas C., Elizabeth A. Peck, and G. Geoffrey Vining. Introduction to linear regression analysis. John Wiley & Sons, 2021. 6. Breiman, Leo. "Random forests." Machine learning 45 (2001): 5-32.
  • Breiman, Leo. "Random forests." Machine learning 45 (2001): 5-32. https://doi.org/10.1023/A:1010933404324
  • Friedman, Jerome H. "Greedy function approximation: a gradient boosting machine." Annals of statistics (2001): 1189-1232. https://doi.org/10.1214/aos/1013203451
  • Zhang, G. Peter. "Time series forecasting using a hybrid ARIMA and neural network model." Neurocomputing 50 (2003): 159-175. https://doi.org/10.1016/S0925-2312(01)00702-
  • Ouyang, Zuokun, Philippe Ravier, and Meryem Jabloun. "STL decomposition of time series can benefit forecasting done by statistical methods but not by machine learning ones." Engineering Proceedings 5.1 (2021): 42. https://doi.org/10.3390/engproc2021005042
  • Zhou, Tian, et al. "Fedformer: Frequency enhanced decomposed transformer for long-term series forecasting." International Conference on Machine Learning. PMLR, 2022.
  • Liu, Shizhan, et al. "Pyraformer: Low-complexity pyramidal attention for long-range time series modeling and forecasting." International conference on learning representations. 2021.
  • Athiyarath, Srihari, Mousumi Paul, and Srivatsa Krishnaswamy. "A comparative study and analysis of time series forecasting techniques." SN Computer Science 1.3 (2020): 175. https://doi.org/10.1007/s42979-020-00180-5
  • Mehtab, Sidra, and Jaydip Sen. "Analysis and forecasting of financial time series using CNN and LSTM-based deep learning models." Advances in Distributed Computing and Machine Learning: Proceedings of ICADCML 2021. Springer Singapore, 2022. https://doi.org/10.1007/978-981-16-4807-6_39
  • Yamak, Peter T., Li Yujian, and Pius K. Gadosey. "A comparison between arima, lstm, and gru for time series forecasting." Proceedings of the 2019 2nd international conference on algorithms, computing and artificial intelligence. 2019. https://doi.org/10.1145/3377713.3377722
  • Elsworth, Steven, and Stefan Güttel. "Time series forecasting using LSTM networks: A symbolic approach." arXiv preprint arXiv:2003.05672 (2020).
  • Salinas, David, et al. "DeepAR: Probabilistic forecasting with autoregressive recurrent networks." International Journal of Forecasting 36.3 (2020): 1181-1191. https://doi.org/10.1016/j.ijforecast.2019.07.001
  • Zeng, Ailing, et al. "Are transformers effective for time series forecasting?." Proceedings of the AAAI conference on artificial intelligence. Vol. 37. No. 9. 2023.
  • Hieu, T. T., Chieu, T. Q., Quang, D. N., & Hieu, N. D. (2023). Water level prediction using deep learning models: A case study of the Kien Giang River, Quang Binh province. River, 1–12. https://doi.org/10.1002/rvr2.63
  • Dakshin, D., Rupesh, V. R., & Kumar, S. P. (2019). Water Hazard Prediction using Machine Learning. In International Journal of Innovative Technology and Exploring Engineering (Vol. 9, Issue 1, pp. 1451–1457). https://doi.org/10.35940/ijitee.a4245.119119
  • Singh, N., & Panda, S. P. (2020). Stimulating Deep Learning Network on Graphical Processing Unit To Predict Water Level. In International Journal of Engineering and Advanced Technology (Vol. 9, Issue 4, pp. 1222–1229). https://doi.org/10.35940/ijeat.d8452.049420
  • Perunkolam, A., Baskaran, M. P., Tripathi, S., & Sahu, O. P. (2020). Smart Water Distribution System using Machine Learning and IoT. In International Journal of Recent Technology and Engineering (IJRTE) (Vol. 9, Issue 2, pp. 1189–1194). https://doi.org/10.35940/ijrte.b4115.079220