Development of a hyperparameter optimization method for recommendatory models based on matrix factorization
Description
Many advanced recommendatory models are implemented using matrix factorization algorithms. Experiments show that the quality of their performance depends significantly on the selected hyperparameters. Analysis of the effectiveness of using various methods for solving this problem of optimizing hyperparameters was made. It has shown that the use of classical Bayesian optimization which treats the model as a «black box» remains the standard solution. However, the models based on matrix factorization have a number of characteristic features. Their use makes it possible to introduce changes in the optimization process leading to a decrease in the time required to find the sought points without losing quality.
Modification of the Gaussian process core which is used as a surrogate model for the loss function when performing the Bayesian optimization was proposed. The described modification at first iterations increases the variance of the values predicted by the Gaussian process over a given region of the hyperparameter space. In some cases, this makes it possible to obtain more information about the real form of the investigated loss function in less time.
Experiments were carried out using well-known data sets for recommendatory systems. Total optimization time when applying the modification was reduced by 16 % (or 263 seconds) at best and remained the same at worst (less than 1-second difference). In this case, the expected error of the recommendatory model did not change (the absolute difference in values is two orders of magnitude lower than the value of error reduction in the optimization process). Thus, the use of the proposed modification contributes to finding a better set of hyperparameters in less time without loss of quality
Files
Development of a hyperparameter optimization method for recommendatory models based on matrix factorization.pdf
Files
(462.8 kB)
| Name | Size | Download all |
|---|---|---|
|
md5:6c4a68af8c80385b18f778b2d48ed988
|
462.8 kB | Preview Download |
Additional details
References
- Ricci, F., Rokach, L., Shapira, B., Kantor, P. B. (Eds.) (2011). Recommender Systems Handbook. Springer, 842. doi: https://doi.org/10.1007/978-0-387-85820-3
- Rendle, S., Zhang, L., Koren, Y. (2019). On the Difficulty of Evaluating Baselines: A Study on Recommender Systems. arXiv.org. Available at: https://arxiv.org/abs/1905.01395
- Dacrema, M. F., Cremonesi, P., Jannach, D. (2019). Are we really making much progress? A worrying analysis of recent neural recommendation approaches. Proceedings of the 13th ACM Conference on Recommender Systems. doi: https://doi.org/10.1145/3298689.3347058
- Aggarwal, C. C. (2016). Recommender Systems. Springer, 498. doi: https://doi.org/10.1007/978-3-319-29659-3
- Fathan, G., Bharata Adji, T., Ferdiana, R. (2018). Impact of Matrix Factorization and Regularization Hyperparameter on a Recommender System for Movies. Proceeding of the Electrical Engineering Computer Science and Informatics, 5 (5), 113–116. doi: https://doi.org/10.11591/eecsi.v5i5.1685
- Harper, F. M., Konstan, J. A. (2016). The MovieLens Datasets. ACM Transactions on Interactive Intelligent Systems, 5 (4), 1–19. doi: https://doi.org/10.1145/2827872
- Galuzzi, B. G., Giordani, I., Candelieri, A., Perego, R., Archetti, F. (2020). Hyperparameter optimization for recommender systems through Bayesian optimization. Computational Management Science, 17 (4), 495–515. doi: https://doi.org/10.1007/s10287-020-00376-3
- Nechaev, A. A., Meltsov, V. Yu. (2021). Investigating the hyperparameter configuration space of matrix factorization recommendation models. Nauchno-tekhnicheskiy vestnik Povolzh'ya, 5, 96–100. Available at: https://elibrary.ru/item.asp?id=46124660
- Tran, T., Lee, K., Liao, Y., Lee, D. (2018). Regularizing Matrix Factorization with User and Item Embeddings for Recommendation. Proceedings of the 27th ACM International Conference on Information and Knowledge Management. doi: https://doi.org/10.1145/3269206.3271730
- Feurer, M., Hutter, F. (2019). Hyperparameter Optimization. The Springer Series on Challenges in Machine Learning, 3–33. doi: https://doi.org/10.1007/978-3-030-05318-5_1
- Snoek, J., Larochelle, H., Adams, R. P. (2012). Practical Bayesian Optimization of Machine Learning Algorithms. arXiv.org. Available at: https://arxiv.org/abs/1206.2944
- McLeod, M., Osborne, M. A., Roberts, S. J. (2018). Practical Bayesian Optimization for Variable Cost Objectives. arXiv.org. Available at: https://arxiv.org/abs/1703.04335
- Chen, Y., Chen, B., He, X., Gao, C., Li, Y., Lou, J.-G., Wang, Y. (2019). λOpt. Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. doi: https://doi.org/10.1145/3292500.3330880
- Xian, Z., Li, Q., Li, G., Li, L. (2017). New Collaborative Filtering Algorithms Based on SVD++ and Differential Privacy. Mathematical Problems in Engineering, 2017, 1–14. doi: https://doi.org/10.1155/2017/1975719
- Shi, W., Wang, L., Qin, J. (2020). User Embedding for Rating Prediction in SVD++-Based Collaborative Filtering. Symmetry, 12 (1), 121. doi: https://doi.org/10.3390/sym12010121
- Cline, A. K., Dhillon, I. S. (2006). Computation of the Singular Value Decomposition. Handbook of Linear Algebra, 45-1–45-13. doi: https://doi.org/10.1201/9781420010572-45
- Nechaev, A. (2021). Speeding up Bayesian Optimization of Matrix Factorization Recommender Models Hyperparameters. GitHub. Available at: https://github.com/dapqa/speeding-up-bo-for-cf-public
- Optimized-for-speed Eigen implementations of SVD, SVD++ and TimeSVD++ algorithms. Available at: https://github.com/dapqa/svdistic
- Nogueira, F. (2014). Bayesian Optimization: Open source constrained global optimization tool for Python. Available at: https://github.com/fmfn/BayesianOptimization