Published October 28, 2007 | Version 2246

Mixtures of Monotone Networks for Prediction

Description

In many data mining applications, it is a priori known that the target function should satisfy certain constraints imposed by, for example, economic theory or a human-decision maker. In this paper we consider partially monotone prediction problems, where the target variable depends monotonically on some of the input variables but not on all. We propose a novel method to construct prediction models, where monotone dependences with respect to some of the input variables are preserved by virtue of construction. Our method belongs to the class of mixture models. The basic idea is to convolute monotone neural networks with weight (kernel) functions to make predictions. By using simulation and real case studies, we demonstrate the application of our method. To obtain sound assessment for the performance of our approach, we use standard neural networks with weight decay and partially monotone linear models as benchmark methods for comparison. The results show that our approach outperforms partially monotone linear models in terms of accuracy. Furthermore, the incorporation of partial monotonicity constraints not only leads to models that are in accordance with the decision maker's expertise, but also reduces considerably the model variance in comparison to standard neural networks with weight decay.

Files

2246.pdf

Files (319.5 kB)

Name Size
md5:a21ff0d65b7f33917f42166b7eb31639
319.5 kB Preview Download

Additional details

References

  • O. Harrison and D. Rubinfeld. Hedonic prices and the demand for clean air. Journal of Environmental Economics and Management, 53:81┬û102, 1978.
  • H. Mukarjee and S. Stern. Feasible nonparametric estimation of multiargument monotone functions. Journal of the American Statistical Association, 89(425):77┬û80, 1994.
  • H.A.M. Daniels and B. Kamp. Application of MLP networks to bond rating and house pricing. Neural Computing & Applications, 8(3):226┬û 234, 1999.
  • H. Kay and L. H. Ungar. Estimating monotonic functions and their bounds. American Institute of Chemical Engineers (AIChE) Journal, 46(12):2426┬û2434, 2000.
  • J. Sill. Monotonic networks. Advances in Neural Information Processing Systems, 10:661┬û667, 1998.
  • S. Wang. A neural network method of density estimation for univariate unimodal data. Neural Computing & Applications, 2(3):160┬û167, 1994.
  • M. Sarfraz, M. Al-Mulhem, and F. Ashraf. Preserving monotonic shape of the data by using piecewise rational cubic functions. Computers and Graphics, 21:5┬û14, 1997.
  • M. Velikova and H.A.M. Daniels. Decision trees for monotone price models. Computational Management Science, 1(3┬û4):231┬û244, 2004.
  • A. Ben-David. Monotonicity maintenance in information-theoretic machine learning algorithms. Machine Learning, 19(1):29┬û43, 1995. [10] R. Potharst and A. Feelders. Classication trees for problems with monotonicity constraints. SIGKDD Explorations Newsletter, 4(1):1┬û10, 2002. [11] P. J. Rousseeuw. Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. Journal of Computational and Applied Mathematics, 20:53┬û65, 1987. [12] E. A. Nadaraya. On estimating regression. Theory of Probability & Its Applications, 9(1):141┬û142, 1964. [13] G. S. Watson. Smooth regression analysis. Sankhya: The Indian Journal of Statistics, Series A, 26(4):359┬û372, 1964. [14] S. Geman, E. Bienenstock, and R. Doursat. Neural networks and the bias/variance dilemma. Neural Computation, 4(1):1┬û58, 1992. [15] C. F. J. Wu and M. Hamada. Experiments: Planning, Analysis, and Parameter Design Optimization. John Wiley & Sons, New York, wiley series in probability and statistics edition, 2000. [16] M. Velikova, H.A.M. Daniels, and A. Feelders. Solving partially monotone problems with neural networks. In Proceedings of the twelfth International Conference on Computer Science, Vienna, Austria, pages 82┬û87. World Enformatika Society, Turkey, 2006. [17] D. J. Newman, S. Hettich, C. L. Blake, and C. J. Merz. UCI Repository of Machine Learning Databases, 1998. http://www.ics.uci. edu/┬ÿmlearn/MLRepository.html. [18] S. Waugh. Extending and benchmarking cascade-correlation. Phd dissertation, University of Tasmania, Tasmania, Australia, 1995. [19] D. Clark, Z. Schreter, and A. Adams. A quantitative comparison of Dystal and backpropagation. In Proceedings of the seventh Australian Conference Neural Networks, Canberra, Australia, pages 132┬û137. 1996. [20] A. M. Abdelbar. Achieving superior generalisation with a high order neural network. Neural Computing & Applications, 7(2):141┬û146, 1998. [21] W. J. Nash, T. L. Sellers, S. R. Talbot, and W. B. Cawthorn, A. J. anf Ford. The population biology of abalone (halotis species) in tasmania. i blacklip abalone (h. rubra) from the north coast and islands of bass strait. Technical Report 48, Sea Fisheries Division, Marine Research Laboratories-Taroona, Department of Primary Industry and Fisheries, Tasmania, Australia, 1994.