Solving Partially Monotone Problems with Neural Networks

Authors: Marina Velikova, Hennie Daniels, Ad Feelders

Abstract:

In many applications, it is a priori known that the target function should satisfy certain constraints imposed by, for example, economic theory or a human-decision maker. Here we consider partially monotone problems, where the target variable depends monotonically on some of the predictor variables but not all. We propose an approach to build partially monotone models based on the convolution of monotone neural networks and kernel functions. The results from simulations and a real case study on house pricing show that our approach has significantly better performance than partially monotone linear models. Furthermore, the incorporation of partial monotonicity constraints not only leads to models that are in accordance with the decision maker's expertise, but also reduces considerably the model variance in comparison to standard neural networks with weight decay.

Keywords: Mixture models, monotone neural networks, partially monotone models, partially monotone problems.

Digital Object Identifier (DOI): doi.org/10.5281/zenodo.1077831

References:

[1] A. Ben-David, "Monotonicity Maintenance in Information-Theoretic Machine Learning Algorithms", Machine Learning, 19, (1995), pp. 29- 43
[2] C. Cybenko, "Approximation by Superpositions of a Sigmoidal Function", Mathematics of Control, Signals, and Systems, 2, (1989), pp. 303-314
[3] H.A.M. Daniels, and B. Kamp, "Application of MLP Networks to Bond Rating and House Pricing", Neural Computation and Applications, 8, (1999) pp. 226-234
[4] O. Harrison, and D. Rubinfeld, "Hedonic Prices and The Demand for Clean Air", Journal of Environmental Economics and Management, 53, (1978), pp.81-102
[5] H. Kay, and L.H. Ungar,, "Estimating Monotonic Functions and Their Bounds", AIChE Journal, 46, (2000), pp.2426-2434
[6] H. Mukarjee, and S. Stern, "Feasible Nonparametric Estimation of Multiargument Monotone Functions", Journal of the American Statistical Association, 89, (1994), pp. 77-80
[7] E.A. Nadaraya, "On Estimating Regression", Theory Prob. Applic., 10, (1964), pp.186-90
[8] R. Potharst, and A. Feelders, "Classification trees for problems with monotonicity constraints", SIGKDD Explorations Newsletter, 4, (2002), Issue 1
[9] P.J. Rousseeuw, "Silhouettes: a graphical aid to the interpretation and validation of cluster analysis", Journal of Computational and Applied Mathematics, 20, (1987), pp.53-65
[10] M. Sarfraz, M. Al-Mulhem, and F. Ashraf, "Preserving Monotonic Shape of The Data by Using Piecewise Rational Cubic Functions", Computers and Graphics, 21, (1997), pp.5-14
[11] J. Sill, "Monotonic Networks", Advances in Neural Information Processing Systems, 10, (1998), pp.661-667
[12] S. Wang, "A Neural Network Method of Density Estimation for Univariate Unimodal Data", Neural Computation & Applications, 2, (1994), pp.160-167
[13] G. S. Watson, "Smooth Regression Analysis", Sankhya, Ser. A, 26, (1964), pp. 359-372
[14] Wu, C.F.J., and M. Hamada, Experiments: Planning, Analysis, and Parameter Design Optimization, Wiley Series in Probability and Statistics, John Wiley & Sons, Inc. New York, (2000).