Abstract
The modern engineering design process often relies on numerical analysis codes to evaluate candidate designs, a setup which formulates an optimization problem which involves a computationally expensive black-box function. Such problems are often solved using a algorithm in which a metamodel approximates the true objective function and provides predicted objective values at a lower computational cost. The metamodel is trained using an initial sample of vectors, and this implies that the procedure by which the initial sample is generated can impact the overall effectiveness of the optimization search. Approaches for generating the initial sample include the statistically based design of experiments, and the more recent search-driven sampling which generates the sample vectors with a direct-search optimizer. This study compares these two approaches in terms of their overall impact on the optimization search and formulates guidelines in which scenario is each approach preferable. An extensive analysis shows that: (a) the main factor affecting search-driven sampling is the size of the initial sample, and such methods performed better in large initial samples, (b) design of experiments methods tended to perform better in lower sample sizes, (c) generating a sample which is space-filling improved the overall search effectiveness
Similar content being viewed by others
References
Acar E (2010) Various approaches for constructing an ensemble of metamodels using local measures. Struct Multidiscip Optim 42(6):879–896
Box GEP, Wilson KB (1951) On the experimental attainment of optimum conditions. J R Stat Soc Ser B (Methodological) 13:1–45
Büche D, Schraudolph NN, Koumoutsakos P (2005) Accelerating evolutionary algorithms with Gaussian process fitness function models. IEEE Trans Syst Man Cybernet Part C 35(2):183–194
Chen VP, Barton RR, Meckesheimer M, Tsui KL (2006) A review on design, modeling and applications of computer experiments. IIE Trans 38(4):273–291
Chipperfield A, Fleming P, Pohlheim H, Fonseca C (1994) Genetic algorithm TOOLBOX for use with MATLAB, version 1.2. Department of Automatic Control and Systems Engineering, University of Sheffield, Sheffield
Drela M, Youngren H (2001) XFOIL 6.9 User Primer. Department of Aeronautics and Astronautics, Massachusetts Institute of Technology, Cambridge
El-Betalgy MA, Keane AJ (2001) Evolutionary optimization for computationally expensive problems using Gaussian processes. In: Arbania H (ed) Proceedings of the International Conference on artificial intelligence-IC-AI’2001. CSREA Press, pp 708–714
Fang KT, Lin DKJ, Winker P, Zhang Y (2000) Uniform design: theory and application. Technometrics 42(2):237–248
Finney DJ (1945) Fractional replication of factorial arrangments. Ann Eugen 12:291–301
Fisher RA (1926) The arrangement of field experiments. J Minist Agric 33:503–513
Forrester AIJ, Keane AJ (2008) Recent advances in surrogate-based optimization. Prog Aerosp Sci 45(1–3):50–79
Gorissen D, Dhaene T, De Turck F (2009) Evolutionary model type selection for global surrogate modeling. J Mach Learn Res 10:2039–2078
Grosso A, Jamali A, Locatelli M (2009) Finding maximin latin hypercube designs by iterated local search heuristics. Eur J Oper Res 197(2):541–547
Hawe GI, Sykulski JK (2006) Balancing exploration exploitation using Kriging surrogate models in electromagnetic design optimization. In: Proceedings of the 12th Biennial IEEE Conference on electromagnetic field computation. IEEE, p 226
Hicks RM, Henne PA (1978) Wing design by numerical optimization. J Aircraft 15(7):407–412
Husslage BGM, Rennen G, van Dam ER, den Hertog D (2011) Space-filling latin hypercube designs for computer experiments. Optim Eng 12(4):611–630
Jin R, Chen W, Sudjianto A (2005) An efficient algorithm for constructing optimal design of computer experiments. J Stat Plan Inference 134(1):268–287
Jin Y (2005) A comprehensive survey of fitness approximation in evolutionary computation. J Soft Comput 9(1):3–12
Johnson ME, Moore LM, Ylvisaker D (1990) Minimax and maximin distance designs. J Stat Plan Inference 26(2):131–148
Jones DR, Schonlau M, Welch WJ (1998) Efficient global optimization of expensive black-box functions. J Global Optim 13:455–492
de Jong KA (2006) Evolutionary computation: a unified approach. MIT Press, Cambridge
Kalagnanam JR, Diwekar UM (1997) An effcient sampling technique for off-line quality control. Technometrics 39(3):308–319
Krishnakumar K (1989) Micro-genetic algorithms for stationary and non-stationary function optimization. In: Rodriguez GE (ed) Intelligent control and adaptive systems, SPIE, Bellingham, Wash. Proceedings of SPIE-the International Society for optical engineering, USA
Laurenceau J, Sagaut P (2008) Building efficient response surfaces of aerodynamic functions with Kriging and Cokriging. AIAA J 46(2):498–507
Liang KH, Yao X, Newton C (2000) Evolutionary search of approximated N-dimensional landscapes. Int J Knowl Based Intell Eng Syst 4(3):172–183
Lim D, Ong YS, Jin Y (2007) A study on metamodeling techniques, ensembles and multi-surrogates in evolutionary computation. In: Thierens D (ed) Proceedings of the genetic and evolutionary computation conference-GECCO 2007. ACM Press, New York, pp 1288–1295
Martin JD, Simpson TW (2005) Use of Kriging models to approximate deterministic computer models. AIAA J 43(4):853–863
McKay MD, Beckman RJ, Conover WJ (1979) A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21(2):239–245
Metropolis N, Ulam S (1949) The Monte Carlo method. J Am Stat Assoc 44(247):335–341
Morris MD, Mitchell TJ (1995) Exploratory designs for computational experiments. J Stat Plan Inference 43:381–402
Mugunthan P, Shoemaker CA (2006) Assessing the impacts of parameter uncertainty for computationally expensive groundwater models. Water Resourc Res 42(10)
Myers RH, Montgomery DC (1995) Response surface methodology: process and product optimization using designed experiments. Wiley, New York
Neri F, del Toro Garcia X (2008) Surrogate assisted local search on PMSM drive design. Int J Comput Math Electr Electron Eng 27(3):573–592
Owen AB (1992) Orthogonal arrays for computer experiments, integration and visualization. Stat Sinica 2:439–452
Parr JM, Holden CME, Forrester AIJ, Keane AJ (2010) Review of efficient surrogate infill sampling criteria with constraint handling. In: Rodrigues H, Herskovits J, Mota Soares C, Miranda Guedes J, Folgado J, Araujo A, Moleiro F, Kuzhichalil J, Aguilar Madeira J, Dimitrovova Z (eds) Second International Conference on engineering optimization. Technical University of Lisbon
Queipo NV, Haftka RT, Shyy W, Goel T, Vaidyanathan R, Tucker KP (2005) Surrogate-based analysis and optimization. Prog Aerosp Sci 41:1–28
Quttineh NH, Holmström K (2009) The influence of experimental designs on the performance of surrogate model based costly global optimization solvers. Stud Inf Control 18(1):87–95
Ratle A (1999) Optimal sampling strategies for learning a fitness model. The 1999 IEEE Congress on evolutionary computation-CEC 1999. IEEE, Piscataway, pp 2078–2085
Sacks J, Welch WJ, Mitchell TJ, Wynn HP (1989) Design and analysis of computer experiments. Stat Sci 4(4):409–435
Senecal PK (2000) Numerical optimization using the GEN4 micro-genetic algorithm code. Tech. rep., Engine Research Center, University of Wisconsin-Madison, Wisconsin
Sherwy MC, Wynn HP (1989) Maximum entropy sampling. J Appl Stat 14(2):165–170
Sheskin DJ (2007) Handbook of parametric and nonparametric statistical procedures, 4th edn. Chapman and Hall, Boca Raton
Simpson TW, Lin DKJ, Chen W (2001) Sampling strategies for computer experiments: design and analysis. Int J Reliab Appl 2(3):209–240
Sóbester A, Leary SJ, Keane AJ (2005) On the design of optimization strategies based on global response surface approximation models. J Global Optim 33:31–59
Suganthan PN, Hansen N, Liang JJ, Deb K, Chen YP, Auger A, Tiwari S (2005) Problem definitions and evaluation criteria for the CEC 2005 special session on real-parameter optimization. Technical report, KanGAL 2005005, Nanyang Technological University, Singapore and Kanpur Genetic Algorithms Laboratory, Indian Institute of Technology Kanpur, India
Tenne Y, Armfield SW (2009) A framework for memetic optimization using variable global and local surrogate models. J Soft Comput 13(8):781–793
Tenne Y, Goh CK (eds) (2010) Computational intelligence in expensive optimization problems, evolutionary learning and optimization, vol 2. Springer, Berlin
Toal DJJ, Bressloff NW, Keane AJ (2008) Kriging hyperparameter tuning strategies. AIAA J 46(5):1240–1252
Törn A, Žilinskas A (1989) Global optimization. No. 350 in lecture notes in computer science. Springer, Berlin, Heidelberg, New York
Viana FAC, Venter G, Balabanov V (2009) An algorithm for fast optimal latin hypercube design of experiments. Int J Numer Methods Eng 82(2):135–156
Wang GG, Shan S (2007) Review of metamodeling techniques in support of engineering design optimization. J Mech Design 129(4):370–380
Wu HY, Yang S, Liu F, Tsai HM (2003) Comparison of three geometric representations of airfoils for aerodynamic optimization. In: Proceedings of the 16th AIAA computational fluid dynamics conference. American Institute of Aeronautics and Astronautics, Reston, Virginia, AIAA, pp 2003–4095
You H, Yang M, Wang D, Jia X (2009) Kriging model combined with Latin hypercube sampling for surrogate modeling of analog integrated circuit performance. Proceedings of the Tenth International Symposium on quality electronic design-ISQED 2009. IEEE, Piscataway, pp 554–558
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Tenne, Y. Initial sampling methods in metamodel-assisted optimization. Engineering with Computers 31, 661–680 (2015). https://doi.org/10.1007/s00366-014-0372-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00366-014-0372-z