Stopping rules for steepest ascent in experimental optimization

Steepest ascent is used for experimental optimization in the area of Response Surface Methodology (RSM). The usual recommendation in RSM is to stop experimenting along the direction of the gradient of the response when no further improvement is observed. This statement has resulted in different empirical stopping rules followed in practice, from stopping at the first point where there is an observed drop in response, to the cautious "3-in-a-row" rule in which the search is stopped only after 3 consecutive response drops. A formal sequential stopping rule has been proposed by Myers and Khuri. This paper investigates the performance of the M&K stopping rule compared to the empirical rules used in practice. A new recursive procedure that iteratively estimates a parabola along the search direction is introduced and compared with the other rules. It is shown, based on simulated line searches, that both the proposed recursive procedure and the M&K procedure perform considerably better than the empirical rules. Instances in which each of these two methods should be used are discussed. Among the empirical rules, only the 3-in-a-row rule provides reliable estimates of the location of maximum response, but this is achieved at the expense of a considerable larger number of experiments than other rules. The 2-in-a-row and first drop rules performed much worse, failing to detect the location of the maximum mean response even under favorable conditions (i.e., under low noise and large curvature levels in the response).