UCLA | generative pre training for black box optimization

【 title 】Generative Pretraining for Black-Box Optimization

【 The author team 】Siddarth Krishnamoorthy, Satvik Mehul Mashkaria, Aditya Grover

【 Time of publication 】2021/06/22

【 machine structure 】UCLA

【 Thesis link 】https://arxiv.org/pdf/2206.10786v1.pdf

Many problems in the field of science and engineering involve the optimization of an expensive black box function in high-dimensional space . For such black box optimization （BBO） problem , It is often assumed that the budget for online function evaluation is very small , However, it is often possible to obtain a fixed offline data set for pre training . Previous methods attempt to approximate functions or their converses using off-line data , But it is not accurate enough far away from the data distribution . This paper presents a black box optimized converter （BOOMER）, A generation framework for pre training black box optimizer using offline datasets . stay BOOMER in , Train an autoregressive model to simulate the trajectory of the implicit black box function optimizer . Because these tracks are not available by default , This paper develops a simple random heuristic method , The trajectory is synthesized by classifying the random points of the offline data . This paper shows theoretically that , This heuristic method induces trajectories that mimic those from different low fidelity （ Explore ） To high fidelity （ Development ） Sample transition . Besides , This paper introduces a mechanism to control the speed of trajectory from exploration to development , And use it to generalize beyond the offline data when testing . From experience , This article uses a random mask Transformer To achieve BOOMER, And in Design-Bench Evaluate it on , The average ranking is the best , Beyond the most advanced baseline .

The figure on the left shows an offline problem on a one-dimensional problem BBO example . The red dotted line is the boundary of the value range . therefore , The correct optimal value is x∗, The gradient rise method of the fitting function will output the distribution points outside the domain x¯.

The figure on the right shows two-dimensional Branin Function trace example , The two dashed lines represent the trajectories in the offline data set of this paper , The solid line refers to the trajectory of the model , The blue dot indicates exploration , The red dot indicates the use of .

The picture above shows BOOMER The schematic diagram of .BOOMER from 3 A series of stages ： Track construction 、 Autoregressive modeling , Launch assessment . In the first phase , Use SPORT-SAMPLE Take offline datasets D Convert to a track data set D_traj. In the second phase , by D_traj Learn an autoregressive model . In the third phase , Put the model on an offline prefix sequence , And further expand to obtain candidates , Introduction Q Candidate points to evaluate the model . Among them Rˆ by Evaluation Regret Budget

The picture above shows the Branin benchmark The result on , This benchmark is a benchmark that has 3 A global optimum （−0.398） The binary function of , For offline optimization , In this paper, we sample uniformly in the domain N=5000 A little bit , And remove from this set the pre - 10%, To remove points close to the optimization point , Make the task more challenging . then , This article is based on SORT-SAMPLE Strategy building 400 The length of each bar is 64 Track of . In the process of evaluation , This article initializes the four prefixes as 32 Track of , And expand it for evaluation . The length is 32 Track of , Unfold the 32 Step , And output the best results , This consumes 128 Query budget .

BOOMER It successfully generalizes the best point in the offline data set of this paper , The best thing about an offline dataset is −6.119,BOOMER The global optimum is obtained −1.79 ± 0.843. Gradient ascending baseline uses offline data sets to train a forward model （ One 2 Layer of NN） To map x To y, Then on x To infer the optimal value by ascending the gradient , The effect is −3.953 ± 4.258.

 Branin The task is in multiple Rˆ Value . chart （b） It shows that the prefix length is 32 Track of , In the figure （a） in , Change the prefix length to 16, And in the picture （c） in , Update Evaluation in suffix subsequence RB. You can see , With Gao Rˆ comparison , low Rˆ Launched higher quality points . To verify, this article will regret The semantics of budget as a knob to control the degree of exploration and development , In the figure 5c in , This article also draws the update suffix in RB Value trajectory . If RB Become non positive , Just stop pushing . Obviously , For the smaller Rˆ, The agent model is rapidly accelerated to high-quality areas and continuously developed , And for high Rˆ, It gradually shifts to high-quality points . This shows that Rˆ How to control the speed of transition from exploration to development .

 

Next , In this paper Design-Bench Of 5 On a complex real-world mission BOOMER. These tasks are considered challenging , Because they have high dimensions 、 Low quality points in offline data sets 、 Approximation in some cases oracle, And high sensitivity with narrow optimal region landscape.

in general ,BOOMER To obtain the 0.772 The average sum of 2.4 The average ranking of , This is the best of all baselines .BOOMER stay TF-Bind-10（ discrete ） and D'Kitty（ continuity ） The best results were achieved on these two tasks . Besides , This paper ranks the top two in five of the seven tasks . stay TF-Bind-8、TF-Bind-10、Ant and D'Kitty On ,BOOMER It's better than that MINs or CbAS And so on COMs Obvious improvement of equal forward mapping method . This article also notes that , although BOOMER stay Ant Second in the list , But its standard deviation （0.012） Better than best CMA-ES Much lower , The standard deviation of the latter is 0.928. In all tasks ,BOOMER The mean standard deviation of is also the lowest , This shows that BOOMER Compared with other methods , Low sensitivity to poor initialization .

 

Innovation points

This paper proposes BOOMER, A new generation framework , Used to pre train the black box optimizer with offline data .BOOMER It includes a three-stage process . In the first phase , Use a novel PORTSAMPLE Policies generate tracks from offline data , These trajectories use sorting heuristics to transition from exploration to development . This article further provides a mechanism , adopt Regret Budget To control the degree of exploration and development . In the second and third stages , In this paper, the autoregressive converter is used to train the model , And use it to generate candidate points that maximize the black box function .