Abstract: Gaussian process regression (GPR) takes advantages of programming easily; self-adaptive acquisition of hyperparameters and prediction with probability interpretation. At present the hyperparameters are got by maximizing likelihood function of training samples based on conjugate gradient algorithm. However, the algorithm embodies the disadvantages of too strong dependence of optimization effect on initial value, difficultly determination of iteration steps, easily falling into local optimum. On the basis of mentioned above, particle swarm optimization (PSO) was used to optimize the hyperparameters of the GPR then formed the PSO-GPR coupling, and the corresponding source code was programmed. So the PSO-GPR with the neural network covariance function (NN), squared exponential covariance function (SE) and rational quadratic covariance function (RQ) is respectively applied to analyze slope displacement time series in order to test generalization ability of the algorithm. The analysis results of slope displacement time series show that the proposed PSO-GPR based on different single kernel function are all suitable for all kinds of slope deformation analysis and difference of prediction accuracy is little based on different single functions. Furthermore, the problem that there is differential the regression and generalization ability of kernel machine with different functions is solved; and the kernel function compatibility with different data is boosted

Key words: landslide, particle swarm optimization, Gaussian process regression, time series, displacement prediction