In order to train a high-quality background model adapted to the video compression domain,this paper proposes a video background modeling algorithm with low complexity based on the minimum second derivative according t o the distribution of pixels in the time domain.Firstly,estimate the wave characteristics of the function according t o its second derivative.After that,get the stability of every pixel by using the second-order difference to fit the second derivate of pixels in the time domain.Finally, extract the steadiest value of every pixel during the training period in the basis of threshold value,then take it as the corresponding background model value.The experimental results indicate that comp ared with AVS2,BD-rate is saved by 9.83%,and BD-PSNR i s improved by 0.37dB.Compa red with the background modeling algorithm of AVS2-S,this algorithm not only effectively improves the problem of foreground pollution,but also reduces the algorithm complexity.