Video object segmentation is important for object tracking,video surv eillance and semantic classification.In order to overcome the limitation of existing video object segmentation methods u nder dynamic background,a video segmentation algorithm based on motion cue and color information is proposed in this paper.At first,a new motion trajectory classification method is designed.The proposed method can accurately divide the motion trajectory set into background and moving object ones by combining the low rank constraint and cumul ative acknowledgment strategy. Then,the superpixels are acquired by over-segmenting method.And the co lor similarity of adjacent superpixels is computed according to their common boundary.At last,taking the superpixels as n ode,an energy function of Markov random field model is designed,which has combined motion trajectory classificat ion information and color similarity of superpixels.The classification of each s uperpixel can be obtained by finding the minimum of the energy function. The proposed algorithm is tested on several publicly available videos.Experimen tal results demonstrate that the proposed method can accurately segment the moving objects from the dynamic backg round,and it has better segmentation accuracy compared with classical methods.