Video segmentation has played a critical role in most video processing fields.A new algorithm of moving object segmentation in the H.264compression domain for video surveillance and retrieva l is proposed.According to the practical application characteristics,this algorithm mainly utilizes motion vectors and partitioned bl ock size information that are directly extracted for H.264bit streams to analyze the reliability of original motion vector field (MVF),and p rocesses the MVF using spatial filter and mean-shift clustering,thus gets reliable MVF.First,the original motion vectors are obt ained from the H.264stream and normalized after two steps of spatial filter.Then,appropriate weight is assigned for partitioned block size as the weight coefficient of each sample and then the processed MVF is used as the sample space for mean-shift clustering algorit hm to obtain the real MVF.Finally,the moving target will be segmented from the background according to the marked reliable motion vectors.T he experimental results demonstrate that the proposed mean-shift clustering-based moving object segmentation algorithm coul d achieve good performance and efficiency.