A new algorithm to segment the moving object in the H.264 compressed domain is proposed.This algorithm mainly utilizes motion vectors that are directly extracted from the H.264 bitstreams.In order to improve the robustness of the motion vector information,firstly the intra modes and intra prediction residual energy in I frames are used for region classification.The inter prediction residual energy of P frames is used to update the classification results.Some motion vectors are set to be zero based on the re...