Variable coding modes are employed in H. 264/AVC ,which makes the mode selection a time-consuming compo nent. In this paper,we first develop the concept of mode reference aggregate. Then based on the aggregate,a fast mode decision algorithm is proposed which classifies all possible modes into four different mode reference aggregates. In the proposed algorithm,one aggregate is first chosen from the four reference aggregates based on the special and temporal correlation. We only check the modes in the aggregate determined above and decide the final coding mode. This method can significantly re duce the number of candidate mode to simplify the procedure of mode decision. Experimental results show that the algorithm can increase the encoding speed significantly with negligible PSNR drop and bit-rate increase.