This paper proposes a real-time depth extraction algorithm based on Kinect and a multi-view rendering algorithm based on single color plus depth.At the acquisition side,we acquire color information and depth information by Kinect,and present a fast inpainting algorithm to inpaint holes in depth map output by Kinect.At the display side,to solve the big hole filling problem of depth image based rendering(DIBR) based on single color plus depth format,this paper presents a DIBR algorithm based on background estimation and foreground segmentation.Experiments show that our method can get good depth image in real-time,and effectively inpaint the big holes due to DIBR,so multiple virtual view images with good quality can be got.At last we implement a multi-view stereo video system based on the depth extraction and rendering algorithm.A good stereo perception can be achieved by interlacedly displaying the virtual views on auto-stereoscopic display,which further verifies the effectiveness of our method.This system can be used to record and display multi-view stereo videos for live scenes.