It is actualy simulated that return to zero format(RZ) and non return to zero format(NRZ) transmission at 40 Gbit/s in the single channel.Taken into account second order dispersion,third order dispersion,polarization mode dispersion and noise of EDFA,using Q value discriminant method,maximum transmission distance is calculated.The eye pattern is drawn.The results show that maximum transmission distance of RZ is larger than the NRZ.The RZ format system transmission capability ismore excellent than the NRZ.