The characteristics of the Curvelet transform are studied, and a novel algorithm to fuse multi-sensor images based on the Curvelet transform is proposed in this paper. Firstly, the curvelet transform is used to perform a multi-scale and multi-orientation decomposition of each image. And then,the Curvelet coefficients for the fused image can be obtained by means of different fusion rules. Finally,the fused image is reconstructed by the inverse Curvelet transform. The proposed method is successfully used to merge several sets of multisensor images with different modalities. The experimental results indicate that the proposed approach can avoid the introduction of the artifacts and the high frequency noise and can significantly outperform the traditional wavelet-transform-based image fusion method in terms of both visual quality and objective evaluation criteria.