To address the challenges of inaccurate lesion localization and unclear segmentation boundaries in colorectal polyp images, this paper proposes a colorectal polyp segmentation algorithm that integrates Transformer and dual graph convolution techniques. First, the Transformer encoder is utilized to parse multi-scale feature information within the images, establishing long-distance dependencies between pixels. Second, a dual graph convolution semantic detail injection module is designed to integrate the spatial and structural information of deep features, enhancing boundary representation. Third, a local pyramid attention module is employed to calculate global and local attention weights on shallow features, precisely localizing the lesion areas while suppressing irrelevant information. Fourth, a dynamic feature fusion module is introduced to adaptively aggregate multi-scale features, improving the handling of pathological images with large scale variations and irregular shapes. Experimental results on four public datasets, Kvasir, CVC-ClinicDB, CVC-ColonDB, and ETIS, show that the Dice coefficients are 0.924, 0.943, 0.817, and 0.813, respectively, and the mean Intersection over Union (IoU) scores are 0.871, 0.896, 0.733, and 0.732, respectively, verifying the effectiveness of the proposed algorithm.