Manual surface detection of cable-stayed bridge cables is low accuracy and high labor-intensive.The speed of conventional image processing and convolutional neural networks is too low to meet the requirements for timely detection.Therefore,a novel encoder-decoder network is constructed to detect cable surface defects.The optimized MobileNetV2 is used as the encoder to reduce the model parameters and increase the training speed.The UNet idea and pyramid pooling (PSP) module are used in the decoder to enhance the feature extraction.Moreover,skip connections connect the encoder and decoder to fuse the deep and shallow feature information effectively.The PASCAL VOC dataset is used to pre-train the network to obtain the weight values of the network, which are then loaded into the network to obtain the final parameters through the training of defect datasets such as holes,gaps and damages.The experiments demonstrate that the novel encoder-decoder network is robust.The mean pixel accuracy,mean intersection over union and the processing time of single image are 89.88%,79.25% and 41.34 ms respectively,which are better than the methods,such as PSPNet,UNet and DFANet. In summary,the novel network meets the requirements of accuracy and speed for surface defect detection of cable-stayed bridge.