Abstract:In the case of co-existence of cubic and quintic nonlinearity,the condition and power gain spectra of modulation instability induced by the cross-phase modulation for two optical waves of different frequencies with the same polarizations in loss,single-mode,decreasing dispersion fibers were derived.The synthetic effects of the quintic nonlinear coefficients,the dispersion longitudinal varying parameters,and the size relation of the two perturbation frequencies on the condition and power gain spectra of modulation instability were analyzed in detail in the negative dispersion regime of decreasing dispersion fibers.The results show that,the positive and the negative quintic nonlinearity will intensify and suppress modulation instability induced by the cross-phase modulation,respectively.If the size relation of the two perturbation frequencies is different,the dispersion longitudinal varying parameters influcence the gain spectra of modulation instability in different ways.Depending on different parameters,the gain spectra have two different shapes.Some may have cut-off perturbation frequencies while others may have not.