The asymmetric single-sideband modula ted vector millimeter waves can effectively improve the spectrum efficiency,and the non-uniform modulation can effectively overcome the nonlinear effect of the channel by optimizing the Euclidean distance between these constellations,so as to improve the signal-to-noise ratio of the transmission system.A scheme of a symmetric single sideband modulated vector millimeter wave signal generation by non-uniform modulation is proposed.Under the development environments of VPI an d MATLAB,the theory analysis and simulation verification of geometric shaping on a W-band vector millimeter wave signal which generated by a single I/Q modulat or are carried out by using generalized paired optimization algorithm.In Gaussia n channel,firstly,the mutual information gain of the non-uniform geometric shap ing constellation and the traditional uniform constellation are compared by simu lated analysis in detailly,and then,the shaping gain brought by the 8PSK non-un iform modulation format in back-to-back (BTB) transmission and optical fiber l ong-distance transmission conditions are compared.The results show that th bit error rate (BER) performance of the geometrically shaped signal is always better than that of the unshaped signal in BTB transmission and optical fiber long-di stance transmission systems,and the optical power is increased by 0.5dBm at lea st.