With the application of various wireless communication technologies, the electromagnetic environment has become more complex, and the recognition of signal modulation has become increasingly difficult. In this paper, a hybrid model based on deep learning, which aims to quickly classify received modulated signals and help to plan spectrum resources, is proposed. The model is designed by considering the characteristics of convolutional neural networks (CNNs), global average pooling (GAP), gate recurrent units (GRUs), and other structures. Firstly, signal spatial features are extracted by convolution using a CNN, the dimension of the high-dimensional feature map is reduced by GAP, then the signal temporal correlation is extracted using GRUs. Finally, modulation modes are classified in the softmax layer to classify and recognize the modulation modes of the received signal. Experimental results show that the average recognition rate of the model was 60.64% and the maximum recognition rate was 90%. The proposed method not only improves the recognition performance but also enhances the interpretability of the network.