Banana (Musa Paradisiaca) has rich in sources of vitamins, minerals, and carbohydrates. Image processing is a method or technique that can be used to process images or images by manipulating them into desired image data to obtain specific information. The use of image processing technology is expected to improve accuracy in determining the maturity of the fruit. This study uses the Convolutional Neural Network (CNN) method, one of the effective methods used by computers to do learning from images. With this method, the machine can later identify objects from the given image. Each object in the picture has specific features (such as color, size, texture), so to get the features of the image is very important to do convolution with the appropriate filter. This research resulted in the identification system of cavendish banana ripeness with four classes, including "Ripe", "Through Ripe", "Almost Ripe" and "RIpe". Of the 4 classes will be identified using CNN. The accuracy obtained in this study with Adam optimization parameters get 93.25% accuracy of training data, and 69.34% accuracy of test data and the results obtained by SGD optimization parameters get 94.12% accuracy of training data and 71.95% accuracy of test data.