The number of crane being used in loading and unloading is rising along with growth of shipping activity. However, the growing number of crane is not simultaneous with crane technology development. Crane operation requires the skilled operator to move containers from ship to shore or vice versa. This is not efficient because a container moving pace and the accuracy depend on operator’s skill, and also for a port to operate 24-hour needs to shift the crane operator. To reduce work accident due to crane operator error and to increase the optimum time of crane use significantly, we offer autonomous container gantry crane to simplify crane operation based on image processing and distance calculation. Using Convolutional Neural Network, and centralized crane operator, it will be able to reduce the number of crane operator and increase the accuracy that will affect crane productivity. It can be operated easily, efficiently, and in the future it is expected to solve the loading and unloading problem.