The Vehicle Routing Problem (VRP) has traditionally served as a significant problem in various fields and got more complicated as the time goes on. This paper considers a complex VRP by combining capacitated time-window time-deliveries on the problem of large-scale distribution of medicine in a megacity. Genetic Algorithms (GA) and Multi- Agent Dueling Deep Q-networks (DQN) are compared as typical representatives of metaheuristic method and reinforcement learning method. Demand coverage, distance, time and energy cost are considered as assessment of performance. The comparison brings forward the robustness of the evolutionary and the learning-based approaches, which provide both methodological and empirical frameworks of optimizing logistics in high-dimensional VRP logistics.