We study the multi-location inventory management problem where orders are to be placed at each of multiple locations and inventory can be transshipped between locations. Orders placed arrive after a certain lead time, whereas transshipments are completed instantly. In this setting, transshipments can help to prevent stockouts that arise due to unexpectedly high demand, as excess demand at one location can be met by transshipping excess inventory from another location. The objective is to minimize the total cost, which consists of inventory costs, ordering costs and transshipment costs. Demand at each location is time-varying and possibly correlated, and the demand forecast is used by the central planner to make the ordering and transshipment decisions. We model the inventory problem as a Markov decision process and apply deep reinforcement learning (DRL) to it.