All players, including generation companies as sellers and distribution companies as buyers, try to accurately forecast the electricity price to increase their benefits in the market. The players have been using various techniques to improve their forecasting accuracy because only a little modification can increase the profit significantly. Recently, the machine learning based techniques have been used for forecasting especially price forecasting. The Bi-directional Long-Short Term Memory (B-LSTM) network, as one of the advanced machine learning techniques, can accurately forecast the electricity price. In this paper, the Ontario electricity price data is analyzed firstly. The bad data is denoised and the refined data is used to train the network in the next step. In order to increase the accuracy of the process, the structure and parameters of the designed B-LSTM network are determined optimally. Finally, the designed network are tested in various scenarios both on working days and weekends. The ability of the designed network is investigated for hourly forecasting of the day ahead market especially for spike points prediction.