Enterprise Resource Planning (ERP) packages have transformed the way organizations go about the process of providing Information Systems. Existing literature has suggested that there are different critical success factors that contribute to success of ERP implementation. However, the factors more peculiar to the public and private organizations are not well identified. The main goal of this research is to identify those critical success factors of ERP implementation that mainly work for private and those for public organizations in Ethiopian context. Understanding this can help in successful implementation of ERP systems in those two organizational contexts. A descriptive survey design is employed to identify the more critical factors that affect ERP implementation in private and public organizations. To identify the critical factors that affect ERP implementation in both organizations, Pareto approach was applied. Purposive sampling technique was used to collect the data in both cases. The findings show that availability and facilitation of IT infrastructure, teamwork composition, top management support and commitment are crucial issues that were identified in both cases. However, business process reengineering and software configuration, training and support for users, end user involvement, and existence of communication plan, consultant selection and relationship are identified as crucial factors that were found in private organization. On the other hand, budgetary constraints, financial plan, implementation strategy and time frame, customization/localization of software according to environment are identified as the top critical factors that found in public organization. Based on key findings, recommendations are forwarded for practice in order to enhance their ERP implementation.