Lathe machines are a technological advancement involved in cutting and shaping materials and are widely used across several industries. However, many injuries and fatalities arise due to the improper handling of the machine and its malfunction. A range of safety approaches have been developed to reduce incidents from training to PPE; nevertheless, incidents still occur due to the unpredictability of the machine’s function and human error. In our approach to taking on this issue, we proposed a system that identifies potential accidents through computer vision and records potential risks in real time with the help of a deep learning model. Based on previous studies, our proposed method shows great potential in the advancement of health and safety measures to reduce on-site damage to equipment and personnel.