We investigate the impact of Chain of Thought (CoT) prompting on smaller language models. While CoT has shown significant improvements in the performance of large language models (LLMs), our research suggests that this technique may be detrimental to the performance of smaller models, showing 15~30% decrease in accuracy for SLMs when using CoT prompting compared to standard prompting. We conducted experiments using a range of model sizes and found that CoT prompting consistently degraded the performance of models below a certain parameter threshold. This work highlights the importance of considering model size when applying prompting techniques and suggests that alternative strategies may be necessary for enhancing the capabilities of smaller language models.

Keywords

CoT, Chain of Thought, LLMs, SLMs and GPT-2,