An Evaluation of Prompt Engineering Strategies by College Students in Competitive Programming Tasks

Generative AI, powered by Large Language Models (LLMs), has the potential to automate aspects of software engineering. This study implemented a monostrand conversion mixedmethods approach to examine how computer science students utilize generative AI tools during a competitive programming competition across multiple campuses. Participants used tools such as ChatGPT, GitHub Copilot, and Claude and submitted transcripts documenting their interactions for analysis. Drawing from prompt engineering literature, the study mapped six key strategies to 14 areas of best practices for competitive programming. These practices included clarifying instructions, one-shot and few-shot prompting, chain-of-thought prompting, feedback to refine solutions, and leveraging of LLM meta-capabilities. The transcripts were analyzed through a directed content analysis to assess adherence to these practices and then converted to descriptive statistics. Findings revealed significant variability in adherence, with an average compliance rate of 34.2% across practices. While simpler practices achieved adherence rates as high as 98%, eight practices saw minimal or no usage. These results highlight that students often adopt basic prompt engineering techniques but struggle with more complex strategies, suggesting the need for structured prompt engineering instruction in computer science curricula to maximize the potential of generative AI tools.