Test preparation and test quality assessment: What I wish someone had told me in the beginning

How does an instructor prepare a test and feel confident that it is fair, balanced and the correct length? That issue worried me as I entered the teaching profession three decades ago. After a decade of writing multiple choice questions for the Professional Engineering exam and two decades of national involvement with the ABET accreditation process, it is time to pass some of the "tricks of the trade" along to the next generation of new faculty members. As a starting point for this discussion, the assumption that course outcomes have been linked to not only Program outcomes but also ABET outcomes will be reviewed. While discussing the selection of an appropriate evaluation method, I will also discuss some of the topics that do NOT lend themselves to timed tests. Suggestions on alternative methods of evaluation for those topics will be offered. This paper will focus on how to prepare good True/False and Multiple Choice questions to test knowledge and skill. It will also present a simplified version of the method psychometricians use to evaluate the quality of these types of exam questions for both difficulty and discrimination. Since many engineering and technology questions involve calculations, a test evaluation method that works for me will be discussed. This method is a good first step to demonstrate course improvement to ABET. Since much of how we teach is driven by ABET criteria and the need for continuous quality improvement, this experience should prove valuable to new faculty who may never have prepared for an ABET visit.