Field Tests

Purpose

End-of-Course field testing is an important part of psychometric quality assurance. The purpose of field testing is to examine both individual test items and entire test banks. Field testing is a quality-assurance process that helps CETE staff to select better items to create operational test forms. Field-test data can also be used to adjust cutoff scores when finalizing operational test forms to 40 items.

Secondary instructors from across Ohio participated in item-writing workshops at CETE at The Ohio State University. Preliminary End-of-Course test banks for selected pathways were created at these workshops. Each fall, previously written items are field tested with students as part of the process for creating final forms of 40 items.

You may review the WebXam about the tests web page to see which tests are currently in field testing.

Transition to Courses

Field-tested forms have replaced older tests that were end-of-program or modular. The new End-of-Course test are intended to be administered upon a student’s completion of a course. For more information on the Ohio CTE move from a program-based to a course-based system, see the following web page: Ohio career tech goes from program-based to course-based system

Test Administration

Posttests are administered at no cost to Ohio local-education agencies via WebXam (www.webxam.org), the online assessment delivery system maintained by CETE. Tests are not available in paper-and-pencil format. District personnel, including testing coordinators and others, can help you set up assessments on WebXam.

NOTE: Field tests will be available until the cutoff date listed on the WebXam calendar (mid-May) so rescoring can be accomplished.

Availability

Field-test forms are available to all students enrolled in the corresponding career-technical education programs. The larger the number of examinees participating in the field test, the more reliable the results; therefore, all teachers and school districts with relevant programs are encouraged to participate. Students should complete End-of-Course tests only if the student is enrolled in the corresponding course and has completed the related coursework related to that course.

Student Scores

Scores for assessments in field testing will be reportable through Information Technology Centers (ITC) to local districts for transmission to EMIS, Ohio’s statewide data-collection system. However, assessment results will be analyzed and scores adjusted before reporting by CETE at the end of the school year.

During the field-testing period, students will only receive a raw score for each EOC test taken. Students will not receive a pass/fail indication for field-tested assessments at the time of testing. Field testing (and only field testing) will end on the cutoff date. At this time, CETE psychometric staff and ODE pathway personnel will analyze the field-test results. Poorly performing items will be dropped, and final test forms of 40 items will be selected. Every student’s test result will then be rescored based on the final set of items chosen. The percentage score may increase or decrease from the raw score shown to the student during field testing. CETE staff will provide final scores with pass/fail indications for all field-tested assessments based on the published timeline on the WebXam Calendar. Districts will be able to run reports on the WebXam website, and field-test results will be included in the end-of-year score output provided to ITCs for reporting purposes.

Assessment Structure

In general, each overall test and bank contains items of two levels of challenge: C1 and C2. Approximately 30 percent of items will be at the C2 level (higher in Bloom’s taxonomy – mainly application, analysis, or evaluation). The remaining items are at the C1 level (knowledge and comprehension). In addition, approximately 40 percent of items for each assessment are scenario based. A scenario is a short situation, story, or passage of text that provides information needed to answer several associated items. Graphics (tables, charts, figures, and maps) may be included as well to provide information needed to answer the test item.

Each test corresponds to a course as defined by the Ohio Department of Education. Specific information regarding each individual pathway test—forms to administer, tests included in each pathway assessment, and number of items in each form—follows in the next section. To find links to the competencies in the Ohio Career Field Technical Content Standards covered by each test, go to the WebXam About the Tests page at https://www.webxam.org/admin/abouttheassessments.

Students are expected to test for each course in which they are enrolled. They are not required to take all End-of-Course tests available for the entire program. Because programs vary in length and the minimum number of courses approved for a program is four, concentrators cannot take less than 3 tests without getting a zero score counted for the missing tests when calculating the overall score.

2017-18 Field Testing Update:  Pretest Withdrawal and New Courses

WebXam users asked about any new field tests available for the 2017-18 year.  There are six withdrawn pretests (based on course revisions) as well as new courses in field testing.  Most new courses are in the Education and Training career field, with others sprinkled across Construction (2), Engineering (2), and Law-Public Safety (3).

Withdrawal of pretests, six in number for 2017-18, occurs under specific circumstances to be explained. If a course is a new one, however, there would NOT be a pretest therefore no issues arise; those new courses are also listed below.

First, if an existing course is revised then the item/question bank must be revised as well, and the revised bank is again field tested (now a one-year cycle).  In 2016-17 CETE revised six courses from Health Sciences and Engineering, listed below with subject codes.  Those pretests are not available for the upcoming year.

Pretests WITHDRAWN for 2017-18 (REVISED courses in field testing)

Career Field

Course Title

Subject Code

Engineering

Engineering Principles

175002

 

Digital Electronics

175007

Health Sciences

Exercise and Athletic Training

072000

 

Fitness Evaluation and Assessment

072020

 

Clinical Laboratory Techniques

072100

 

Medical Terminology

072150

Second, CETE annually receives new courses developed to patch pathways.  Seven new courses in field testing for 2017-18, outside of the Education and Training Career Field, are listed below.  Because they are new, they will not have a pretest until QA is complete.  Note that field testing is now limited to a single year.

NEW Courses in Field Testing for 2017-18 [EXCLUDING Education and Training)

Career Field

Course Title

Subject Code

Construction

Fundamentals of Architecture and Construction

178040

 

Principles of Woods Construction

178030

Engineering

Engineering Logic

175017

 

Principles of Manufacturing

176010

Law & Public Safety

Firefighter I

170343

 

Firefighter II

170344

 

EMT

170345