Test Analysis Report. IBLCE Certification Examination Administration. Prepared for the:

Transcription

1 Test Analysis Report IBLCE Certification Examination 2015 Administration Prepared for the: International Board of Lactation Consultant Examiners (IBLCE) Prepared by: Lawrence J. Fabrey, PhD Senior Vice President, Psychometrics Division and Lily Chuang, MS Research Associate, Psychometrics Department

2

3 Test Analysis Report International Board of Lactation Consultant Examiners (IBLCE) IBLCE Certification Examination 2015 Administrations Overview The International Board of Lactation Consultant Examiners (IBLCE) certification examination 2015 test form was administered in 667 locations across 61 countries and territories in 17 languages: Croatian, English (British), Danish, Dutch, French, German, Greek, Hungarian, Indonesian, Italian, Japanese, Korean, Polish, Portuguese, Slovenian, Spanish, and Taiwanese. A total of 4,220 candidates sat for the examination in July and August Of all candidates, 3,733 (88%) candidates tested through computer-based test (CBT), and 487 (12%) candidates tested through paper-pencil test. Data from CBT and paper-pencil administrations were combined for analysis in this Test Analysis Report. ~ 1 ~

4 Test Construction The IBLCE certification examination is based on a 2-dimensional content outline, or test blueprint. This document was derived from a practice analysis by the Board, in conjunction with its Examination Committee members and a Representative Panel of Experts (RPE). On the basis of this study, the Board arranged the examination content according to scientific disciplines and developmental stages, which has since evolved into chronological periods. The examination blueprint appears in its entirety on the IBLCE website ( The examination is scored as a single integrated test. However, it has two multiple-choice item formats: text-based and image-based multiple-choice. The images usually present a photograph depicting an aspect of breastfeeding, or breast anatomy or pathology that the candidates must resolve. These test items have a particularly high degree of clinical relevance, and because most of the item is contained in the graphic, translation is facilitated. Of the 175 items comprising the test, 100 reference images. Following standard operating procedures, the 2015 examination was developed by an Examination Committee that prepared, reviewed, edited, and selected test items, and that participated in a cut score study. The Committee includes broad geographic and practice setting representation of subject matter experts. A preliminary item analysis was conducted after test administrations using all candidate score data (n = 4,220). Psychometrics staff at Applied Measurement Professionals, Inc. (AMP) reviewed the preliminary item analysis with members of the Examination Committee to confirm the appropriateness of potentially problematic items (i.e., those with slightly irregular statistics or adverse candidate comments). Appropriate item scoring changes were made as needed. In addition, an analysis of item performance by the various linguistic versions of the examination was conducted, and items with potential issues were reviewed by the Examination Committee, and the comparability of the items in the various languages was confirmed. ~ 2 ~

5 Test Scoring These statistics are computed for the first-time candidate population of 2,699 candidates, based on the combined text-based and image-based portions. The statistics are presented for the overall test rather than for the two subtests because the examination was designed and intended to be one comprehensive, integrated test. The distribution of raw scores for first-time candidates (n = 2,699) had a slight negative skew (see Figure 1). Reliability (KR20) of scores from first-time candidate responses to the 175-item test was 0.88, the standard error of measurement (SEM) was 5.00, the conditional SEM at the cut score was 5.52, and single administration decision consistency index estimates were and The mean test score was with a standard deviation of Figure 1. Distribution of Raw Scores for First-Time Candidates ~ 3 ~

6 The performance for each content discipline and chronological period is shown in Tables 1 and 2. These data indicate total test performance as well as content discipline and chronological period performance for all candidates. Table 1. Candidate Performance Summary for All Candidates by Discipline (n = 4,220, % passing =84.34) n of DISCIPLINES Items Mean A. Maternal and Infant Anatomy B. Maternal and Infant Normal Physiology and Endocrinology C. Maternal and Infant Normal Nutrition and Biochemistry D. Maternal and Infant Immunology and Infectious Disease E. Maternal and Infant Pathology F. Maternal and Infant Pharmacology and Toxicology G. Psychology, Sociology, and Anthropology H. Growth Parameters and Developmental Milestones I. Interpretation of Research J. Ethical and Legal Issues K. Breastfeeding Equipment and Technology L. Techniques M. Public Health Total ~ 4 ~

7 Table 2. Candidate Performance Summary for All Candidates by Chronological Period (n = 4,220, % passing =84.34) n of CHRONOLOGICAL PERIODS Items Mean 1. Preconception Prenatal Labor/birth (Perinatal) Prematurity days days days months months months Beyond 12 months General principles Total ~ 5 ~

8 Test Results Established statistical procedures for test equating were conducted to adjust for differences in difficulty across test forms and to maintain the minimum competency standard pre-established by the IBLCE. After adjusting scoring for the item, the 2015 form was equated to the 2013 form using 41 equator items as identified at the exam development stage. A post-equating method called Tucker equating was applied, and a passing point of 121 was indicated. The IBLCE approved and adopted a final cut score of 121 raw score units (out of 175) for the 2015 test form. Final results for the examination forms are shown in Table 3, displaying means and pass/fail summary by certification status. Table 3. Mean Score and Pass/Fail Summary by Certification Status Certification Status N Mean % Passing Initial Certification 2, Recertification by Exam at 5 Years Recertification by Exam at 10 Years Recertification by Exam at 15 Years Recertification by Exam at 20 Years Recertification by Exam at 25 Years Recertification by Exam at 30 Years Repeater Lapsed Total 4, ~ 6 ~