Nick Item-Writing Academy
About 40 minutes · Academy module: Difficulty Calibration.
Difficulty is the proportion of examinees who answer correctly — the p-value. A p-value of 0.85 is an easy item; 0.35 is a hard one. Difficulty is not discrimination: a hard item can discriminate well or badly, and an easy item is not automatically a bad item. What matters is whether the item discriminates at its difficulty level — do the stronger examinees get it right more often than the weaker ones? The item-analysis module teaches the full reading.
Our read: faculty conflate "hard" with "good" constantly. A hard item that strong and weak examinees miss at equal rates measures nothing — it is noise with a low p-value. Difficulty without discrimination is just confusion.
You can move difficulty without changing the tested point. To make an item easier: strengthen the cues (make the key's findings more classic), weaken the distractors (replace the near-miss with a merely plausible option), or simplify the decision (recognition instead of prioritization). To make it harder: add a near-miss distractor from the key's family, require two-step reasoning (interpret cues then prioritize), or bury the discriminating cue among plausible-but-irrelevant findings.
The lever to avoid is trickiness — unemphasized negatives, ambiguous wording, trivia details. Tricky items look hard (low p-value) but discriminate poorly, because strong examinees overthink them and weak examinees guess. Difficulty should come from the clinical reasoning demand, never from the reading demand.
An exam needs a difficulty spread, not a difficulty point. Decide the band before you write: for a mastery check, most items in the 0.60–0.85 range with a few stretch items below 0.60; for a discriminating final, a wider spread. Then write to the band deliberately — draft each item with its target difficulty in mind and note which lever you pulled.
After administration, compare your estimates against the actual p-values. Every author has a calibration bias — most write harder than they think. Track yours: if your "medium" items consistently land at 0.45, adjust the mental model, not the items.
(intended key: B)
Stem: A nurse is reviewing the medication administration record. Which of the following is NOT an inappropriate action when the nurse is NOT caring for a client who is NOT prescribed warfarin?
A. Administering the medication without checking the client's identification B. Checking two client identifiers before administration C. Documenting the administration after giving the medication D. Asking the client to state their name and date of birth
difficulty manufactured by triple negatives and reading traps.
This item is "hard" only because it is nearly unreadable — three negatives in one lead-in. Strong examinees will misread it; weak examinees will guess. The p-value will be low and the discrimination near zero: the worst combination. The rewrite tests the same content domain (safe medication administration) with a clean positive lead-in at an honest medium difficulty.
— cold-solve verified: 2/2 (solvers chose B, B)
Stem: A nurse is preparing to administer oral medications to four clients. Which situation requires immediate intervention before the nurse proceeds?
A. A client who asks what each pill is for before taking it B. A nursing assistant offering to administer the medications to save time C. A client who requests water to swallow the pills D. A client whose identification band is present and legible
Key: B
Medication administration cannot be delegated to unlicensed assistive personnel — that is the scope-of-practice violation requiring immediate intervention. Asking about medications, requesting water, and having an intact ID band are all normal and safe. Both independent cold solvers selected B and named no other defensible option.
Recording it adds the module to your Academy completion record on this device — module, track, level, and date, ready to download from your account page for faculty-development documentation.