Accuracy in Decision Making
The topic of July’s SDM News
was “consistency,” or the degree to which workers make the same decision about how to proceed with a child welfare case given the same information.
But what if everyone agrees on the wrong
decision? While consistency in decision making is important, it is also important to make accurate
decisions. With the Structured Decision Making® (SDM) model
, accuracy is about ensuring that SDM®
assessments measure what they are supposed to measure—not more and not less. This helps workers make decisions that support the most positive outcomes for children and families.
During the process of developing and implementing SDM assessments, the NCCD Children’s Research Center
(CRC) measures accuracy based on the decision at hand and the assessment tool. At intake, for example, we want screening or hotline staff to screen in allegations of abuse and neglect only if those allegations meet a jurisdiction’s legal, statutory, and ethical thresholds for a response. This is critical to ensuring the SDM intake tool is accurate for the work it is intended to do.
Regarding the SDM risk assessment, accuracy is very different. The SDM risk assessment is an actuarial tool that uses data to classify families most likely to have repeat system involvement. This gives child welfare agencies an opportunity to intervene preventively for families most likely to return to the system. Accuracy for this tool is about outcomes, which CRC evaluates at the time the tool is developed and in an ongoing manner: Have we identified the correct population? Do we see the high-risk population coming back into the system more often than the moderate-risk population?
Additionally, CRC staff help jurisdictions during their quality assurance process by conducting case readings of SDM assessments. Case readings confirm that workers are selecting items on the assessment correctly based on SDM definitions and worker documentation in the narrative.
One size fits all is not possible or beneficial in determining accuracy. Different SDM tools have different purposes and require different tests for accuracy. However, regardless of the tool, high standards for accuracy is an SDM model value. Future issues of SDM News will describe the remaining SDM values of equity and utility. Stay tuned!