Recently, I submitted a public comment in response to the Department of Health and Human Services' proposed rule changes updating existing SACWIS regulations to CCWIS. I used Eckerd's Rapid Safety Feedback (ERSF) as an example. ERSF has had tremendous success decreasing the number of child fatalities in Florida. As a result, the Los Angeles County's Blue Ribbon Panel Commission on Child Protection identified them as a best practice. I wanted to highlight their work; I interviewed Eckerd's Director of Quality Bryan Lindert. He was on the team that came up with the idea for the ERSF mechanism, which is a computer system that leverages predictive analytics to decrease U.S. child fatalities.
Marquis Cabrera: Can you give me a short history of ERSF development?
Bryan Lindert: In Hillsborough, FL - Child welfare homicides were occurring on cases that had open court orders, after [care and protection] cases were already open; these were not accidental or rollovers. The Department of Children and Families Secretary Mike Carroll said, "I am tired of reading post-mortem reports and fatalities reviews saying the same thing about the same kids. Use QA resources in a preventative fashion." So we were able to find the kids most at risk for fatalities -- we mined the State Automated Child Welfare Information System (SACWIS), and then looked at those cases to see if you can assist the caseworker with making the best decisions around the case.
Per working with Department of Children and Families and Mindshare, we learned which cases had high fatality rates: parental substance abuse, young kids and little children were some of the characteristics of families involved in child fatalities. We asked ourselves, "What are the practice issues? Is safety plan sufficient? Is supervisory plan sufficient? Is the quality of the home visits sufficient?"
The Florida SACWIS system (the state child welfare data system) had limitations in its ability to provide real-time data. Therefore, Eckerd contracted with Mindshare, its child welfare technology partner, to provide system overlay software that produces real-time data and agency performance dashboards. This allows cases to be mined in real time for the common risk factors identified with cases that have a high risk of serious child injury or death.
Marquis Cabrera: Can you explain how the department makes decisions using ERSF?
Bryan Lindert: Computer is saying here are cases that need a second look; then a human being not assigned to case, who is also not directly involved with day-to-day happenings of case, comes up with own opinion. We often form an opinion about what is happening in the family. By having a completely different person review, we mitigate against confirmation bias errors.
Upon review of the case: If the safety plan is insufficient, we work with the case manager to rectify identified safety concerns. For example, we say grandmother is going to be responsible for documenting and providing updates on designated safety plan indicators. If it doesn't happen, we can send notifications to highlight when activities are not done.
One part is science; one part is art. "If all you do is identify the list of children who have a higher probability of poor outcome, all you have identified is the train coming at you on the track. What you have to do is change the track and change the trajectory of the case. The coaching and independent review is how we switch the track."
Marquis Cabrera: How many counties is it serving, and do you have any expansion plans?
Bryan Lindert: Currently, we are in three Florida counties. We are expanding to Maine, Connecticut, and most recently Illinois, Alaska, and Oklahoma to develop similar models for them. We will launch in Maine and Oklahoma by January. Alaska is further behind. The biggest bottleneck is getting access to the data. We have four states we have a grant to do work with.
Marquis Cabrera: If you gave me an estimate, how much does implementing ERSF cost?
Bryan Lindert: An estimated $200,000 -- to develop predictions, host software, and to do ongoing Technical Assistance (TA) and fidelity monitoring. It does not include reviewers who have to be from the local jurisdiction. [These reviewers] must be hired by the state or working for the state. We don't need to hire new staff, but rather train current staff. It is certainly cheaper than redefining their practice model and retraining everyone on a practice model. This approach is not making social workers come up with new methodologies. This is a QA intervention. The people whose jobs changed are in evaluation and quality and oversight, not the whole department.
Marquis Cabrera: According to the Tampa Bay Times: "In 2013, the Denver Post looked at six years of child deaths. The Post found 72 in Colorado; the Herald found 477 in Florida. Florida had three times as many children, but six times as many deaths." Has ERSF prevented child deaths? What has been the results to date?
- No child abuse deaths for kids receiving services from Eckerd since implementation of ERSF.
- 53 percent improvement in the sharing of critical case information among the various providers (mental health, substance abuse, domestic violence and others) that are assisting the family in the completion of their case plan. This information is used to document the extent of positive behavior change in the family necessary to ensure child safety.
- 43 percent improvement in the effectiveness of supervisory reviews and follow-up by case managers on critical tasks that need to be completed, such as safety plans, home studies, and collateral contacts with case stakeholders.
- 33 percent improvement in the effectiveness of safety plans. For example, plans are more actionable, verifiable and less reliant on parental promises of changed behaviors.
- 24 percent improvement in the quality of the contacts that case managers have with the families and their children. For example, discussions are focused on real case issues and behavior changes the family is making.
Note: These results must be tested. Eckerd is working with Casey Family Programs on an evaluation of four states who will be using the model to see if they can replicate results in other jurisdictions.
Marquis Cabrera: What problems do you hope ERSF will solve in the future?
Bryan Lindert: We can use predictive analytics to report on multiple child welfare outcomes; for example, returns to care, and delayed permanency. We can replicate the process to search the SACWIS system; then we can come up with predictions. Then the process is a similar one as affecting the outcome and applying QA lenses. The key prerequisite is a well-defined problem. Twenty years ago, this would not have been possible. Let's use the time and resources that have been invested into SACWIS systems and harness the power of what is already in front of us.