About This Project

The core problem isn't that pretrial risk tools might not work — it's that current evaluation designs can't tell us whether they do, because they compare bundled reform packages to a pre-reform baseline instead of isolating the tool's own marginal contribution. The fix doesn't require a new algorithm. It requires a minimum data-logging standard that any jurisdiction adopting a risk assessment tool must implement before rollout, so the evaluation is buildable after the fact instead of impossible after the fact. Mechanism: require four timestamped decision points to be logged separately in the case management system for every pretrial case, starting before the tool goes live: (1) arrest/booking decision — cite vs. custody, (2) prosecutorial charging and detention-request decision, (3) risk assessment score generation and recommendation, (4) judicial release/detention decision. Most jurisdictions already generate this data somewhere across separate police, DA, and court systems; the change is requiring these four timestamps to be joined into a single case-level record accessible to researchers, with a decision attributed to whichever actor made it and whether the risk score was available to them at that point. With that data, evaluators can run the comparison the field has been unable to run: cases where the prosecutorial/policing decision alone would have produced release vs. cases where the risk score specifically changed the judicial outcome. That decomposition — pioneered in miniature by the California Policy Lab's San Francisco isolation study — becomes a standard output instead of a bespoke research project requiring a funded policy lab. Vendors selling risk assessment tools (Arnold Ventures/APPR's PSA being the dominant one) should be required to support this logging format as a condition of new contracts, and existing SJC/MacArthur Foundation-funded sites should retrofit it as a condition of continued grant funding, since they already have the institutional relationships to enforce data-sharing agreements with courts. This doesn't resolve every methodological problem — it doesn't fix racial bias baked into arrest history, for instance — but it directly answers the specific question this challenge raises: is the tool doing anything, or is it along for the ride. A jurisdiction that finds its risk score changes zero judicial outcomes relative to what prosecutors already requested has just learned it's paying a vendor for theater. A jurisdiction that finds the opposite has real evidence to defend the tool against political attacks. Either result is more useful than what the field has now.

Timeline