Safety Tech Challenge Fund – REPHRAIN (original) (raw)

Evaluation Report

The REPHRAIN research centre are pleased to announce to release of “Towards a Framework for Evaluation CSAM Prevention and Detection Tools in the Context of End-to-end encryption Environments: a Case Study”.

Through the Safety Tech Challenge Fund, the UK Government awarded funding to five projects to prototype innovative, automated technologies to help keep children safe in E2EE environments, such as online messaging platforms, while ensuring user privacy is respected.

To enable academic scrutiny, the UK National Research Centre on Privacy, Harm Reduction and Adversarial Influence Online (REPHRAIN) was requested to act as an independent, external evaluator to each of these five projects. To our knowledge, this is the first independent, public evaluation of automated industry tools developed for online child protection.

The final evaluation report presents:

• the finalised version of the evaluation criteria, which aside from performance also include criteria focusing on human rights impact, security, explainability, transparancy, fairness, accountability, etc. The criteria are intended to highlight the trade-offs that are faced when selecting different approaches for online child protection purposes in the context of E2EE environments. Additionally, they can be used as a guidance by safety tech industry to help build public trust in their systems, to positively influence AI technology developments for online child protection, and to ensure all users benefit from these solutions.
• a case study in which these criteria were implemented as part of a formal evaluation of each Proof-of-Concept (PoC) tool funded by the Safety Tech Challenge Fund. We describe the presence or absence of different measures that assure compliance with each criterion and provide guidance where possible when certain criteria were not met.
• a discussion on how future research can support a framework for evaluating CSAM detection and prevention tools.

We welcome feedback and constructive comments on the report. Please send comments to rephrain-centre@bristol.ac.uk.

Evaluation Criteria Document

REPHRAIN is pleased to announce the release of our Scoping the Evaluation of CSAM Prevention and Detection Tools in the Context of End-to-end-encryption Environments document.

This evaluation criteria document describes the scoping stage of REPHRAIN’s independent evaluation of Proof-of-Concept tools for preventing and detecting child sexual abuse media (CSAM) within end-to-end encryption (E2EE) environments that are currently being developed within five different projects funded by the Safety Tech Challenge Fund.

Given the tensions that arise between protecting vulnerable users and protecting user privacy at large, the key steps in REPHRAIN’s evaluation process are (1) to seek input from the community and (2) to publicly publish all results, whilst ensuring that academic rigour and objectivity remain the core of our work, and to inform future directions in this area.

A full report including all feedback provided by the community, and a discussion on how requests for changes were addressed, including any changes made (if suitable) and what was the rationale if a change was not made will be made available shortly.

Thank you to all who submitted feedback during the consultation period earlier this year!