Bug Bounties For Algorithmic Harms?

Bug Bounties For Algorithmic Harms? Lessons from Cybersecurity Vulnerability Disclosure for Algorithmic Harms Discovery, Disclosure, and Redress is a report published by the Algorithmic Justice League on 27 January 2022 and authored by Josh Kenway, Camille François, Sasha Costanza-Chock, Inioluwa Deborah Raji, and Joy Buolamwini. The report is the foundational publication of AJL's Community Reporting of Algorithmic System Harms (CRASH) project — originally launched in July 2020 as the Algorithmic Vulnerability Bounty Project — and asks whether the bug-bounty model that the cybersecurity field developed for paying outside researchers to find and disclose security vulnerabilities can be adapted to the discovery, disclosure, and redress of algorithmic harms.

The report opens from the observation that bug-bounty programmes (BBPs) — once a contested practice — are now a routine part of how Google, the U.S. Department of Defense, Starbucks, and hundreds of other organisations buy security flaws from outside researchers, and that a small number of operators (Rockstar Games, Twitter, and others) had begun by 2022 to extend the BBP frame to socio-technical issues including algorithmic bias and content-moderation harms. The authors draw on interviews with BBP experts and practitioners, a literature review, and a case study of Twitter's 2021 algorithmic-bias bounty pilot to set out a design framework for what BBPs would have to look like in order to surface algorithmic harms rather than only security flaws. The five summary takeaways — reproduced in the report's launch coverage — call on operators to prepare BBPs to include socio-technical concerns rather than treating algorithmic harms as bolt-ons to a security programme; to look across the lifecycle of an AI system, accompanying BBPs with other accountability mechanisms; to nurture a community of practice that does not exclude non-computer-scientists; to intentionally develop a diverse, inclusive community of researchers and community advocates with fair compensation; and to foster and protect participatory adversarial research with a guarantee of public disclosure of findings. The report's recommendations sit alongside pre-publication public framings by AJL researchers that argued for adapting the bug-bounty model specifically to biometric algorithm bias.

Within the corpus, Bug Bounties For Algorithmic Harms? is the third Algorithmic Justice League-anchored Publication after Unmasking AI and Comply To Fly?, and the second AJL-as-publisher artefact in the corpus exercising publisher: org-algorithmic-justice-league after Comply To Fly?. The report sits earliest in time of the three AJL Publications and provides the design framework that the Freedom Flyers participatory-audit campaign and the Comply To Fly? report later put into practice at scale: where Comply To Fly? applies AJL's evocative-audit method to a single federal facial-recognition deployment, Bug Bounties For Algorithmic Harms? sets out the broader institutional question of how outside researchers and affected communities can be brought into the harms-discovery process at all. The MacArthur Foundation's grantee-publications page for the report records AJL's grantee status under the Foundation's Technology in the Public Interest programme, and the report's published acknowledgements (carried in the AJL landing page and the report PDF) credit the Alfred P. Sloan Foundation, the Rockefeller Foundation, and the Mozilla Foundation as supporting funders. The report has since been a frequent reference point in subsequent AJL programme work and in adjacent academic and policy writing on algorithmic auditing, participatory audit, and community reporting of AI harms.