Trust and safety tooling is ripe for intentional, collective investment and focus. More effective, openly available tooling could lower barriers for small and medium-sized entities seeking to build stronger trust and safety practices.
Implementing trust and safety practices is as much a logistical challenge as a policy challenge. It requires a technical implementation layer that is highly complex and often built over time through homegrown tooling suites and organizational structures. Through workshops and interviews with approximately fifty T&S expert professionals, the task force outlined the tooling needs of a variety of types of organizations (companies, nonprofits, vendors) of different sizes and focuses.
The conclusion? T&S tooling is an area that remains ripe for intentional, collective investment and focus. More effective, openly available tooling—as well as more accessible best practices for development of T&S tools—could lower barriers to the development of a diversity of services, and prevent each organization from needing to reinvent the wheel. Moreover, it can help address what is essentially a market failure—individual services may not internalize all the social costs of harmful content and behavior, and, thus, may not invest sufficiently in socially optimal T&S.
A core challenge in providing open-source tools is the need for ongoing maintenance and customization. Nothing in T&S is “set it and forget it”—tools need constant maintenance to stay relevant to the evolving threat landscape online. An important opportunity exists to fill these systems gaps and facilitate a wider range of T&S options for small and medium companies. An institutional hub for best practices could provide technical tooling support for T&S and serve as a one-stop shop for practitioners who are navigating the tooling landscape. It could also contribute to the development of new tools, and—critically—act as a steward of open-source tools that other actors are willing to contribute but may not be willing to maintain.