Protects the LGBTQ+ Community with Proactive Moderation Efforts

When a young person reaches out on TrevorSpace, they may be in crisis. In the case of suicide and self-harm, it's crucial to ensure the team takes action quickly to help a user access life-saving care in a timely manner. That's the environment The Trevor Project operates in every day.

Given the volume of content posted to TrevorSpace on a daily basis, it's impossible to monitor every interaction on the platform. Prior to Alice, the team relied on manual moderation, historical systems, and user flags to catch content that violated their policies. As a non-profit with limited resources, this approach couldn't scale with the growth of the community.

When launching TrevorSpace, the team understood the importance of implementing safety by design. Yet after launch, the popularity of the site meant they needed a vendor to help reduce reliance on user flags and automate content moderation for specific abuse areas, taking action on harmful content in a more operationally efficient way.

Detection

The Trevor Project needed a content moderation vendor that could cover the violations critical to their community: harassment and bullying, hate speech, child solicitation, suicide and self-harm. Coverage alone wasn't enough. The quality of the models was crucial. In an effort to reduce undetected content, they turned to ActiveScore, Alice's contextual AI automated detection capabilities.

A key challenge was ensuring detection models understood the difference between harmful content and the language LGBTQ+ youth naturally use in their community. Alice customized its hate speech models to identify relevant keyword lists that would remove words commonly used among the LGBTQ+ youth community from being flagged, aligning detection to The Trevor Project's specific policy rather than applying a generic model.

Enforcement

To strike a balance between providing a safe space and allowing the necessary freedom for community expansion, The Trevor Project needed a partner to implement their warnings and penalties guidelines quickly and effectively on TrevorSpace.

When a user violates a guideline and the moderation team becomes aware of it, users are issued warning points. TrevorSpace leverages no-code workflows to automatically implement these policies. Anyone with 0-5 points automatically receives a warning. Anyone with 6-7 points receives a two-week suspension. Those with over 8 points are permanently banned. They also use moderation queue management to manually moderate community messages with greater efficiency.

By using Alice, The Trevor Project is able to ensure greater protection against the most egregious harms facing their community on the TrevorSpace platform. Detection models are tuned to their specific community rather than working against it. Enforcement is automated, consistent, and aligned to their policies without requiring manual intervention at every step.

By incorporating a proactive approach to moderation, they have moderated thousands of forums on the platform and ensured that their users have a safe space to discuss the issues that matter most to them.