The shift away from in-house trust and safety teams has created an opportunity for consultancies and startups to introduce something novel: trust and safety as a service.
When the militant group Hamas launched a devastating surprise attack on Israel on Oct. 7, some fighters breached the country’s defenses in motorized paragliders. In the following days, photos and illustrations of Hamas forces coasting by wing became highly charged, controversial symbols: an emblem of Palestinian resistance to some, a glorification of terrorism to others.
The startup ActiveFence, a trust and safety provider for online platforms, is one company sounding the alarm about how predators are abusing generative AI, and helping others in the tech industry navigate the risks posed by these models.
TikTok became the world’s window into the conflict in Israel. Clips from a music festival in southern Israel, where 260 attendees were killed and more taken hostage according to Israel rescue agency Zaka, broke through the algorithm’s regularly scheduled lighthearted programming. For the most part,Noam Schwartz thinks TikTok has played a positive role in the conflict. “People would not believe the magnitude of this event without it being amplified in social media,” he said.
ActiveFence, one of the bigger startups building tech for trust and safety teams, has acquired Spectrum Labs, another key startup in the space building AI tools to track online toxicity.
Russian propaganda is spreading into the world’s video games. Propaganda is appearing in Minecraft and other popular games and discussion groups as the Kremlin tries to win over new audiences.
