If you run a UGC platform (User Generated Content) you either die with an MVP ("minimum viable product") or live long enough to the point that you have to build content moderation. It’s a story as old as the internet and a topic that is seldom discussed. It often happens behind closed doors. It’s the dirty little product secret that no one talks about.
When people ask "How’s your startup doing? You’re still working on that video app, right?". You rarely reply with: "Oh yeah, it’s great. We had to spend the majority of last week trying to come up with clever ways to block users from abusing our platform by uploading NSFW content."
You might sugarcoat that a little bit and respond with a slightly enhanced version: “Oh yeah, it’s great. We have so many users, we can’t keep up with the growth. People are finding all kinds of creative ways to use our product. It’s really an honor and a privilege to build a platform that allows users to really express themselves in creative ways”. (And express themselves they did...)
On day one of building your startup, your biggest fear is that no one is going to even know you exist. All you focus on is trying to get your name out there and get users. When the day finally comes that you get abusive users, it’s almost seen as a rite of passage. And I can tell you first hand, it sucks to spend time on this problem. You are so close to product-market fit you can taste it. You’re just getting near the crest of the hill to catch your breath when a giant boulder comes loose and starts rolling toward you. Depending on the resources you have at the time, it can be an existential threat.