On a technical level, user count matters less than the user count and comment count of the instances you subscribe to. Too many subscriptions can overwhelm smaller instances and saturate a network from the perspective of Packets Per Second and your ISPs routing capacity - not to mention your router. Additionally, most ISPs block traffic traffic going to your house on Port 80 - so you'd likely need to put it behind a cloudflare tunnel for anything resembling reliability. Your ISP may be different and it's always worth asking what restrictions they have on self-hosted services (non-business use-cases specifically). Otherwise going with your ISP's business plan is likely a must. Outside of that, yes, you'll need a beefy router or switch (or multiple) to handle the constant packets coming into your network.
Then there's a security aspect. What happens if you're site is breached in a way that an attacker gains remote execution? Did you make sure to isolate this network from the rest of your devices? If not, you're in for a world of hurt.
These are all issues that are mitigated and easier to navigate on a VPS or cloud provider.
As for the non-technical issues:
There's also the problem of moderation. What I mean by that is that, as a server owner you WILL end up needing to quarantine, report, and submit illegal images to the authorities. Even if you use a whitelist of only the most respectable instances. It might not happen soon, but it's only a matter of time before your instance happens to be subscribed to a popular external community while it gets a nasty attack. Leaving you to deal with a stressful cleanup.
When you run this on a homelab on consumer hardware, it's easier for certain government entities to claim that you were not performing your due diligence and may even be complicit in the content's proliferation. Now, of course, proving such a thing is always the crux, but in my view I'd rather have my site running on things that look as official as possible. The closer it resembles what an actual business might do, the better I think I'd fare under a more targeted attack - from a legal/compliance standpoint.