Comments are pushed out by the community's server, they're not pulled in by yours. So if you're missing comments from communities hosted on lemmy.ml for example, it may be that lemmy.ml is overloaded and not sending out comments to the fediverse properly.
The other common issue with missing comments and posts is misconfigured language settings in your profile. You need to make sure at least Undefined and English are both selected, lots of people only have English selected which will make a lot of posts hidden.