This is not true. Data will only be sent to your homeserver if a user on your homeserver joins a room on another server. And only the data for that room is sent, not the whole network. The room data only contains all state changes, and a small amount of recent messages. The amount of state changes is the biggest problem.
Matrix protocol does have a giant problem regarding spam joins though, which make a complete instance basically unusable. Last time I talked with people related to the protocol they didn't want to or know how to fix it, because the need to verify all room state changes.
There's a page explaining it in more detail, but basically, all servers need to verify the complete chain of state events in order to trust data and messages about the room. This is because otherwise malicious servers could make bogus state events and messages that are not valid, like scam messages and unauthorized room setting changes.
In matrix, when you create a new room, or edit room settings, a state event is made. The same is true for changes in user permissions like who is admin, and for settings related to who can join the room.
The last one is key, because this means that in order for servers to trust other servers' messages, they need to verify if the user that sent the message joined the room in a legit way.
In order to do this, when a user joins a room it must cause a state event. However, this makes it easy for people to abuse, by joining a room with a ton of accounts, it spams state events to all connected servers, which bogs them all down because they are required to process all state events in order for chain of trust to function.
Even for rooms with non-malicious usage, servers can still be bogged down if the room is very big, which might be what happened with you or your friend joining a big public room.
Basically, in my opinion, Matrix cannot be used with public rooms as it stands today.