It's the issues with XMPP's spec: you don't just use XMPP, you use XMPP + your favorite optional spec implementations.
If your friends aren't on the same server/client combo then you won't be able to communicate with them (effectively).
I loved XMPP, still do, but haven't used it in years. If it were to get a single, matrix-style "spec release" (think an aggregation of existing features into one collection) that contains/requires a bunch of modern chat features I've come to expect from programs, then I could see it potentially having a resurgence.
I like this reddit comment's explanation: