I think it's moreso the fault of the Olympics trying to create a new sport in such a small amount of time. One thats based on a subjective art that didn't have a long history of regulated competition.
The reason we didn't get the absolute best break dancers from every nation is simply because there wasn't an established organization that was already holding regulated competitions.
Eh.... Unless you are actually Japanese, you're probably going to be hanging out with other ex-pats, or just very lonely.
Japan is an extremely conservative and insular country. They don't really mind people visiting for the most part, but they don't really think highly of people actually immigrating there.
There are ethnic Koreans who have lived in communities in Japan for hundreds of years who are still considered outsiders and are treated like second class citizens.