Are we just talking arresting brown people, or should we start arresting anyone who speaks something besides German in public, since they’re eroding the culture too?

The fuck are you on about.

If you want to get a handle on this I suggest you start with concepts such as the assimilation capacity of a population as well as the possible speed of different kinds of natural cultural drift. If you want to avoid to avoid fuelling xenophobic ressentiment, what you need to make sure is that cultural drift caused by new arrivals is lower than what people accept, in that case people become more tolerant of that kind of shift, though of course that has a limit (and that's fine). OTOH if you exceed it, people become less tolerant of shifts. In other words: Culture is a non-newtonian fluid. You create resistance by pushing too hard, if you go in gently there very well might be no resistance at all.

The erm force applied to that non-newtonian fluid is more or less number of arrivals multiplied by germane cultural difference multiplied by economical impact. When Germans flock to Sweden the Swedes worry about those "closed up and private" people, they're somewhat taken aback by directness but secretly also somewhat glad that there's someone actually complaining in public, not just in private. In the numbers that we're talking about the Swedes aren't worried in cultural terms, though there's some gripes among some around housing prices in rural areas (not among the Swedes selling the houses, of course). Berliners are way more worried about Swabian arrivals.

And, really, let's take Sweden as an example because they've been so... Swedish about the whole thing. Over decades their immigration worked just fine, they had a certain number and that number didn't exceed the assimilation capacity, and then Swedes said "we are the best so of course we'll take in more" and more came and assimilation failed -- and the Swedes, being Swedes, never complained in public. It's a high-trust environment, of course you trust others, even if government policy led to, one way or the other, segregation: Arrivals live in one place, native Swedes in another. Which then makes it even harder for the new arrivals to even acculturate much less assimilate, leading to more segregation, leading to more difficulties. At some point a dam broke and Swedes stopped complaining in private and complained in public -- the backlash. Which led to people who were born in Sweden from perfectly assimilated parents suddenly found themselves on the outside of their own culture.

If, instead their politicians had started early saying "we need to actively work against that segregation, we need to change our public housing policy to make sure that neighbourhoods are mixed, and if that doesn't suffice we need to limit the number of new arrivals" things would've went very differently -- such a policy would have increased assimilation capacity. But that would've implied things such as Sweden not being perfect which is unthinkable to a Swede... at least to say aloud. Fucking swots.