In my mind there are two main branches of reasoning: One comes from either racism or a feeling of aggrievement ("why do they get something I dont" kind of thing). The second stems from a misunderstanding of systemic issues, a sort of demographic blindness like "this is a policy that only affects X and that's racist" kind of thing. Arguably the aggrievement fits in here too. This obviously ignores the fact that demographic differences do exist.
Of course there's also the "progressive no" argument that people like Lidia Thorpe argue for, but imo the other two are more common.