It reminds me of Google back in the day (probably early 2010s). If you searched for White Women, it returned professional and respectable images. But if you searched for Black Women, it returned explicit images.
Machine learning algorithms are like sponges and learn from existing social biases.