For the same reason that I don’t believe gay couples should be legally forced to accept services from a MAGA photographer. A private contract in this situation is just that, a contract. Both parties have the power to set whatever terms and conditions that they want.
For example, imagine a black couple wanted a photographer for a family event and said something along the lines of “we’d like to support members of the black community by hiring an independent black photographer.” If a white photographer saw this and sued, everyone would (rightfully) react negatively to him