Suppose you're hiring. You're also a racist douche. In particular, you believe that (other things being equal) a white employee would be more productive on the job than a black employee. As a result, you're willing to pay more for white employees than for equally qualified black employees. Your preference for white employees not only expands the demand for white labor, but also contracts the demand for black labor. This raises the price of white labor, while lowering the price of black labor. Consequently, your non-racist competitors substitute towards black labor in order to reduce their labor costs. If your belief is wrong (which it is--you're a racist douche, remember), then your competitors can sell the very same stuff as you for lower prices, without sacrificing quantity or quality. Unless consumers boycott your competitors for employing black people, they will drive you out of business. Only if your belief is correct, or if the the market for your goods or services is non-competitive, will this fail to occur. Free, competitive markets, in due course, put arbitrarily prejudiced employers out of business.
Suppose instead that black people tend to be less productive than white people (other things not necessarily being equal). The result is that blacks are not proportionally represented in different walks of life (e.g., underrepresented among business executives, overrepresented among manual laborers). Would we really want them to be better represented at the expense of these jobs being performed less productively? Or would it be better to first let competitive markets sort workers on the basis of productivity, and then enable blacks to share in the wealth? In other words, which is better: a bigger slice of a smaller pie, or a bigger slice of a bigger pie? (Derp.)
Alternatively, consider the possibility that blacks are underrepresented in certain walks of life, and overrepresented in others, because they have different preferences from whites. What good does it do black people, or the broader society, to forcibly interfere with their choices?
If underrepresentation is due to arbitrary discrimination, then this probably has more to do with the unintended consequences of government policies (e.g., the drug war) than with competitive labor markets. If it is due to bona fide differences in productivity, the best course for everyone involved is to let the market rip, but provide means by which the resulting prosperity may be broadly shared. If it is due to differences in preferences, then the government can only make things worse. Frankly, I do not know what else could be the cause.
We find, in none of these cases, a compelling rationale for the government making distinctions between races. Even if blacks happen to be disproportionately poor, or low-skilled, or whatever, the case for helping them economically is just as strong as the case for helping similar non-blacks economically. Don't get me wrong: slavery is the single worst thing this country has ever done. Racial segregation is certainly up there, too. But if there exist alternatives to preferential hiring policies that make literally everyone better off, victims of these past injustices and their descendants included, why should we settle for preferential hiring? Why shouldn't we try to do better?