The Confederate battle flag’s removal across the South, a necessary change that took far too long to occur, has been debated in the sports world for years.

When Mississippi’s two Republican senators on Wednesday suggested replacing the state flag, they acted almost two decades after Ole Miss chancellor Robert Khayat led an initiative to dissociate his school from Confederate symbols — a move motivated in large part by the mediocre football team’s struggle to recruit black players.

“We can’t recruit against the Confederate flag,” head coach Tommy Tuberville famously said.

The Confederate flag has also cost Southern states the opportunity to host some marquee athletic events. In 2001, the NCAA voted to ban Mississippi and South Carolina from hosting some postseason events, including March Madness, due to “the prominent display or sovereign position of the Confederate battle flag” in those states. (Georgia was initially also included before it removed the Dixie cross from its state flag in 2003.) And since 2009, the Atlantic Coast Conference has refused to allow Myrtle Beach, S.C., to host its baseball tournament after sustained pressure by the NAACP.

Of course, this is far from the first time the economics of sports have been used to further social justice. Integrating baseball wasn’t so much a moral imperative as an economic one — it was about opening up an untapped fan base and talent pool that would field a better, more lucrative product.

That’s something NASCAR, whose fan base probably has a stronger association with the Confederacy than any other in sports, has struggled with. The racing association has joined the chorus of voices backing South Carolina’s removal of the flag, and it has long banned the use of Confederate symbols in any official capacity. Yet it has stopped short of disallowing fans to bear the flag at races. “While NASCAR recognizes that freedom of expression is an inherent right of all citizens, we will continue to strive for an inclusive environment at our events,” the association said in a statement.

It’s certainly disconcerting that in 2015, “freedom of expression” is invoked to protect fans flying symbols of violent racism at the racetrack, but that’s not to say NASCAR can directly ban fans from wearing Confederate paraphernalia. The sport can, however, do more to create that “inclusive environment.”

NASCAR has acknowledged the need to attract fans beyond its aging core, a big part of that being outreach to minority youths. But if the Confederate flag impedes football teams from recruiting black players, what does that mean for racing? NASCAR is caught trying to strike a balance between appeasing its traditional white Southern fan base and opening itself up to future demographics.

Will the Confederate flag be a wedge issue?

Dale Earnhardt Jr., following in his father’s footsteps, has spoken out against the Confederate flag in the past, yet he has acknowledged the risk of alienating his core fans. “I know that these are the fans that pay my salary, so I’m hesitant to tell him the rebel flag represents closed-minded, racist views that have no place in today’s society,” he wrote in his 2001 autobiography.

NASCAR’s initiatives like “Drive for Diversity” are nice, but the real change needs to come from those fans taking it upon themselves to remove the flag, and the hostility it represents, from its races.

And that goes for the rest of sports, and for the nation.

In the wake of the Charleston massacre, we should retire those symbols of “heritage” and “tradition” that serve only to further our disunity. Our sports could help to finally lower that flag once and for all.

Kavitha Davidson writes about sports for Bloomberg View.