Exactly! And for the record, white women were also part of this struggle and were also in the middle of it. Really, the true adversary was not the entire group of whites, rather the males of that particular group who manipulated pretty much the entire system to work in their favor.
The system of racial oppression in this country was perpetrated by both White men and White women, it wouldn't have been as successful and lasted as long as it did otherwise. Look at any pics of lynching parties back in the 1920s and 1930s and you will see BOTH White men and White women skinning and grinning at the dead Black body hanging from the tree.