The real difference

The difference is that (most) adults, be they black, white, male, female, gay, straight, pedo, or martian, have already learned SOME set of rules and know how to live by them. They also understand the consequences of following or not following social rules. A child, on the other hand, does not have the life experience to understand full consequences of various decisions. This is true for every aspect of life.

In another thread, we even applied it to sex with different aged children and what "level" of contact they are able to consent to.

Again, you are comparing apples and oranges - immoral laws that suppress or oppress a group of people can not be compared to a system of raising a child.