Its a mental left-over from the repressive victorian era. People still see sex as something "bad" and "dirty" they still feel a guilty pang in themselves that they enjoy it, but this "naughtiness" is part of the thrill.

It essentially why our society is so messed up, our very psychology is based around thinking ourselves to be unclean in someway deep down.

This weird way of looking at sex isn't something a child could possibly understand, this is what causes them problems. If they enjoy sex at a young age, they eventually learn that it is shameful, but find difficulty in linking that with their own experiences. Its not just a matter of discounting society as the social norm is a HUGE factor on a developing mind.

Until we as a society break down the barriers around sexuality, paedophilia will never come close to being accepted.