I love being around kids, obviously little girls preferably, and I prefer this to most other social interactions. Still, I do enjoy and seek the company of adults as well.
I've had little girls friends grow up, and most of them still love me and being around me and talking with me. We are not really romantic anymore sadly and none of them know I am a pedophile. So I can't comment there.
I get your angry and frustration though. It is ironic, the only people in my life that would not care if I told them I was a pedophile would be the little girls I have become so close with during my life. Nearly every adult would care, most would care a lot, in a bad way.
So while most people have a side of themselves they keep hidden, for us it is much bigger.
Anyway, we are not natural enemies with other adults, it's just a bad social time for us to live in.