Depends on how well behaved the child is. If you are babysitting a unlikeable brat than yeah that can be challenging and give you headaches. But I find that the majority of children are fairly easy to manage. Then there are those girls who you like so much it actually hurts not to be around them.
I've baby sat very often in my life. My parents actually ran a daycare so I was use to it.
Biggest trouble usually comes from babysitting two children that don't get along and you have to deal with constant fighting.