I think that aside from specific local laws, when you engage in photographing people in public, you should expect the same thing as if innitiating any kind of contact. And yes, it is contact, and it is in fact easily seen as intusive - so shoot at your own risk. I won't hold it against anyone that they took a picture, but... I also will have no sympathy for anyone who gets their teeth kicked in doing it.
Then again, I have no sympathy for people who go hiking and get eaten by bears - you know the risks, you take them, and some times, s**t happens.
Same goes for racing drivers - you go 200mph, you die in the process - too bad, so sad, you knew the physics better than anyone. People who collect stamps don't need a HANS device or a 30-06.
Photography fits in there somewhere. I am obviously a photography enthusiast or I wouldn't be here, and yes, I take street photos on occasion. I have also had photos taken of me in public - and I have to tell you, sometimes I didn't care, sometimes... well, I plead the 5th. I expect the same span of possibility while on the other end of the lens.
And as far as pictures of children go... well, come on: it is bred, taught and further reinforced by events around us to be very protective of our little ones. They are that much more vulnerable. They are that much less capable of protecting themselves and the damage that could be done is so much more long-term and devastating than could be done to an adult.
I doubt anyone here takes photos of children with any ill intent - but they ( the kids) are the one exception I make: ALWAYS ASK, I could give a rat's ass about your artistic vision and intentions, etc., when a child is involved and I fully expect any parent to feel the same. Anyone who fails to see this needs to have their head examined.
Peter.