Someone mentioned a children's museum at the Palace of Fine Arts. I think they are referring to the Exploratorium
http://www.exploratorium.edu/
But it has moved and will soon open at Pier 15 on the waterfront. A great place for kids of all ages, and for adults. Highly recommended, even without seeing the new place. It has been designed by the same firm that did the Monterey Bay Aquarium (another highlight going south), and the Exploratorium people pioneered the whole hands-on science thing, so it should be good.
The Americas Cup sailboat racing is happening this summer. I don't know the schedule. Its main public relations area is right down the waterfront from the Exploratorium, and its main building and piers should be interesting without an actual race. In fact, the actual race days might be a good day to go away from the waterfront!
http://www.americascup.com/
Then head down to the Ferry Building for some nice food. Maybe keep heading south and catch a ball game at AT&T Park, or maybe they'll be simulcasting the opera with free admission.
http://sf.funcheap.com/free-opera-ballpark-att-park/
I'll agree with others- avoid the Haight Asbury. Tourists and panhandling kids and speed freaks.
And the Fisherman's Wharf area- basically tourist country. If you do end up there, find the energy to keep going west along the waterfront. The Maritime Museum is at Aquatic Park, and the Hyde Street Pier is a collection of great old ships of all types. Go up to the community garden at Fort Mason, then down to the old piers where troops left the US for the Pacific theatre in WWII.
Dolores Park can be a nice afternoon. On the edge of the hipster part of the Mission district. There are some interesting stores on Valencia that would entertain kids. The Dave Eggers literary/art project site at 826 Valencia is worth dropping into- interesting displays, items and books for sale, etc.-
http://826valencia.org/
Anyone into pirates needs to visit 826-
http://826valencia.org/store/
A few 'curiosity shoppes' with stuffed animals, skulls, bones, etc. further down the street Then head over to 18th Street and Pizzarea Delfina, then get some Bi-Rite ice cream and walk into the park. They recently renovated the playground at the south end- lots of families, musicians, etc. This whole thing- Valencia, Dolores Park, etc.- is a weekend thing for the crowds and energy. Somewhat monocultured, dominated by young techies.
I would suggest avoiding the California Academy of Sciences. Expensive, small, not very kid friendly from what I have observed (I go there and photograph on free days, and notice the lack of engagement by most kids). Well, I'm just one person. If you are thinking of dropping a couple hundred bucks to go there, do some research and see what others have said.
Angel Island is worth a visit. A ferry ride to it, then continue to Tiburon and eat at one of the waterfront restaurants. Then the ferry back to SF. Among other things, Angel Island was a major entrance point for immigrants from Asia.
http://en.wikipedia.org/wiki/Angel_Island_(California)
Sausalito is another tourist trap, especially on weekends.
Try not to wear shorts. If you plan to move more than five three blocks in one day, you will most likely end up in cold wind. Carry clothes for 50 degrees F and fog, no matter what it looks like when you head out.
Oh, photographing people: almost no one cares. The only problems I have had are with street kids who see every encounter as a chance to hustle some money, and older Asian street people (not certain of the reason but am pretty certain it includes Chinese for 'fu** you' and other bodily manipulations). So many tourists, so many people using phones to post every other minute to instagram, just be aware, be ready to smile. And if you are shooting film, be ready to discuss your camera, films, etc....
Oh yeah, speaking of film, there's a nice store in the Lower Haight area, Glass Key Photo. Facebook page only, but worth a visit if you get nearby. Near to Alamo Square and the 'Painted Ladies' houses, near Memphis Minnie's BBQ joint, Rookie Ricoardo's Record Store, Upper Playground if any of the kids like skateboards or street art...
http://www.facebook.com/pages/Glass-Key-Photo/119199231470385