I guess it depends. What do you mean by "a safe place"?
One of my favorite places is across the highway from Neptunes Net at the county line. It is north of LA proper. Haven't been there in years. It used to be a pretty tough beach - but the violence was usually out in the water, during the day. Surfers can be a very a territorial lot.
Except for stong winds and uncooperative light / weather, I never had any problems in the late afternoon there. If you don't look like a tourist, you'll be fine.