The coastline is very accessible around La Jolla Cove, if you can just find a place to park. I’ve been to the caves – to the left in cover shot below – by going through the Cave Store, but the kayak version looks exciting too.
A guide:
http://www.san-diego-travels.com/la-jolla-walking-trail.html