Lithium is the lightest metal, atomic number 3. It's the prime component in one of the major drugs that fights bipolar disorder, lithium carbonate (discovered after lithium saccharin was used in Diet 7-Up shortly after cyclamates were banned). It is chemically similar to sodium and potassium, not calcium, and thus is not retained in the bones.
All batteries should be disposed of as specified by your local solid waste authority, but lithium won't leach into groundwater and poison your great-grandkids if a few of the batteries go to a landfill, the way mercury, cadmium, and lead can.
BTW, even strontium and cesium are hazardous mainly in their radioactive forms -- they're so rare in the environment you'd never notice them in your bones unless they were continously dosing you with alpha particles, and the only significant source of the radioactive forms is nuclear fission.