OK, get an Octopus card (used on all public transport which is clean, safe and very very efficient) and hop on the trains, Star Ferries and Buses. Try some of the street markets, there is a good one in Wan Chai that goes over several blocks and it is near the MTR Station, exit A3 I think, also Shau Kei Wan has a good market and is an interesting area to walk around and photograph. Wong Tai Sin is a very colourful, active and often noisy, Taoist Temple complex complete with fortune tellers, and not far from it is Diamond Hill and its very nice, quiet zen like Chai Lin Buddhist Nunnery and "Scholars Garden". Go back out to Sha Tin and walk up the hill to the 10,000 Buddha Temple, great place and rather photogenic. The back streets of Tsim Sha Tsiu, Mong Kok, Yau Ma Tei. Go to the Bus station under the IFC towers on Hong Kong Island and catch the 6 or 66 (not the 6X) to Stanley, takes you up over the hill to the "South side" and a different, wealthier world. Take a tram on Hong Kong Island. Very cheap, hop on and hop off when you want. Eat, lots. try small restaurants, they may seem a little off hand to you but they are generally helpful and will make suggestions.
How long are you there for, I know a local, foreigner/expat photographer/artist who does guided photo tours I might be able to put you in touch with if you are interested. I think he is in town this week.