Although I live pretty far from Tokyo and hardly visit there, I get asked the same question every once in a while. The following is what I told another fellow not long ago.
For mostly new stuff:
Yodobashi Camera
http://www.yodobashi.com/enjoy/more/index/index.html
Map Camera
http://www.mapcamera.com/
Bic Camera
http://www.biccamera.com/
Yodobashi is the mecca, but that doesn't mean they have everything unlike B&H. Map Camera deals with used gears also, and it's a pretty good store. Bic Camera is a corporate chain and you can find stores in other big cities besides Tokyo.
For used ones at good price (like KEH-ish):
Fujiya Camera
http://www.fujiya-camera.co.jp/
Camera No Nitto
http://www.nitto-camera.co.jp/
These two stores above are in the area called Nakano, and if you take a train (JR line) and get off at Nakano stop it's right near the stores. You take the north exit. Fujiya Camera has a good map:
http://www.fujiya-camera.co.jp/floorguide.shtml
And Camera No Nitto is a litle north of it within a walking distance.
For Leica, I don't know any of them personally, but if you go to Ginza district, check out:
http://www.sukiya.co.jp/sukiya.S.top.html
Also in Ginza, this shop has some variety:
http://www.ginichi.com/
I don't know the prices in Europe, but compared to the prices in the U.S., more expensive in general. The only big benefit you can get in the Japanese market is that you can find the Japanese-only products that you wish you had in your market...
To take photos, wherever you feel like. Just take a walk and snap. I think the whole culture is so unique and peculiar. Oh, the famous Tsukiji fish market (where you can watch big tuna getting chopped up) will probably be removed to a new location in the near future, so check that one out if you have a chance. A lot of fish sellers are currently opposing this plan because the new location is so contaminated and cannot be clean for selling fish.