The Canadian Lodge is reputed to be among if not the best hostel in North America (as espoused by their website, many online reviewers, and a friend of mine who has done a lot of backpacking and staying in hostels in North America and Europe--he said his next trip is to South America!). One thing he said was that staying at a hostel is for those who can pack light. You are in a room with a number of other people who are usually complete strangers and so you have to be able to pack everything and carry it with you when you leave the hostel to avoid theft (after 6 years of "hostelling" he has never had anything stolen, but he says that it is still best to keep the opportunities for it to happen at a minimum). Many hostels now offer private, secure lockers to their guests into which you can lock your backpack at night.
Because of the location away from the conference I am hoping to split a hotel room a few minutes away from the gallery by foot and then if I have time to stay in Toronto longer (fingers crossed!) I am planning on switching to a hostel in downtown.