For me for fly out of Japan, the air fare is less of a concern, usually, but the other expenses such as hotels/hostels, food, as well as the conference fee(s) will be.
So, I wouldn't just go to the conference in the cities like Tronto and come back within my time frame, which is just too costly and too insufficient. I've been to the city of Tronto before, stayed about a week, loved it, actually.
It might be better if the conference time will be shared with our vacation time in a way and leave us enough free time to fit in our schedules. Most people have jobs, other jobs, and their "prior commitments" etc.
For example if the conference is held 3 days at the most on the weekend (Fri-Sun) as intensive workshop like 12 hours (with enough break every two hours) on each day, that will leave extra couple of days of free time and the time for traveling with in 7 days.
If some people can afford and plan to stay longer than that, they can get together and learn more from each other afterwards. So, others won't feel that they have missed something when they get home.