It depends on how you want to work and how you want your work to look.
Even with Tri-X or HP5+, a fairly tight window-light portrait at f/11 and a little bellows extension can put you into exposure times of around 1 second. ISO 125 or 50 film is going to take a lot more patience on the part of the model, and you'll probably have fewer "keepers."
If you're using flash, you have your choice. You also have a little more room to play if you are of the "shoot it wide open" school.
The nice thing about higher ISO films in 4x5 is that you will not see any grain in the final prints, so there's no reason not to use them. Tri-X 320 is designed for portraits and gives beautiful results. I'm sure HP%+ and T-Max 400 will do as well.
Peter Gomena