One of the reasons it remains an apples to oranges question, even if the focal length and aperture are the same, is that the image circles on the film are different. The 35mm lens is designed to project an image circle not much larger than the diagonal of the 35mm film area. Some are right on the edge, and with certain apertures, you'll see vignetting at certain apertures.
A 100mm lens for medium format will project a substantially larger image circle onto the film. But lest you think this means you'll get the same angle of view (same "zoom") as a 35mm, you won't. On medium format, a 100mm lens seems like a "wider" lens. But that said, you get the same qualities as the 35mm lens in terms of image distortion - a good quality 80mm lens on medium format, while giving a "normal" angle of view, is still a good portrait lens. You can just get a bit closer because you have a wider angle of view on the film than with 35mm.
When you move up to large format, the lenses get less and less complicated. Does that make the image quality better? I have no idea, someone who is an expert should weigh in. I don't know the qualitative impact of a larger image circle, but an expert can probably talk about the tradeoffs of image circle and the multiple layers of glass required to deliver a sharp result on 35mm vs, for example, 4x5.