No, the general public has never asked me about the camera format. Only other photographers have mentioned format.
I have, however, had artists ask about the camera I used. When they do, I have to bite my tongue because I want to retort with the question, “… and what brush did you use on your paintings?”
I don't think it tells you anything to note that photographers ask what format or what camera you used for the medium format images while non-photographers don't. Most non-photographers have no idea what different formats mean so would not even know to ask the question.
I think a better comparison, if one wants to do such, would be to show non-photographers prints as otherwise similar as possible from both 35mm and medium format and simply ask which they preferred. Unless grain was a part of the appeal (in which case they'd probably prefer 35mm on fast grainy films) I think you'd start to get answers favoring medium format at 8x10 or certainly at 11x14. For my own uses I don't like 35mm Tri-X, for example, larger than 8x10 for most subjects, while printed 10" square on 11x14 paper from 6x6 or even 11x14 (or as close as possible crop) from 645 I think it still looks superb.
In general I don't like going larger than 8x10 from 35mm, but with some caveats. If you use 35mm like a larger format camera, carefully, with slow to medium speed film, on a tripod, it can look great. But then I might as well use the larger camera.
I think it matters and I find myself shooting more and more medium format. I mostly shoot 35mm black and white when I need fast film (TMZ until now, or Tri-X in Diafine) and fast lenses, or I'm working fast and want to be able to frame with zooms. Otherwise I mainly shoot black and white in either 6x6 or 645. My 645 lenses are 2.8 and 3.5, the Yashicamat 3.5 - the 2.8 645 lenses are fairly fast but I have no zooms, and nothing longer than the 80mm at 2.8; my 150 is the 3.5N. So it's a convenience thing. If I need the convenience of shooting with the 35mm cameras and lenses, I do, otherwise I shoot medium or large format. Color is a bit different because I've been shooting a lot of slides this year and I don't have a medium format projector (yet) so that gets shot on 35mm.
I wonder if the money for the Mamiya 80mm 1.9 would be worth it to me, but even so the shallow DOF would become more of an issue than it is with a 50mm 1.7 in 35mm.
In the other direction, medium format supplants some of my large format. The difference going up to 4x5 is much less in normal print sizes than the difference going down to 35mm. Sometimes I wonder why I bother with the hassle of sheet film at all and don't just sell the 4x5 stuff and get an RB67, but I do enjoy working with the view camera (and even then mostly if I shoot color it's with a 6x7 back, but that's largely due to the insane prices of color sheet film these days.)