I'm not sure I completely agree with the "cohesive show" idea -- but maybe, like so many things in life, it depends on your definition of "cohesive."
For some further insight, I edit and publish a newsletter and maintain a website (at my expense) for a local art club I belong to. That puts me on the board and I have been entangled in this dubious business of mounting shows for a dozen years. Our organization, like many, has about 110 members, of whom about 9 do most of the work. It's like pulling teeth to get people to actually start the process far enough ahead of time to pull it off without last minute panics. As such, we frequently are left with accepting judges who are available and willing to do it for a fairly meager stipend, rather than any fancy interview and selection process. We try to have different judges every year and not repeat a previous judge for eight or ten years. Often we are left with getting recommendations from previous judges or other art groups. With that sort of approach, I could hardly say we grill the candidates about their philosophies.
In my experience, some of the best judges are those who teach art, especially at the college/art school level. Though they may personally specialize in one area, their career requires a broader understanding of the art world. Our annual open show is divided into five major categories -- oils, watercolors, "graphics" (prints, drawings, pastels) sculpture amd photography. The general intent is to exhibit the best work in each of those categories. So for us, I guess "cohesive" is the best of a diverse selection of subjects, media and styles. There are special awards that can go to any piece, but we have a 1st, 2nd and 3rd in each category to make sure there is some spread. (We do have a disclaimer that only a 1st may be given if there are too few entries in a category.) There are always results at the end that others may question, but no matter how objective anyone tries to be, it's still a crap shoot.
The show that started this thread I view as fairly prestigious and established, so I would expect it to be well run. But my own group has been doing their open show for fifty-one years and I know what it's like from behind the scenes, and one is optimistic to expect too much formal organization. What appears to have happened might have gone a little better if they had multiple judges, but of course that increases the expense. Some juried shows use separate people for selection and awards too.
Every year some of us scratch our heads about what we could do better or more efficiently, but in the end I'd say there's not a whole lot of difference from "what we did last year."
As eddym alluded to, some of us begin to experience burn-out!
In the end ya pays yer money and ya takes yer chances.
DaveT