The required precision for setting the flange distance can be understood by considering the depth of focus at the film plane. A common value for acceptable circle-of-confusion for 35mm is 0.03mm, meaning a spot size of 0.03mm is considered "acceptable." Most depth-of-field marks on 35mm lenses correspond roughly to this choice. However, for critical focus/best results many people want to do a little better, and rate depth-of-field one or two stops more conservatively (a factor of 1.4-2x in spot size).
Let's say we are going to be about 1.5x more conservative, so our acceptable spot size is 0.02mm. Further let's say that we want to be able to achieve this with a f/2 lens. An f/2 beam emerging from the lens is a cone whose taper is: length is a factor of 2 times the base diameter. This means that the focus offset (length between the true plane of focus and the film) can be 2x the spot size, 2 x 0.02mm = 0.04mm. So the maximum error of the position of the film plane is +/- 0.04mm. That's really quite small - 40 microns, about half the diameter of a human hair. And, if one is 0.04mm off, one has used up all the conservative depth of focus in the film plane (depth of field in the subject plane) at f/2, which is not desirable. So when manufacturers specify the flange-film distance to 0.04mm or better, they aren't just being persnickety. It really does need to be that accurate.
As others have mentioned, you might get away with a small error with an SLR, as long as the focusing screen is off by the same amount (IOW, if you move the lens flange in or out, it should still focus ok, but the focusing scale may be off and you might not reach infinity). But for an interchangeable lens rangefinder it's got to be quite close.
I was negative early in the thread about improvising the measurement, because typical home tools like a caliper aren't precise or accurate enough. As others have said, a dial indicator, an accurate length standard (gauge block), and a very rigid fixture are needed. I'm glad to see that people have gotten it to work mechanically. If one isn't ready to buy that gear, I think optical alignment (autocollimation) is better than trying a vernier caliper.