Moisture is the issue
If you intend to pull it out, and use it, let it warm up for an hour or so. That way any moisture in the air won't want to condense and sink into the otherwise chilled emulsion side of the paper (asuming we are talking RC), or both sides of FB paper. When you are done with pronting, fee free to fold the plastic liner bag over, once or so, and slide the works back into the outer pocket. The amount of air that the thing exchanges to the next time it opens is quite small.
The fridge can be a rather humid environment.
I store most of my papers in the freezer, where the humidity levels are usually quite a bit lower, since there are no open liquids stored there, generally only foodstuiffs that have been first plastic wrapped.
If I am going to print, I take the paper out first thing in the morning, or when I get home from work, and once supper is done and dishes awaym the paper is a near to room temp, and ready to go. If I pull a whole bunch of papers out, I spread them out on the enloarger desk, rather than stacking them in the dark drawer to help them warm up faster.
