You may have run into why polarizers are generally not a good thing.
First (and this is not the bad thing about polarizers), polarizers have an overall darkening effect, because they cut out half of the light (at least - more depending on the material used in the polarizer). They do that always, no matter which way the polarizer is rotated. This is your fixed filter factor.
Secondly (also not quite the bad thing, but already part of it), polarizers darken bits selectively. That's why you use them.
That can however already lead to images appearing too dark, depending on how big the part of the scene is that is affected by the polarizer. The fixed filter factor (of course) does not compensate for the selective darkening effect of the polarizer. But (in a worse case scenario) your entire scene is affected by the filter... If you do not like that, use less polarizer.
Thirdly (another part of the bad thing about polarizers), polarizers can remove reflections. That includes specular highlights in your scene. And there are very many of those in outdoor shots. The result (even if the overall appearance of the resulting image is not too dark) is a dull, flat looking image.
Combined, the effect of a polarizer can be quite horrible.
And it is easily mistaken for underexposure. That, because it is, but selective underexposure. You elected to do that, and it is not (!) fixed by exposing more, but by using less polarizer.
So only use a polarizer when absolutely necessary.
If you should think it is necessary quite often, use your head and eyes, and don't just go for maximum effect.
If you go for maximum effect anyway, look at what that does to parts of the scene you did not pick the polarizer for (it's o.k. to want to darken the sky, but what does that do to leaves, grass, etc.?) You may then want to rethink what you want to do to the sky.