There is fog, and there's fog. A slight fog (I'm suspicious of your safelight) could be causing you grief. When I was taking photography courses, one of the other students complained the safelight in the school darkroom was fogging his paper. The teacher poo-poo'd his claim, and none of the rest of us had noticed anything. Then he came to class with 2 prints. A window, taken from outside, mostly dark tones, but a lily on the sill made a lovely bright contrast. The first print he showed us was very nice. We all admired it. Then he showed us the one he had made in the dark, with the safelight off throughout exposure and development. There was fine detail and differentiation in the highlights (the petals of the flower) that were missing from the first print. There was no visible fogging of the borders of the print, and if he hadn't been determined to get the details onto the paper that he could see in the negative, and clever enough to figure out the problem when it had gone undiagnosed for who knows how long, we would have all continued printing with slight fogging for the rest of the year.
This was decades ago, so I don't remember now how the safelight problem was rectified. I've never forgotten though, those 2 prints side by side. Same exposure, same paper, but the highlights subtly fogged under the safelight. Lesson learned.