I've had this or a similar problem with the meter, but you're correct in saying that it just keeps working anyway. My one is sufficiently old that it is past repairing and I don't use it that often anyway, but exposure still seems pretty accurate and Ilford XP2 would be printable even if it was a couple of stops out. You'll need the batteries for the shutter release as well as the meter. I'd just live with it!
I've had that happen before too. Since the XA is handholdable at very slow shutter speeds, I never found it to be a problem....then I lost it and had to buy another one
I read somewhere that the XA actualy has two lightmeters. One for the shutter and other for the needle in the viewfinder. If so, then I guess just the needle meter isn't working.