As already said , leave the lens on the body unless you want to change lenses .
All the cameras I use have a lens attached , most of the cameras I have , even when not in use tend to have a lens on . ( I probably have over 100 cameras , best not to count them !)
Isn't the Fiji X a digital camera ?
By keep taking the lens off unnecessarily you will be increasing the amount of dust on the sensor .
Even stored with a body cap on dust can still get in .
Leave the lens on and take care when changing .
Some people buy a camera where you can change the lens but never take the kit lens off , ever .
If your happy with what's on , be it a 50mm , a 28-80mm , 18-55mm etc , you may never need to take it off , ever .
Just because you have a camera where you can take the lens off , doesn't mean you have too , especially not a dozen times a day !