I'd recommend superglue, but if there are any screws underneath the rubber that you may want to open one day to clean inside, then that's probably not going to work.
Either double-sided tape will work, or single-sided tape wrapped around the lens a few times might enthicken the mount enough to deal with the stretched rubber.
Or a similar idea is with rubber-bands, around the lens and then the grip over them, but then you might get an annoying ridge under the rubber grip.