Don't use glue if you don't have too. Remember, some poor sod, and it might be you, is going to have to replace the seals in time. A friction/stuff fit, with possibly a dot of glue at a few spots, is more than adequate.
If you do need glue then use something easily removable either with mechanical action or solvent. Rubber cement is a good choice. Pliobond/contact cement is great for gluing kitchen counter-tops and shoe heels - and a very poor choice for attaching a sliver of yarn or foam in a hard-to-clean channel.
Any damn fool can glue two things together. The art is in making it so the two things come apart easily when you want them to.