While you already have a lens cap, I'm not sure you'd want to do this to the interior, but I'll post it any way in case you'd like to give it a try.
I've been sewing for more years than I'd like to count, and my experience has been that felt tends to crush and flatten over time when put to 'utilitarian' use. My husband and I have a vintage lens that we wanted to protect and decided to make our own lens cap cover with a protective interior and hard cover using PVC, corduroy and interlining.
Rather than using felt, we opted to line the interior first with interlining. This can be purchased at most fabric stores that sell upholstery and drapery fabrics. Interlining is what's normally used in the mid section of drapes. Similar to batting, but much nicer and dense. It's a nice, soft fabric that will have some 'give' and have a cushion feel once in place.
After he cut and glued the interlining, my husband then measured and cut the necessary strips from thick black beaded corduroy. He then glued the corduroy on top of the interlining. The ribbing works well to snug the fit and slide.
I've attached two images that show the black PVC Plumbing Cap that my husband purchased at a local hardware store. We brought the lens with us to make sure it was a close fit, then set out to line it with the appropriate thickness to fit snuggly onto the lens. It's perfect and works beautifully for us!!! I've adjusted the contrast/brightness in the photos so that you could see the detail on the fabric that I was referring to.
Outside of Lens Cap Cover:
Interior of Lens Cap Cover:
**Edited to say that the PVC cap itself is quite thick and durable...definitely not a flimsy piece of plastic!