Many years ago, I worked as a Canadian Customs officer, at a US/Canada border crossing.
We had lists of agricultural products that we had to watch for.
The problem isn't with an apple that came from a grocery store. It is with an apple where you can't tell where it came from - an agriculturally inspected orchard, or the traveler's cousin's backyard apple tree with its charmingly tasty and pest or disease infected apples.
The restrictions are generally only there when the apple is being imported to a country that has an apple industry of its own, although it wouldn't surprise me if the EU members may cooperate to protect all their crops.
For that reason, Canada had no restrictions on importing oranges, but the US officers handling traffic the other way did have restrictions on oranges going the other way - the US citrus production industry is/was huge, and back then Canada imported a lot of oranges from other parts of the world that did have pest or disease issues that the US had under control.
I'll always remember having a car arrive at the entry point with a woman and a bunch of kids in it. They were desperately trying to finish eating all their blueberries before I had to tell them that they couldn't bring them in. I paused for a moment before telling them that there were no current restrictions on blueberries at the time

.