I personally don't think the plastic rim prevent much deflection since it's so soft. I would say it's safe to take it off.