It should just drift out, getting them back in is worse, how are you planning on removing the hub? i used long M12 bolts through the hub and pushed against the backplate but the backplate gets destroyed, didn't matter as it was the backplate i was changing, have seen it done with a slide hammer but didn't have the space in my garage.