That's not normal. Mine snap back without issue taight to the seats.
You do not say what age of car but a couple of pointers:
Have you tracked the belt right back to the retractor? It's very common on some BMW's to get a twist in the belt. I had this on both rear belts on an X5 and it made them very poor at retracting, but until I removed the trim I could see no reason.
Secondly while investigating that issue it was pointed out to me that fluff builds up in the mechanism and is also a cause for slow retracting. Suggestion was to vacuum out. Obviously if new an unlikely cause.