The level of noise is very subjective, complicated by different N52s suffering from this condition much more than others. Apparently, some are so loud, there is clearly something wrong. Others just sound like a solid lifter engine with a few valves in need of adjustment.
For most, the noise isn't so much cold related as related to length of trips. People that tend to take short trips will get the noise, those that only take longer trips, say more than 30 min., may not notice anything. Another factor is what rev range one typically uses. Higher revs tend to ensure the HVAs fully fill with oil.
But considering short trips can mean cold engine, maybe there is a relationship. If it just sounds like lifters out of adjustment, there's little reason for concern. Just keep short trips to a minimum. If it sounds like hammering so loud that something could be damaged, then maybe you should consider replacing the lifters.