Man.. I've been through it all with E85/86 Halogen headlights in terms of modifying them to get the best output without buying Xenon version of the headlights (and some vanity of me wanting them looking better), so I have much experience in this.
In terms of halogen upgrade bulbs (Philips Xtreme-Vision, Osram Nightbreakers, etc):
These upgrade halogen bulbs are fantastic! The beam is 100% LEGAL so you will NOT dazzle oncoming drivers with your lights. It will NOT look like Xenon/HID/LED by a longshot. But the range in definitely improved in the Z4's projector lens. The filament is smaller and hotter; therefore brighter, which gives you a more uniform and focused beam than with standard H7 halogens. The difference is NOT dramatic but it is noticeable. To scale it for everyone here: out of 10, the stock H7 halogen gets a 3 in terms of vision. An H7 halogen upgrade bulb (non-blue tinted bulbs) mayyyybe bumps it up to a 5. Not substantial but anything is welcomed. THE DOWNSIDE: The life of the bulbs are much lower than the standard halogen H7s, so be prepared to have a spare in your glovebox if you're doing lots of night time driving.
In terms of LED:
TBH, this is the worst upgrade of the three. The light IS dramatically whiter and actually looks much more modern than a halogen headlight--but the beam pattern is no where near as sharp, uniform or considerate to other drivers compared to halogen. Brightness might be subjective to some people because of the whiter light, it SEEMS brighter but in fact it could be dimmer than an upgraded H7 halogen bulb. Most of these LEDs run at a 5500-6500K color temperature--which is nice and modern looking but if it rains; forget about it. You will barely see the road at night if there's inclement weather. ALSO: If you drive in the winter with snow everywhere, LED bulbs don't get warm enough to melt any ice or snow on the headlight--so it can be dangerous.
In terms of HID/Xenon:
These are the best and it is what I personally use on my Z4. The beam is not legal and there is a little bit of glare but it is mostly controlled by the projector lens itself and the decent positioning of the HID bulb itself. I tried them all: 4,300K, 5,000K, 6,000K and even 8000K. I've gone back and fourth and I stuck with the OEM looking 4,300K. It is by far the brightest color temp, easiest on the eyes, looks the best (IMO) on these cars because it looks like an OEM Xenon headlight and driving with them at night is an absolute joy. You need a CANBUS ballast for these cars because the Z4 checks the bulbs upon startup. If you get a slim CANBUS ballast kit, you can actually fit the entire thing inside the headlight housing and close it with a nice seal--no drilling needed. It won't overheat in my experience since I've ran them for many years.
One thing about the HID/Xenon kit if any of you get it: The bulb return wire (the wire that runs along side the bulb) has to be facing DOWN in our H7 projectors. Most HID kits come with an H7 HID bulb that makes the return wire face upwards inside the projector, which causes a long dark shadow in your beam pattern from the return wire. Most people don't care and it isn't a big deal but if you want the best night time vision with these cars, you'll need to just modify the base by just mimmicking the same tab but on the opposite side. If you do this, you'll get an even, sharp, bright beam pattern on the road ahead.
TL; DR: For the H7 halogen projectors in our cars, HID/Xenon is the best but it is not legal. Upgraded halogen bulbs are the 2nd best but are not as bright as HID/Xenon and do not last as long and are quite expensive for what they are. LED bulbs are mostly junk. They look nice but in terms of actual headlight power they're last place.