Not the same dilemma, but I provide taxi services to an elderly parent so Z4 only was never an option!
I had a 325ti Compact as a daily until last month when I replaced it with an E91 325i SE that cost me less than a third of your lower budget (mind you it is a bit leggy).
But it's a great place to spend some time when I'm on the M3, M25, A3 commute to my Mum's place, and it's showing 35.7 mpg on the OBC now! Sparkling Graphite with Lemon leather, and it even has heated seats - lovely! If I want a back road blast that's a job for the Z4, and I like the contrast between the two.
I read loads of horror stories about BMW N47 engines and cam-chains years ago which was one of the main reasons I sold my 123d at 81K miles in 2014 while it still worked and bought a Z4C - obvious alternative really!
Just google BMW N47 cam-chain for some insights - it isn't happy reading.
I wondered about a diesel myself, but my local BMW Indy recommended 6 cylinders, and ideally petrol - which is why I bought the 325ti which I was very pleased with, and with that advice ringing in my ears the 325i Touring happened and is looking promising so far! But he also said that in a diesel E9* model he reckoned the N57 engine was a better bet than the M57 early ones had.
But I realise an 80 mile commute makes a diesel look really appealing, for the economy if nothing else (OBC on my 123d was showing 48.7mpg when I sold it).
If you love your Z4 and will get the chance to use it properly now and again I'd be tempted to keep it and get a 325d or 330d with the N57 engine to go with it.
But if you decide you won't really get to use your Z4 so you might as well just have one car I'd probably look for a low mileage 335i or 330i - that way at least you can some fun when you get the chance!
Good luck whatever you decide - and sorry to be the "bringer of bad news" on the 320d option.