I did mine while I had the manifold off to remove the ccv. Took a whole day for the manifold but the starter was very easy with it out.
People have done it from below but I'm not quite sure how you can angle torque it properly. Taking the intake off before the throttle is quite easy so you could do that and see how much room you have with it out.
It's the solenoid not disengaging the starter motor quickly enough after it starts so it makes a horrible noise. Mine does it too and has done for the last 12,000 miles. Perhaps foolishly I'm waiting until it dies...
