Sound like a nice custom order. Good luck with your decision. I too would vote you keep the 650i and ditch the Volvo...but I also certainly understand the need to negotiate.
By the way, the 4.8i with the sport package doesn't necessarily require you to go with performance tires. The 19" 255/50 tires on 211 rims come standard with the 4.8i w/sport (the 20" low profile performance tires on the style 214 y-spokes are an option). Granted these 19s are slightly lower profile than the 18s (roughly 1/2" less sidewall), but I didn't notice a difference in ride comfort on a test drive (and they shouldn't be more susceptible to damage than the 18s). What I definitely did notice was the active roll stabilization on the sport package when cornering hard. It was almost impossible for me to induce ANY body lean on an X5 with the sport package. The electronic damping control didn't seem to change the overall ride very much (actually if anything it was more supple...not what you would normally expect from a "sport suspension").
All that being said, I think the X5 w/out the sports package handles incredibly well. So if you don't plan on driving too agressively, you may be better off saving $3700 (especially if you don't care for the anthracite headliner with the beige interior). However, you shouldn't shy away from it purely for concern about the performance tires.