I put vista ultimate x64 on my machine at work and had a bit of trouble with my nvidia card with it. I'm running a quadro card there(quadro nvs 440), so its a different driver than the geforce chips, but the 64 bit driver would freeze the pc every 10 seconds for a fraction of a second. even replaced the card (had a 1x card, replaced with a x16 card).
did the same exact thing with the x64 linux drivers, but the 32 bit (on both win and linux) didn't do it.
if you ever plan on putting more than 3 gigs of mem in a box though, you'll want a 64 bit os. with a fat vid card (512M or higher) you're gonna lose that from your available memory map (plus some memory for other devices that need their memory mapped). dual cards mean even more mem lost.
PAE would be a way around it, but 32 bit windows doesn't support PAE correctly (even in server OS's).
unless the games or programs you want to run are written for 64 bit you're not going to see any benefit from them, but you will see a benefit on your over-bloated Vista OS.