Yes, support for more memory is the most observable difference for most users. IMO, few people need more than 4GB of RAM unless they do lots of multi-tasking but there's no hurt in having the ability to add more RAM, if needed. Pretty much all new desktops & laptops being sold today will be 64-bit. Windows tablets and some netbooks may still be shipping with 32-bit.
One caveat regarding Windows systems. A 32-bit Windows OS can run 16 & 32-bit programs while a 64-bit system can run 32 & 64-bit programs. If you have any old, 16-bit programs and you upgrade to a 64-bit machine, you will no longer be able to run those old programs.