I keep JAVA disabled in my browsers by default. There are a few sites that I rarely access which require JAVA to function, network related, but don't need it otherwise. Don't mistake it with Javascript, which is completely different, and integral to the functioning of most websites. You can't post to these forums without Javascript being activated.
Flash has become just as big a problem as JAVA. I keep it as Ask to Activate, as there are just too many vulnerable issues that crop up with it on a frequent basis. It's become hackers preferred exploit tool as it has allowed remote machine control on not just Windows, but on MAC and Linux machines. The latest vulnerability being the perfect example. Some hackers have even injected YouTube videos with code in an attempt to exploit vulnerabilities.
The problem has become so critical that some browsers will disable Flash if you don't have the most current patched version. Others, such as Chrome, auto update Flash plugin.