Sometimes if your water system is pressurized and you turn the heater on, the thermal expansion of the water will overpressurize your system a little and cause this to leak. Just run some hot water out of the inside faucet to lower the pressure back to normal. Or it could be some dirt caught in the seat of the relief valve. When the heater is off and the water in the tank is cold, remove the red dust cover, get a plastic bag to cover the electrical connections, stand clear of the outlet and lift up on the silver lever. This will blow some water out (probably quite forcibly) and might clear the dirt out if that is causing the problem. Or just change the safety valve for one of a similar size and pressure rating.
You never want to plug this or your water heater could explode if the thermostat fails and it overpressurizes. Mythbusters had an interesting episode on what happens if you plug the safety valve on a house water heater and then overheat the water.
I also drain and flush out my tank occasionally (usually when winterizing). It looks like you can do that with the plastic plug in the lower left of the photo.