A quick google on "How to use a voltmeter" should get you about 10000000 hits. They're relatively simple. Two leads, a dial to set, and a meter or digital readout. You could even use a simple test light to see if there's power. Getting to know how to use a test light, a volt meter, and other simple diagnostic tools is PARAMOUNT when owning an RV, since there are several areas that could cause trouble. The plumbing is not usually so bad. But as you've noticed, there are several levels of complexity regarding the electricity.
The way to check the dash battery gauge is to throw a meter on the battery one at a time, If it shows good voltage, then the problem is upstream. If it doesn't, then take the battery(s) back to Advance Auto and have them replaced under warranty. Again, any semi-qualified auto guy should be able to show you how to do it.