A crack or break in the exhaust system would make that noise. Just driving around the back pressure is not enough to make a noticeable difference in sound. Under load the back pressure is increased and you are hearing unruffled exhaust. If you put a gloved hand over the tail pipe opening, you should hear the leak (and with help be able to find it) - if that is the issue. If there is no leak you will not hear a change, and the pressure will want to push your hand away. Don't leave the pipe blocked long. This is a two person test, that is do not block the pipe and then go looking for a leak. You will hurt the engine and/or cat.
