Sounds to me like the pilot being lit opens the main gas valve to the stove, then only the user controlled valves have it turned off. Either the main valve, or one of the user controlled valves is leaking. The problem is not in the pilot light, itself.
Start checking at the place the thermocouple attaches to the main valve, and check all connections heading toward the stove eyes for leaks.
Bobbo and Lin
2017 F-150 XLT 4x4 SuperCab w/Max Tow Package 3.5l EcoBoost V6
2017 Airstream Flying Cloud 23FB