Firstly, the furnace should run the blower for a time after the burner shuts off to extract the maximum heat possible for the living space.
Beyond that, I'm baffled that merely taking the t-stat cover off should enable the burner to cycle on and off. Only thing I can think of is that somehow the cover interferes with the contacts. You might look to see if there's any obvious mechanical reason for this. Absent finding such a reason, I'd just replace the thermostat entirely. They are pretty cheap and the job is really simple; couple wires, couple screws, job done.