Changing the Output Voltage of a Power Supply with a Diode?

It sounds like your power supply may have foldback current limiting versus straight line current limiting, based on your description of the issue. If this is true diodes won't help. Best to use a power supply without holdback limiting.

Straight line limiting is when the current stays essentially constant but the output voltage drops when in current limiting. Holdback limiting is when the voltage and the current get reduced to zero when a short is applied.