50 Amp Pedastal/Nr. 4 AWG

How are you going to terminate #4 onto the 50 amp recept. terminals? You probably need to splice #4 to a smaller gauge at the recept. to make it work. If so, the splices will be physically large and require a large junction box adjacent to the 50 amp recept. Same at your panel. I think you'll need split lugs to do this. Before you use the #4 you might want to price out the lugs and junction boxes vs buying some new wire.

Absolutely no reason not to use the #4 wire, except as above and also as noted by another poster above, the insulation jacket on the wire must be code approved for the location and application.