We stayed at rainbow Springs State Park in Florida for a week from December 20th to 27th. On the 27th Saturday we began to disconnect and put away some things prior to leaving before the 1 pm check...
If you paid to stay there until 1 PM then that is your right to stay there until that time, regardless of what anyone else says. It's in the contract you signed to rent that RV spot until you leave. If that guy asked you if you were leaving 1 PM, then all he has to do is reserve your site with the Park Ranger then leave you alone. There is no need for him to come back to your RV spot and encroach on your RV spot and ease drop on your conversation with the Ranger in your rented space.