I would agree with you that there is something not quite right with either the controller or the battery. It does appear to be operating as if the battery is fully charged and the controller is in constant voltage mode when from the battery state of charge it clearly shouldn't be. It does seem to be a shortcoming of either the charge controller voltage and current sensing (which could also explain the 0.3V offset you see) or the charge controller firmware. Can you update the firmware?
This maybe another case of false economy, the Renogy is cheaper, but is also not as well designed and supported product as the Victron. As far as I know, Victron is a real hardware development company - they have engineers in house in the Netherlands that design and test all their hardware, whereas Renogy is a branding company that primarily buys hardware designed and built in China with their brand/logo on it.
