I've found going either way through large or small ports is all about the same.
The larger ports I've found can be easier to navigate with the RV than some of the smaller ports. Portal, ND in 2014 had a height restriction going north where our RV would have never fit if we didn't stay in the "commercial" lane. That port took 90 minutes to get through due to the traffic going north at the time we hit the port.
Check the US & Canadian Border Patrol Crossing times, they keep it upto date and will give you an idea of the delay for both US and Canadian sides of the same port of entry.