Often, you will have two fresh water connections. One connects to a fresh water hose (usually labelled "city water") and runs directly to your sink, shower, etc. The other doesn't have a fitting, and is for the fresh water tank.
The fresh water tank is for when you can't connect to city water, and you need to bring your water with you. I usually keep this tank 1/3 full, for emergencies (water issue at the campground, stopping to go to the restroom, etc), but don't use it if I can connect at the campground.
If you're connected to city water, you're not using your fresh water tank at all.
However, you said that your fresh water tank is connected to your "city water" inlet, so I'm not sure what's going on there. Some have a bypass to allow your fresh water tank to be filled from the city water inlet, so check to see if you have any valves that bypass one way or the other.
Annoyingly, my fresh tank inlet is on the wrong side, so I want to install a bypass so that I can fill my tank from the city water.