We aren't the touristy types either. We don't need to see or do every nook & cranny of a place. We more or less just get the flavor of an area. So, we went to US side one day & looked around for a couple of hrs., and then went to Canadian side the next day & did same. We didn't do any of the extra activities. We enjoyed going to Niagara on the Lake and having lunch.
So, it really depends on what type of "tourist" you are. If you want to see and do everything, I would think you'd need 4 days to have time to enjoy and not exhaust yourselves.