My first trailer was an old used 1998 Nash 22H.  LOVED that camper!  Nice big bathroom, solid construction, HEAVY trailer for a 22' (mine came in at 6500 lbs loaded) and a good layout for my wife and I.  We introduced our daughter to camping in that trailer a month after she was born.  For a little one, it was great.  When we got pregnant with child #2 we knew the 22H wouldn't work out for us.
Before kids we were hardly inside the camper.  We ate outside, sat outside, listened to music outside, rode bikes outside, etc.  With a toddler we had to be concerned with making it dark for nap time, had to worry about interior floor space to play, had to worry about enough interior space for all of us if it rained, etc.
We upgraded from our 22' Nash to a 32' Jayco with a big slide.  HUGE difference in camping style.  We find that the longer camper does effect where/when we stay.  With the 22', we could literally fit on ANY camper site in any park.  With the 32' we have still been able to camp at all of the same places, but we have to plan ahead and make sure to get a longer spot.  Some parks make that more difficult than others.  We had some issued with our steep and narrow driveway with the longer trailer.  Fuel mileage took a dip with a longer and heavier trailer.  Other than that, there aren't many negatives to a longer trailer.
I think you'll find the move from the 22' to 25' to big a big upgrade, but not much more difficult to tow, back or find spots to camp.  That slideout makes an incredible difference in interior space.  No more shuffling past your bride or child in a tight hallway between the dinette, couch and kitchen... now you can walk past each other without any problems!  The only thing you will be lacking is that couch, which we enjoyed having.