I will never do 15 South from Frederick through Leesburg on a Saturday ever again....getting too built up for the 2 lane stretch of road to handle the traffic and all the locals are out shopping / taking kids to soccer games and whatnot.
270 / 495 is USUALLY not bad at all. Just keep an eye / ear out for traffic reports for any accidents. Saturday through that stretch can still have what seems like "rush hour" traffic just from the the locals out shopping / kids activities traffic, just like the Leesburg area, but not quite as bad IMO.
Now, going to Williamsburg, specifically; I'd suggest that when you factor all of this together, you may "adjust" a little more.
Even once you get to DC itself and start heading down 95 (assuming that the plan by the comments made so far), the be prepared....95 South can get way stupid on traffic for no apparent reason. The last 3 trips (we go from RT15 @ PA line down to W-burg at least twice a year) on that rout have been 1 to 2 hours extra time added for traffic sitting / crawling on 95 in that stretch. Every trip was a "light traffic" time of day, not rush hour, too. Should be an under 4hr trip for us and it has taken up to 7. Accidents aren't even needed - just some whim of certain driver to brake check or a guy changing a tire on the side of the road and you get a back-up for miles.
It may all be perfectly fine and you'll breeze through the area never dipping bellow normal highway speed....but if potential for those slow downs bug you enough, you might want to try avoiding the whole area.
One trip I did 301 / 17 and, while it was slow going, it didn't make me want to rip the steering wheel out and throw at someone in frustration nearly as bad as sitting parked on 95 at 7 AM one Sunday morning...and I got there in like 5hrs.
I also generally find that Baltimore traffic is a lot less likely for those non-rush hour random backups.