Driving in downtown Toronto will be no less difficult than finding parking will be.
I would highly recommend you consider using the excellent public transit system in Toronto instead. It's very clean, safe and will be a bargain compared to trying to drive / park downtown. The subway comes up into the Yorkdale Shopping Mall (Hwy 401 at Dufferin Street) which makes it easy on / easy off. There are several others that are similar in the Toronto area also.