It's not quantity with food, it's what you bring in. Fruits, vegetables, meats and dairy are regulated with differing degrees of restrictions.
You have to pay duty on alcohol and tobacco products. However there are personal exemptions for each passenger and the longer you stay the greater the exemption.
Also remember, if anyone in your party has had a DUI or any other felony conviction, they won't let them into Canada.
Like other's have said, you might get across scott free, or it might be your lucky day. They don't kid around, especially the agents on the American side. It's not a big deal. Just make sure you declare everything that you are bringing across. They just confiscate the illegal stuff and make you pay the duty on the other stuff. But they do sometimes they do get testy if you withhold info from them.
If you cross over, why not just wait and buy your groceries over there? Eliminates any problems. Just remember to only buy enough so that you don't have a lot to bring back across.