weep holes

Any water that manages to get past the exterior seals on the window is directed back out through the weep holes, so no, don't seal them up. I doubt that there is any way for the bugs to get in that way, but if you want to look very closely for gaps on the interior of the windows and carefully seal them that may help. For the sliding windows take a close look at the condition of the weather stripping where the two windows overlap to verify it's in good condition.