Not sure about your E2 Fastway, but my E2 - 4 point really cannot be used without the bars. The hitch head assembly is on a pivot with a set screw (or bolt) that turns to adjust the hitch head angle easily. If the bars are not attached, the shear weight of the trailer on the hitch will cause the hitch head to pivot downward (backwards). With the bars on, the weight of the camper actually forces the hitch head up until it rests against the bolt adjuster.
My old Reese system had the hitch had fixed. It couldn't be adjusted, but my E2 hitch cannot be used JUST by itself. My old Reese, the hitch head actually bent and there was wear and tear on the metal so the bolts could never tighten enough to stop the slack in the actual head. So I switched to the E2 shortly after we got our current camper.
If the hitch head is fixed and cannot tilt (like mine does), yes, you can tow with just the hitch with no bars. I've moved my camper around with just the Reese Hitch, even though it bent, but not on the road at all.
The answer is, "YES" you can tow without the bars. The question you need to ask yourself is, "Do you really want to compromise the safety and functionality of your system?" Without the bars, the weight on your to vehicle will be distributed far different. For even a short distance, you might not like the way it handles.
All I can say is, hitch up without the bars, and go for a short drive. See how it feels. But take those bars with you. If you drive a thousand feet from your driveway and you realize ... OMG did I make a mistake, you can pull over and hook up those bars immediately and get back in your driveway. If it handles OK, they you have your answer.
