If your children's main concern is only what if this and what if that and you think you are more than capable of continuing to do what you want in a safe manner then continue doing that...
I'm sure you realize that the time will come when you will have to give up what you love and from what I read you have a plan B...the trick here is realizing when the time has come based on what you feel and what your children are trying to tell you...
There is one thing I have always tried to tell people based on whatever situation they are in in... When something becomes more of a pain than a pleasure it is time to move on...
In the mean time...HAPPY TRAVELS TO BOTH OF YOU
