we've been to Crater Lake twice and loved it both times. It's a beautiful deep lake with pretty scenery but some people wouldn't travel 250 miles out of the way to see it. It's beautiful, peaceful and a great place for pictures and a picnic. You can look up the park on NPS.Gov and see if you think it's worth it. Klamath Falls is also worth seeing south of there if you decide to head south before heading east. I would take US97 south to Crater Lake rather than US197 which is more steep.
From another post: At The Dalles, I84 enters the dry side of Oregon. US197 runs south to US97. It has a drop and climb a Maupin.
US97 is further along, and has a gradual climb to uplands of central Oregon.
One way or other you have to climb from sealevel along the Gorge to 5000' around Bend (and points south). But for the most part the climbs are gradual. Once you leave Crater Lake, you can go back north on US97 to US20 and go east to get back to I-84. US20 is a good road. There is a nice little RV park in Burns (Burns RV Park) where you can overnight. Call ahead if you need a pull thru.
Collier Memorial State Park is right off Hwy 97 and has full hookups. Our friends stayed there in a fifth wheel. 45 min. drive to Crater Lake. There are also RV parks at Diamond Lake which is closer to Crater Lake but farther from US97. You can also stop along the way in Redmond at the Expo Center RV Park (no trees) but very nice. There are also several RV parks in Bend.