You may or may not have a problem. I would not check the refrigerator temperature for 24 hours after starting it. That the freezer is already making ice has no bearing. The cooling cycle for an absorption refrigerator starts at the freezer; i.e., the most cooling is there. Moreover, the freezer has a much smaller volume from which heat must be absorbed.
If the empty refrigerator has not cooled 24 hours after starting, then I would become concerned. (Adding food to the refrigerator will lengthen the time required to reach an acceptable temperature.)
Tom
