I understand the reluctance to have a former contestant host the show, but still think Colby would be a great choice. He's great looking, in the biz, has proved that he can handle a harsh climate and one of the all time favorite contestants ever.
I think having an unknown or comedian host the show would be too much of an adjustment for viewers. Jeff gets snarky, but doesn't joke about sending people home. Jeff has made the host job important not incidental and while I don't really think he'd leave, a replacement would have to be carefully considered.