Jeff is not replacable, to me. He is an integral part of the fabric. I also do not believe, for one minute, that they would ask a former survivor to host this show. Not everyone, even actors, have the skill to do this job as evidenced by Chevy Chase. The Mole died when Anderson Cooper left. The host is a crucial element.
In the unlikely event that they replace Jeff, we will see a new face.