WARNING: SPOILERS AHEAD
Some screencaps from the cbs.com promo for episode five:
1) We see that Travis (or possibly Chris) remains on Lopevi, as he is swimming to the Lopevi buoy. In the background, we see Lisa (neon green bathing suit) standing on the same platform as Rory.
2) We see Twila and Sarge are both members of Lopevi.
3) We see that Leann, Eliza, Ami, and Lisa remain on Yasur. We can also assume that Rory has been swapped to Yasur.
4) We see an albeit blurry shot of one of the women in a red and white tanktop- previous appearance can let us ascertain that it is Julie. She remains on Yasur.
So here is where we stand:
Yasur: Rory, Eliza, Leann, Lisa, Ami, Julie
Lopevi: Travis, Lea, Twila
Unaccounted for: Chad, John K., Chris, Scout
Now, realise that Yasur has a numbers advantage and will sit out one member during the diving challenge. I wouldn't sit out John, Chad, or even Chris over Eliza in that particular challenge. So it's logical that Scout remained on Yasur and sat out of the challenge. This means John, Chad, and Chris stay on Lopevi.
The tribesman asks for only one chief from each tribe. It's not too farfetched that Yasur would pick Twila and Lopevi Rory (at least to get him off their hands for a while.) It looks like, a la Apprentice, only one person from each team is switching: Twila and Rory.