No, the yurt that removes the character from the game is the one that returns it. Should that yurt be destroyed the character remains RFG for the rest of the game.
While it was mentioned previously in this thread that in response to a character being yurted your opponent may be capable of killing him or the yurt, in response to his response, if you have a second yurt, is to use the second yurt to attempt to save the character again. This can be repeated for however many yurts. He will re-enter play from the yurt used last should he be saved.