The yeast Golgi membrane protein Rer1p is required for the retrieval of various endoplasmic reticulum (ER) membrane proteins such as Sec12p and Sec71p to the ER. We demonstrate here that the transmembrane domain (TMD) of Sec71p, a type-III membrane protein, contains an ER localization signal, which is required for physical recognition by Rer1p. The Sec71TMD-GFP fusion protein is efficiently retrieved to the ER by Rer1p. The structural feature of this TMD signal turns out to be the spatial location of polar residues flanking the highly hydrophobic core sequence but not the whole length of the TMD. On the Rer1p side, Tyr152 residue in the 4th TMD is important for the recognition of Sec12p but not Sec71p, suggesting that Rer1p interacts with its ligands at least in two modes. Sec71TMD-GFP expressed in the Deltarer1 mutant cells is mislocalized from the ER to the lumen of vacuoles via the multivesicular body (MVB) sorting pathway. In this case, not only the presence of polar residues in the Sec71TMD but also the length of the TMD is critical for the MVB sorting. Thus, the Rer1p-dependent ER retrieval and the MVB sorting in late endosomes both watch polar residues in the TMD but in a different manner.