He's talking about milk teeth, not adult teeth. There is no implication that teeth are related to memories, just that your milk teeth come out at around the same age that your memories are wiped.