How Do Language Models Represent and Use Phonological Information for Allomorph Selection?

The latest research confirms that despite masking the phonetic structure of words during training, language models can still reliably generate morpheme forms constrained by phonetic conditions. Regarding the selection mechanism for the indefinite articles ‘a’ and ‘an’ in English, the study found that these phonetic conditions are encoded in a single linear direction of the trigger word embedding, and this direction因果ly drives the article selection in the token-level ‘wug’ test. When predicting the position of articles, the model predicts the upcoming trigger words and uses the phonetic features of these predicted words to decide whether to use ‘a’ or ‘an’. These results indicate that language models possess the ability to make cross-linguistic and explicit phonetic judgments based on rules, providing a mechanistic explanation for the selection of morphemes constrained by phonetic conditions, and distinguishing this ability during generation from metalinguistic judgment.