Sasha Boguraev

We show how causal interventions in Transformer models provide insights into English syntax by focusing on a long-standing challenge for syntactic theory: syntactic islands. Extraction from coordinated verb phrases is often degraded, yet acceptability varies gradiently with lexical content (e.g., "I know what he hates art and loves" vs. "I know what he looked down and saw"). We show that modern Transformer language models replicate human judgments across this gradient. Using causal interventions that isolate functionally relevant subspaces in Transformer blocks, attention modules, and MLPs, we demonstrate that extraction from coordination islands engages the same filler-gap mechanisms as canonical wh-dependencies, but that these mechanisms are selectively blocked to varying degrees. By projecting a large corpus of unrelated text onto these causally identified subspaces, we derive a novel linguistic hypothesis: the conjunction "and" is represented differently in extractable versus non-extractable constructions, corresponding to expressions encoding relational dependencies versus purely conjunctive uses. These results illustrate how mechanistic interpretability can inform syntax, generating new hypotheses about linguistic representation and processing.

@article{boguraev2026causal,
  title   = {Causal Drawbridges: Characterizing Gradient Blocking of Syntactic Islands in Transformer LMs},
  author  = {Boguraev, Sasha and Mahowald, Kyle},
  journal = {arXiv preprint arXiv:2604.13950},
  year    = {2026}
}

Sentences like "She will go to France or Spain, or perhaps to Germany or France." appear formally redundant, yet become acceptable in contexts such as "Mary will go to a philosophy program in France or Spain, or a mathematics program in Germany or France." While this phenomenon has typically been analyzed using symbolic formal representations, we aim to provide an account grounded in artificial neural mechanisms. We first present new behavioral evidence from humans and large language models demonstrating the robustness of this apparent non-redundancy across contexts. We then show that, in language models, redundancy avoidance arises from two interacting mechanisms: models learn to bind contextually relevant information to repeated lexical items, and Transformer induction heads selectively attend to these context-licensed representations. We argue that this neural explanation sheds light on the mechanisms underlying context-sensitive semantic interpretation, and that it complements existing symbolic analyses.

@inproceedings{boguraev2026france,
  title     = {France or Spain or Germany or France: A Neural Account of Non-Redundant Redundant Disjunctions},
  author    = {Boguraev, Sasha and Yao, Qing and Mahowald, Kyle},
  booktitle = {Proceedings of the Annual Meeting of the Cognitive Science Society},
  volume    = {48},
  year      = {2026}
}

Many languages mark either accusative case (for objects of transitives) or ergative case (for subjects of transitives), but some "split ergative" languages mix the two systems depending on the type of nominal. It has been noted that these languages tend towards marking the less frequent case for each nominal type. This raises the question of what mechanism could underlie the emergence of such an efficient system. We propose a model that can provide an explanation, based on a simple reinforcement learning framework and simple assumptions about asymmetries between the kinds of nominals (e.g., pronouns vs. full noun phrases) that appear in subject vs. object position.

@inproceedings{boguraev2025reinforcement,
  title     = {Reinforcement learning produces efficient case-marking systems},
  author    = {Boguraev, Sasha and Erk, Katrin and Mahowald, Kyle and Shearer, James and Wechsler, Steve},
  booktitle = {Proceedings of the Annual Meeting of the Cognitive Science Society},
  volume    = {47},
  year      = {2025}
}

Math is constructed by people for people: just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models are trained on reflects not just idealized mathematical entities but rich communicative intentions. We contend that treating mathematics as situated linguistic communication offers benefits, particularly for language models: we present two main findings that language models interpret the equals sign in humanlike ways, generating different word problems for identical equations presented differently, and that they prefer logically equivalent proofs when ordered naturally.

@inproceedings{boguraev2024models,
  title     = {Models Can and Should Embrace the Communicative Nature of Human-Generated Math},
  author    = {Boguraev, Sasha and Lipkin, Ben and Weissweiler, Leonie and Mahowald, Kyle},
  booktitle = {4th Workshop on Mathematical Reasoning and AI at NeurIPS'24},
  year      = {2024}
}

@mastersthesis{boguraev2024what,
  title  = {What Do You Mean by That? - Idiolects, Casual Miscommunication, and the Evolutionary Fitness of Languages},
  author = {Boguraev, Sasha},
  school = {Cornell University},
  type   = {Undergraduate Honors Thesis},
  year   = {2024}
}