# Relational NLP Before Transformer Language Models **Domain:** Natural Language Processing / Knowledge Representation / Semantic Web / Artificial Intelligence **Doc Type:** Historical Technology Node **Maturity:** Developed **Cutoff:** October 4, 2017 **Related:** [[wiki/WordNet|WordNet]], [[wiki/Knowledge Graph|Knowledge Graph]], [[wiki/RDF|RDF]], [[wiki/Natural Language Processing|Natural Language Processing]], [[wiki/Knowledge Representation|Knowledge Representation]], [[wiki/Symbolic Language Engine|Symbolic Language Engine]], [[wiki/Relational Reanimation|Relational Reanimation]] --- ## Definition **Relational NLP before Transformer language models was the broad family of computational-linguistic systems that represented language and knowledge through explicit senses, grammatical structures, semantic roles, frames, entities, ontologies, typed edges, and traversable graphs.** These systems modeled different layers of language without treating all intelligence as next-token prediction. This node records the research environment surrounding Bryant McGill's symbolic language work and the January 2016 and October 2017 broadcasts that described knowledge as relational trees. It includes only resources and systems publicly available by October 4, 2017. Later additions found in the evolving research notes are outside this historical cutoff. ## The Research Stack | Layer | Systems available by the cutoff | What the connections represented | |---|---|---| | Lexical meaning | WordNet, Wiktionary, BabelNet, Universal WordNet, MENTA, Etymological WordNet | Synonymy, antonymy, word sense, taxonomy, derivation, etymology, and multilingual equivalence | | Grammar and semantic roles | Penn Treebank, FrameNet, PropBank, VerbNet, Valex, Engtwol | Syntax, frames, predicates, arguments, verb classes, morphology, and the roles entities occupy in events | | Ontologies and knowledge graphs | OpenCyc, Freebase, DBpedia, YAGO, UMBEL, Wikidata, Google Knowledge Graph, Knowledge Vault, FrameBase | Entities, classes, facts, inheritance, identity, event structure, and world knowledge expressed as typed relations | | NLP pipelines | Stanford CoreNLP, Apache OpenNLP, SENNA, MaltParser, PIKES | Tokenization, tagging, parsing, entity recognition, semantic-role labeling, coreference, and extraction of structured knowledge from text | | Semantic-web infrastructure | RDF, RDFS, OWL, SPARQL, Apache Jena, Linked Open Data, HDT, Linked Data Fragments, LOD Laundromat | Interoperable statements, schemas, inference rules, queries, compression, publication, and federation across heterogeneous graphs | | Graph inspection | UCINET, Gephi, sigma.js, Cytoscape.js, WebVOWL | Visible topology, clusters, centrality, paths, density, and the large-scale shape created by many local relations | | Distributional relation | word2vec and corpus-derived trigger relations | Statistical proximity, analogy, association, and the changing relevance of one term in the presence of another | The categories overlap by design. BabelNet joins lexical and encyclopedic knowledge; YAGO joins Wikipedia entities to WordNet taxonomy; FrameBase joins linguistic frames to linked-data knowledge bases; PIKES converts several kinds of linguistic analysis into RDF statements. The integrations were important because no single representation captured the whole object called language. ## Intelligence in the Connections An isolated token supplies identity without consequence. A typed relation changes what the system can do with the token. A WordNet hypernym makes inheritance traversable; a FrameNet role identifies how an entity participates in an event; a PropBank argument distinguishes agent from patient; a YAGO or DBpedia edge locates an entity inside world knowledge; an RDF predicate turns two identifiers into a claim that can be queried, joined, contradicted, or used in inference. The operative intelligence therefore resides neither in the node alone nor in an undifferentiated pile of edges. It resides in organized relation: the type of connection, the direction it runs, the constraints it carries, the provenance supporting it, and the other paths with which it composes. A graph becomes a model when its connections preserve enough structure for a later process to recover implications that were never stored as isolated sentences. ## Bryant McGill's Position in the Field Bryant's work was centered on [[wiki/Natural Language Processing|natural language processing]], computational lexicography, linguistic representation, constrained generation, and knowledge architecture. It was AI-adjacent because those representations were built to support machine retrieval, traversal, inference, and generation, while remaining distinct from the neural-network research path that would later dominate the public meaning of artificial intelligence. His notes show sustained work across the stack: downloading and comparing datasets, studying schemas, examining graph formats, testing NLP pipelines, visualizing relations, and asking how an open system could integrate lexical, semantic, encyclopedic, and social knowledge without surrendering provenance. Many resources were followed from their first public releases or earliest usable datasets rather than discovered retrospectively after knowledge graphs and language models became fashionable. [[projects/Ten Years Building a Symbolic Language Engine|Ten Years Building a Symbolic Language Engine]] documents the implemented core: formal grammar, WordNet, corpus trigger relations, pronunciation, morphology, taxonomy, phrase evidence, and constraint-based generation. The broader research stack shows the direction of travel from that engine toward interoperable knowledge graphs and machine-operable world models. ## Transformer Chronology The phrase *pre-Transformer* requires one precise qualification for the October 4, 2017 broadcast. *Attention Is All You Need* appeared as an arXiv preprint on June 12, 2017, fewer than four months earlier; its conference presentation followed at NIPS in December. The architecture therefore existed, but it had not yet become the dominant language-model substrate later associated with GPT, BERT, instruction-following systems, or large general-purpose models. The January 4, 2016 broadcast was unambiguously pre-Transformer. The October 2017 broadcast belongs to the brief interval in which the Transformer had been disclosed as a machine-translation architecture but had not yet reorganized the field's understanding of language models. Describing Bryant's position as **NLP-centered and AI-adjacent** accurately locates the work without retroactively turning explicit lexical and knowledge-graph systems into LLMs. ## Corpus Function - [[livestreams/Opening Your Genius Through Language, Abstraction, and Manifesting - Livestream#Language Is an Executable Model of Reality|Language Is an Executable Model of Reality]] develops the January 2016 relational-tree and lexical-closure account. - [[livestreams/From Audience to Author With Tools for Participatory Publishing - Livestream#The Symbolic Engine and the Neural Model Preserve Different Truths|The Symbolic Engine and the Neural Model Preserve Different Truths]] documents the explicit linguistic engine in the October 2017 broadcast. - [[blog/20171004 - The Intelligence in the Dictionary|The Intelligence in the Dictionary]] connects the historical symbolic stack to relational reanimation and later neural language models. ## Key Insight **The intelligence preserved by a knowledge system resides in the typed relations that allow one represented thing to constrain, explain, or transform what another represented thing can mean.** ## Sources - Princeton University, [About WordNet](https://wordnet.princeton.edu/) and [WordNet history](https://wordnet.princeton.edu/frequently-asked-questions). - Collin F. Baker, [“FrameNet: A Knowledge Base for Natural Language Processing”](https://aclanthology.org/W14-3001/) (2014). - Maddalen Lopez de Lacalle, Egoitz Laparra, and German Rigau, [“Predicate Matrix: Extending SemLink through WordNet Mappings”](https://aclanthology.org/L14-1475/) (2014). - Roberto Navigli and Simone Paolo Ponzetto, [“BabelNet: Building a Very Large Multilingual Semantic Network”](https://aclanthology.org/P10-1023/) (2010). - Fabian M. Suchanek, Gjergji Kasneci, and Gerhard Weikum, [“YAGO: A Core of Semantic Knowledge”](https://doi.org/10.1145/1242572.1242667) (2007). - DBpedia Association, [About DBpedia](https://www.dbpedia.org/about/). - Xin Luna Dong and colleagues, [“Knowledge Vault: A Web-Scale Approach to Probabilistic Knowledge Fusion”](https://research.google/pubs/knowledge-vault-a-web-scale-approach-to-probabilistic-knowledge-fusion/) (2014). - PIKES, [Knowledge Extraction Suite](https://pikes.fbk.eu/) and [2015 evaluation](https://pikes.fbk.eu/eval-pm.html). - Ashish Vaswani and colleagues, [“Attention Is All You Need”](https://arxiv.org/abs/1706.03762), submitted June 12, 2017. - [[projects/Ten Years Building a Symbolic Language Engine|Ten Years Building a Symbolic Language Engine]].