Why Agentic Systems Need Ontologies - Frank Coyle, UC Berkeley

My notes

Summary

Frank Coyle (UC Berkeley) argues agentic AI systems need ontologies, formal graph representations of entities, relationships, and properties, as a guardrail layer, since LLMs are inherently probabilistic and hallucination is “a feature, not a bug.” He frames this as neuro-symbolic AI: pairing LLM-driven agent loops with symbolic reasoning and knowledge graphs to validate outputs before they trigger real actions.

Key Insight

• Core thesis: agent loops (tool call -> check stop reason -> act) have no built-in correctness check. An ontology or reasoner sits outside the LLM and validates the tool’s output against domain rules before the agent is allowed to act on it.
• Proposed pattern, “Pydantic at the door, ontology at the ledger”: use Pydantic to type-check tool call parameters going in, then check the semantic correctness of results against a domain ontology before committing any side-effecting action.
• Building an ontology, two approaches: top-down (domain experts define entities and relationships up front, the same method that drove 1980s expert systems, which failed to scale) or bottom-up (derive entities and relationships from real user and customer interaction data as it accumulates).
• Don’t reinvent taxonomies. Reuse established ones: schema.org (general entity and relationship vocabulary), FOAF (“Friend of a Friend”, for social network modeling), Dublin Core (bibliographic and document metadata), DBpedia (the graph that underlies Wikipedia’s own search).
• RDFS and OWL give free inference and constraints that live outside the graph itself. Domain and range: if “teaches” has domain=teacher and range=student, then “Bob teaches Scooter” lets you infer Bob is a teacher and Scooter is a student. Transitive properties: “ancestor of” chains automatically, so Sue to Mary to Ann implies Sue is Ann’s ancestor. Functional properties: “has father” can only have one value, so if two different names both claim to be the same person’s father, that is a signal they are duplicate entities for the same individual.
• Concrete failure modes an ontology catches that plain LLM text reasoning misses: a second refund issued against the same order; a payout routed to the support rep instead of the buyer (a disjoint-property violation between “customer” and “support rep” roles); an invented status value like “probably shipped” instead of a constrained enum of paid, shipped, refunded.
• Design rule: keep agents side-effect-free until validated. Run tool outputs through the ontology check before they are allowed to write to a database, send a payout, or change a record.
• Historical framing: expert systems (1980s symbolic AI) couldn’t scale; neural nets existed since the 1960s but couldn’t scale until GPUs arrived. Agent loops only became Turing-complete once they got iteration (sequence plus conditionals plus loops, per Bohm and Jacopini, 1966), and that same loop capability is what now needs symbolic guardrails to stay safe.