Draft:Geometric Context Modeling

• Comment: In accordance with Wikipedia's Conflict of interest guideline, I disclose that I have a conflict of interest regarding the subject of this article. Tiffney Renee (talk) 21:29, 22 June 2026 (UTC)

---

Geometric Context Modeling is a proposed paradigm in artificial intelligence introduced by Tiffney Bare in May 2026. It proposes that conversational context should be represented as an explicit trajectory through a structured geometric semantic space, rather than as a probability distribution over token sequences (as in statistical language modeling) or as a set of discrete facts and rules (as in symbolic knowledge representation).

The paradigm identifies a gap shared by both dominant AI foundations: neither maintains a continuous, explicit, geometric representation of where a conversation currently is in semantic space. Geometric Context Modeling proposes a third foundation in which the basic unit of meaning is a position in a structured space with defined axes, distances, and directions. KAIA (Knowledge Architecture for Intelligent Agents) is the first implemented system in this class, introduced alongside the paradigm by the same researcher.

Definition

Geometric Context Modeling is defined as the practice of representing, maintaining, and reasoning about conversational context as an explicit trajectory through a structured geometric semantic space. Each of the three words in the name carries a distinct technical meaning.

Geometric means the representation uses position in a space with defined structure: axes that correspond to semantic dimensions, distances that reflect semantic similarity, and directions that correspond to semantic change. The reasoning operations available are geometric operations, including projection, nearest-neighbor lookup, centroid averaging, and directed stepping. No probability distribution is required. No discrete fact is required. The position is the meaning. The representation is inspectable: at any point one can ask where a concept sits on a given axis, how far it is from another concept, and in which direction it leads from its current position.

Context means the system models where a conversation is rather than what word comes next. A language model asks: given what has been said, what is most likely to be said next? A context model asks: given what has been said, where are we right now in semantic space? These require different computational structures. Next-token prediction requires a probability distribution over vocabulary. Context position requires a geometric state in semantic space. Generating text is optional and secondary to the context model.

Modeling means the system maintains a structured, continuous, dynamic representation that can be queried, updated, and reasoned over at any point. In Geometric Context Modeling, the model is the trajectory: the history of where the conversation has been, weighted by recency, is the current state. The state is explicit, readable, and auditable.

Motivation

Limitations of statistical language modeling

Statistical language models learn the probability distribution of tokens given prior tokens from large text corpora. Transformers and large language models are all statistical language models. Their representation of meaning is implicit, distributed across billions of parameters, and fundamentally tied to the statistical patterns of whichever corpus was used for training. Context in these systems is a sliding window of prior tokens, not an explicit representation of where the conversation is semantically. The system cannot be queried about where a conversation currently sits on a given semantic axis, because no such axis exists in the representation.

Limitations of symbolic knowledge representation

Knowledge graphs, ontologies, expert systems, and frame-based systems encode meaning as discrete facts, entities, relationships, and rules. They are interpretable, auditable, and logically consistent within their defined domains. But they operate over discrete symbolic structures rather than continuous geometric spaces. They cannot represent the graded, continuous nature of semantic position. They cannot perform geometric operations such as finding the midpoint between two concepts or stepping along a semantic axis.

The shared gap

Both paradigms share a deeper limitation. Neither represents the semantic state of a conversation as a continuous geometric object. Neither maintains a persistent, explicit position in a structured meaning space that updates as the conversation progresses. Neither can state, at any moment in a conversation, exactly where that conversation is located semantically, with what intensity, across which dimensions, and with what recent history. Geometric Context Modeling is proposed to fill this gap.

Theoretical basis

Meaning as position

The foundational claim is that meaning can be usefully represented as position in a structured geometric space. This draws on two bodies of prior work in cognitive science. Conceptual space theory (Gardenfors, 2000) proposed that concepts are regions in a multi-dimensional quality space defined by perceptual and conceptual dimensions. Conceptual metaphor theory (Lakoff and Johnson, 1980) demonstrated that abstract concepts are systematically grounded in spatial and physical experience.

What Geometric Context Modeling adds is computational instantiation: a working system with a structured space, defined axes, efficient geometric operations, and a persistent state that tracks the trajectory through that space.

Context as trajectory

Conversation is treated as a path through semantic space rather than a sequence of tokens. Each word moves the current position: words with strong semantic content move it far; function words and discourse markers move it slightly; confirmations hold it roughly in place. The path has direction, velocity, and history.

A State Space Model (SSM) captures this naturally. The current context state is a weighted average of all prior semantic positions, with recent positions weighted more heavily. Decay rate is a tunable parameter per axis, allowing different semantic dimensions to have different memory characteristics. Temperature context fades differently from social context, which fades differently from temporal context.

Relationship to existing approaches

Approach	Continuous geometry	Persistent trajectory	Explicit axes	Semantic operations	CPU-native fixed memory
Statistical language models	No	No	No	No	No
Contextualized embeddings	Partial	No	No	Limited	No
State space models (Mamba)	No	Yes (unstructured)	No	No	Yes
Semantic trajectory analysis	Yes	Retrospective only	No	Limited	No
Symbolic context systems	No	Yes (discrete)	Partial	Symbolic only	Yes
Geometric Context Modeling	Yes	Yes (real-time)	Yes	Yes	Yes

Contextualized word embeddings

ELMo (Peters et al., 2018) and BERT (Devlin et al., 2019) introduced context-sensitive word representations. The same word receives different vector representations in different sentences. This is a step toward context-sensitive geometric representation, but contextualized embeddings require running the full model on each sentence at inference time. There is no persistent trajectory; no accumulated record of where the conversation has been; each sentence is processed in isolation; and the representations are trained through statistical objectives rather than geometric reasoning operations.

State space models

Mamba (Gu and Dao, 2024) and related structured state-space models compress sequence history into a fixed-size state vector that evolves with each new input. This is architecturally close to the SSM component of Geometric Context Modeling. But current SSMs are designed as language models: their state vectors are learned through next-token prediction objectives and their dimensions carry no semantic interpretation. One cannot ask an SSM state vector where the conversation is on the temperature axis, because its dimensions were never defined in those terms.

Semantic trajectory analysis

Recent work in cognitive science and clinical NLP has framed concept production as movement through embedding space, computing trajectory metrics such as velocity, acceleration, entropy, and distance to centroid. This work explicitly treats meaning as spatial and context as trajectory, but it analyses trajectories retrospectively for clinical or research purposes rather than maintaining a real-time trajectory for reasoning. It uses transformer-based embeddings rather than a structured geometric space with defined axes.

Knowledge graphs and symbolic context systems

Symbolic context modeling systems maintain structured representations of active entities, their relationships, and their relevance to the current conversation. These systems are explicit and auditable, but they operate over discrete symbolic structures rather than continuous geometric spaces. They cannot represent the graded, continuous nature of semantic position and cannot perform geometric operations.

Open problems

As identified in the introductory position paper (Bare, 2026), the following problems remain open as of the time of publication.

Culturally contingent axis grounding. Physically grounded axes (temperature, speed, luminosity) encode clean geometric opposition in distributional embeddings. Abstract axes (moral, epistemic, existential) do not, with top-1 accuracy confirmed at zero across six experiments and three embedding approaches. Resolving this requires either purpose-built embeddings trained with explicit opposition objectives, multi-corpus grounding combining the cultural frameworks of multiple linguistic traditions, or a compositional approach that decomposes abstract concepts into combinations of physically grounded dimensions.

Decay rate derivation. The SSM decay rates in the KAIA implementation are currently set by intuition. No experiment has measured the actual temporal dynamics of semantic context in real conversation. Deriving empirically validated decay rates from annotated conversational data is an open problem.

Route library construction. Route discovery identifies how many semantic routes each polysemous word has in a given corpus. Building the route library at scale, from examples rather than retraining, requires a principled construction method that has not yet been defined.

Multi-corpus integration. Different corpora produce different geometric representations of the same words. A framework for combining route libraries from multiple corpora, handling cases where routes exist in one corpus but not another, is an open architectural question.

Evaluation suite generalization. The seven-benchmark suite used in KAIA was designed for agent-oriented reasoning tasks. Extending the evaluation framework to clinical semantic analysis, educational dialogue, creative generation, and other domains is future work.

First implementation: KAIA

KAIA (Knowledge Architecture for Intelligent Agents) is the first implemented system in the Geometric Context Modeling class. It encodes meaning across a 13-dimensional space defined by antonym pairs; maintains context as a 52-byte fixed-size SSM state vector; retrieves from a dimensional database at 0.065 microseconds per lookup regardless of vocabulary size; and operates at 44,000 to 97,000 tokens per second on a standard consumer CPU with no GPU requirement.

Across a seven-benchmark suite of agent-oriented semantic reasoning tasks, KAIA achieved 70 to 85 percent accuracy, with 100 percent geometric convexity confirmed across 15 consecutive experiments. The system was developed by Tiffney Bare as an independent researcher between April and May 2026 without institutional affiliation, grant funding, or GPU hardware.

Further reading

• Bare, T. (2026). "Geometric Context Modeling: Introducing a New Paradigm for Semantic Representation and a New Class of Artificial Intelligence System." KAIA Research Series, Paper 5. Zenodo preprint.
• Bare, T. (2026). "Physical Universality vs Cultural Contingency: Why Geometric Semantic Architectures Work for Physical Dimensions and Fail for Human Conceptual Categories." KAIA Research Series, Paper 4. Zenodo preprint.
• Gardenfors, P. (2000). Conceptual Spaces: The Geometry of Thought. MIT Press.

Category:Artificial intelligence Category:Natural language processing Category:Geometric semantics Category:AI paradigms Category:2026 in artificial intelligence Category:Independent research