Draft:Neurorepresentationalism

Neurorepresentationalism is a theory of consciousness inspired by contemporary neuroscience and philosophy of mind. The theory characterizes conscious experience as a multimodal, situational survey generated by the brain's representational systems. It was developed by Dutch neuroscientist Cyriel M. A. Pennartz at the University of Amsterdam as a specific form of representationalism grounded in empirical and computational neuroscience. Although representationalism is often taken to be reductionist, neurorepresentationalism is a form of non-reductive physicalism.

Overview

Neurorepresentationalism holds that conscious experience consists of richly structured neural representations that integrate information from multiple modalities into a unified whole. For instance, when we witness an illusion such as Kanizsa's triangle, the illusory properties are not part of the physical world, nor do we localize them literally in the brain – they are rather regarded as properties of a representation synthetized by the brain, which is experienced as being localized outside the brain. Consciousness can thus be identified with specific representations generated by the brain (in particular thalamocortical systems), rather than being something over and above representational brain states.

Neurorepresentationalism bridges the philosophical notion of phenomenal consciousness with mechanistic knowledge of the brain's anatomy and physiology, such as insights on neural coding, network dynamics, topologies and predictive processing. It proposes criteria for identifying consciousness in non-human animals, AIs and other artificial systems, and patients with disorders of consciousness.

Core claims

Neurorepresentationalism (NREP) posits that conscious experience is characterized by five hallmarks: multimodal richness, situatedness and immersion, unity and integration, dynamics and stability, and intentionality.

• Multimodal richness: consciousness involves integration and segregation of information from multiple sensory modalities (such as vision, audition, somatosensation and the vestibular sense) into a single, coherent conscious experience. Segregation or self-identification of modalities takes place based on input statistics, uniqiue feature detectors and the relational topology between modalities.
• Situatedness and immersion: conscious experiences are embedded in a spatially extended environment that encompasses both body and surroundings. The agent finds himself in the midst of the environment.
• Unity and integration: Diverse features are unified into a single experience at a given time, based on a multitude of integrative subprocesses (e.g., binocular integration, feature binding, multisensory integration and saccadic integration). Configurations of objects in the environment are generally experienced as stable, which is enabled by including motor information into brain mechanisms for perception.
• Dynamics and stability: Conscious contents change over time yet display sufficient temporal stability to support coherent perception and action. The world is experienced as located outside our own body, not in our brain.
• Intentionality: Activity patterns are interpreted by the brain as being about objects, events, and states of affairs in the world, not merely as internal signals.

The theory further distinguishes between three (healthy) modes of consciousness: perception (externally driven experience), imagery (internally generated and under substantial cognitive control), and dreaming (internally generated, with reduced cognitive control) as different modes of conscious representation. The biological function of consciousness is to provide a situational, multimodally rich survey to inform brain systems for deliberate decision-making and goal-direction behaviors. In contrast, systems for habits or reflexes are not dependent on such a survey. A key tenet of neurorepresentationalism is that the conscious mind is understood as arising emergently across different levels of representation. At the lower two levels (neural and computational), this concept is similar to Marr's levels of computation. Phenomenology however arises as a third main level of representational organization. This tripartite divison has been refined by applying knowledge on (i) predictive processing networks for single features; (ii) unimodal networks, which unify different features into e.g. visual objects; (iii) metamodal networks that integrate spatially and across modalities in very large multisensory topologies. These representational levels are not to be confused with anatomical 'levels' in the cortical sensory hierarchies.

Comparison to other theories of consciousness

In contrast to other predictive processing accounts, neurorepresentationalism provides a more detailed characterization of the neural basis of representations and expands on the multi-level organization of the conscious brain. Contrasting with another predictive processing account, Active Inference, NREP does not assume that motor behavior or active sampling is necessary for consciousness. At the core of consciousness lie mulitmodal integration and segregation. NREP emphasises that basic predictive coding models mediate low-level computational operations, which collectively generate a 'superinference' at the highest representational level, corresponding to phenomenological experience. The theory differs in major ways from Integrated Information Theory (IIT), which is much more permissive of (partial) panpsychism, and from Global Neuronal Workspace which focuses on access consciousness and global availability of information.

Applications and implications

In clinical contexts, the theory is used in discussions on brain-injured patients and disorders of consciousness, as it has suggested various Indicators of Consciousness in patients unable to express themselves. In a similar vein, the theory has proposed Indicators of Consciousness to estimate whether consciousness occurs in animal species. Scoring animal species on various markers such as brain anatomy and physiolougy, goal-directed behavior, susceptibility to illusions, episodic memory, multisensory integration and segregation yields a total score which can be used to estimate an animal's likelihood of possessing consciousness. Additional requirements have to be set for consciousness in AI systems. It further informs philosophical debates on the nature of qualia, the unity of consciousness, the relation between neural processes and subjective experience, and the (in)sufficiency of computations for achieving consciousness. The theory has given rise to experimental tests to study mechanisms of multisensory interaction and visual perception, and to new models of predictive processing, multisensory perception and motion vision. NREP is one of the three theories being tested in the Accelerating Research on Consciousness (ARC) program of the Templeton World Charity Foundation. The INTREPID project reflects an adversarial collaboration between IIT, NREP and Active Inference.