Hacking Reality: The Convergence of Neuroscience and "Reality" Technologies

Reality, or the world surrounding us that we perceive, is one of the most beautiful engineering designs we know of. The brain is an incredible computer; the physical deliberations and calculations craft our world within the confines of our skull. Our grey matter is a sensory-input interpretation machine, where we take in information visually, aurally, or physically, with our brains, then run simulations of what likely comes next, all of which allows us to navigate our world. It’s the best system we’ve ever known. Now, we teeter on the edge of a rising age where several technologies, including brain-computer interfaces (BCIs) that utilise the brain as a user interface, Virtual Reality (VR), Augmented Reality (AR) and the broader Extended Reality (XR ), are set to reshape the very nature of the physical world with attractive metaverse propositions.

The Brain: Master Architect of Our Experiential Realm

The world out there isn’t sketched on our brains directly, as we always assumed, but is instead constructed by the brain’s own genius and sent to ‘consciousness’. Leading neuroscientists have shown that the brain is continually doing such predictive processing, continually using predictions created by sensory inputs and based on what the brain previously learned to generate your experience of the world. Our ‘mental reality’, in other words, is indeed a hallucination. But it’s not a random one: it’s the brain’s highest-wager hypotheses forming reality.

It conquers all the areas beyond mere sensory information and motor actions; above all, the realm of social reality, which can also be characterised as solely a consensual act, purely a convention, so that concepts such as money, our borders, our qualifications and jobs, etc., are simply collective phantoms, simply language games, that we cannot do without because we have consensually agreed upon and made them a reality. The US neuroscientist Lisa Feldman Barrett has consistently shown that our brains construct functions and meanings for things and events, which leaves us to conclude that all these fine-grained aspects of our reality are all human constructions too.

Technological Frontiers: BCIs, VR, AR, and XR

Today, sitting at the crossroads of neuroscience and technology, the future is wide open, and many of us believe we’re on the verge of a radical change in the way we perceive, interact with, and shape our reality. BCI, VR, AR, and XR dominate the bleeding edge of this imminent change.

The level of insight that BCIs provide is unlike anything we’ve seen before. By deciphering our ‘wetware’s morphological code’, the electric signalling in our brain, BCIs can allow us to operate a computer using only our thoughts. This new level of accessibility, interaction, and personalisation stands poised to truly transform our lives.

For example, the "Galia" headset, made by OpenBCI and Varjo uses headset sensors to measure EEG, as well as EMG and other physiological signals that can be used to drive virtual objects, while also providing real-time biofeedback to the user. While still an advanced niche product, Galia and other devices like it show the direction in which the BCI can progress. In the near future, we might watch a close friend take a 10,000-metre race in the World Championships via gaming apparatus, while we experience the race as if we were there, from the perspective of our friend’s eyes.

Meanwhile, other equally innovative research projects exploring holistic, network-based approaches to BCI signal processing are springing up across the world. These methods aim to combine different types of signals and patient data to produce more effortless and immersive BCI experiences. As some of these techniques are researched in the lab, feedforward efforts are also making inroads into the development of future BCIs.

Artificial Reality Domains (aka Metaverse)

Users can seamlessly switch between experiences that are stunningly realistic and wildly fantastical in VR environments because everything around them is digital . The simulations that VR can create hold important implications for learning, training, therapy, and entertainment, as well as philosophical inquiries into the value of engagement with digital creations . A number of philosophers, including David Chalmers, who studies the mind and consciousness, have written about how VR prompts the question of whether experiences acquired in a VR setting might have meaning on par with reality.

AR, on the other hand, superimposes digital data directly onto the real world, enhancing how we relate to what’s around us. XR brings together those two worlds. Together, VR and AR define a continuum for immersive experience, and they’re poised to revolutionise entire industries by bringing new capabilities that enable us to work remotely and collaboratively, see and visualise things in new ways, and access data in real time. While performing surgery, a doctor could use AR to overlay additional important information to enhance the precision of the operation, and increase the likelihood of a successful outcome.

The Future of Reality Management: Widening applications from Teaching and Learning to social and Identity Exploration

With the development of BCIs, the gradual normalisation and rise in popularity of VR, AR and now XR, we are moving towards a future where reality could be ad hoc designed, enabling us to produce multiple and perfectly adapted representations of the real . One possible sentence could be: When coupled to a BCI system, VR, AR, and now XR (XR = Extended Reality) herald a new era of possibilities as far as designing and modifying reality is concerned, enabling us to produce ever-more adapted representations of the world according to our needs and wishes.

These interactions with VR, like measuring brain activity and the body's internal response, will help BCI and immersive technologies learn specifics about each person so that they can be given individualised rehabilitation treatments. Imagine a stroke or traumatic brain injury patient who is learning to walk again. A therapist will craft a specific right-brain programme for the patient and then monitor the individual with an EEG machine to see if the patient is in sync with the programme. When the brain activity registers properly, therapists can take the patient to the next stage of rehabilitation. As rehab becomes more customised, it will accelerate recovery. Patients will be motivated to do more, and it will have more of an impact. Overall, VR and AR can help create immersive environments where patients will be motivated to perform their therapeutic exercises. It will transform rehab out of the clinical realm and into an expansive virtual world.

Immersive technologies offer a raft of educational and training opportunities where people can learn new skills and acquire knowledge in ways never seen before . For example, in VR, it would be possible to recreate specific scenarios in which a person would be able to practice their skill many times over in an immersive environment that allows them to interact with their virtual surroundings and where there would be no risk to either those undergoing the training or to real-world assets. Medicine offers another useful example here . Medical students could practice their surgery skillsets in VR environments where various kinds of operations could be undertaken, from full-blown surgeries on virtual patients to more intimate scenarios such as subdermal procedures or delicate suture work on more specialist areas. With AR, in-the-moment suggestions and corrections visualised can provide further learning opportunities for people in a wide range of fields, from engineering to the fine arts.

There are inherent benefits from the fact that virtual environments allow social experiences unaffected by physical constraints: people can shift between their identities as both users of the environment, and users can also experiment with different versions of themselves by taking on different avatars as they play with different social roles. The process can be liberating and exhilarating for those who, for whatever reason, feel frustrated and hampered by certain aspects of their embodiment, physical and otherwise.

As with any innovation, transformative technology raises real questions of ethics, and BCIs and immersive technologies are no exception. The questions that arise vary from issues of privacy and consent to those of potential misuse and should be carefully thought through . The data captured by BCIs, for example, could be very sensitive, and personal and proper protections against unauthorised access and use will need to be established.

Also, as we create richer and more persuasive virtual experiences, how do we manage side-effects in associated behaviours, given that we are still dealing with hangovers from the social media era ? Of course, all these technologies can, and most likely will, bring enormous benefit. But there’s also potential for new mental health (and other) challenges as users find it increasingly hard to tell what is ‘real’. Ancestry remains a powerful motivator for us, a beneficial tool that can help mitigate the drawbacks of virtual immersion.

Redefining the Boundaries of Reality

The brain’s reality-constructing capacity and the dynamic nature of the resulting felt reality show us the true power of human cognition or even imagination, as well as its possible challenges. This ever-transformative reality, partially reconstructed by the brain and modulated by new technologies such as BCIs, VR, AR, and XR, carries within it the promise of becoming an ecstatic human condition: it could lead to higher levels of human functionality, novel ways of communicating, or even solve some of humanity’s old dilemmas—so long as we navigate this gorge carefully and responsibly, so that these new technologies become tools that help to augment humans rather than replace them.

The real power of these virtual worlds does not lie in their ability to replicate reality but in bringing to life new worlds that show us what it means to be human in ways we have never known before . We get a little bit closer to a reality where our minds define the boundaries rather than those that physics places on us with each passing day and each new iteration of our technological brain babies.

Each of these new technologies is, after all, a product of our own grey matter !