Spatial computing in the brain - on toroidal spacetime topology

Hello again Metapilots.

In recent years, spatial computing has become an increasingly important field of research and development. From neurobiology to autonomous mobility, advances in this area are changing the way we interact with technology and perceive our surroundings. Here's a roundup of some of the latest developments in spatial computing:

Neurobiology: Scientists are studying the way the brain processes spatial information in order to create more intuitive and immersive virtual reality experiences. This involves studying the way neurons respond to different types of stimuli, as well as the way our brains interpret visual cues to create a sense of depth and perspective.

Augmented Reality: Augmented reality (AR) is becoming increasingly popular in a variety of applications, from gaming to education. AR overlays digital content onto the real world, allowing users to interact with virtual objects as if they were part of the physical environment. This technology is being used to create immersive experiences that blur the line between the virtual and real worlds.

Virtual Reality: Virtual reality (VR) technology is also advancing rapidly, thanks to improvements in display technology and graphics processing. VR can be used to create completely immersive environments, allowing users to explore new worlds and interact with virtual objects. The applications of VR are vast, from gaming to training and education.

Autonomous Mobility: Spatial computing is also being used to develop autonomous vehicles that can navigate and interact with their environments. This involves using sensors and machine learning algorithms to interpret spatial data and make decisions based on that information. Autonomous vehicles have the potential to revolutionize transportation, making it safer, more efficient, and more accessible.

While neurobotx operates at the intersection of all these fields, and you hear a lot about VR/AR and ML, we're going to focus on the neurobiology of spatial computing today as this is my favorite topic and the one with the least experts in the online space.

If you haven't read our primer on spatial computing in industry, please do now. I promise you'll become the most interesting person in the room whenever you enter a discussion about the Metaverse.

In that newsletter, I mentioned that whether we like it or not, approximately 280B $ will go into developing the Metaverse, which includes developments in VR, AR, blockchain, AI, biometrics, haptics, and (my personal favorite) brain implants.

So it is definitely worth it to have a closer look at this space (pun intended) and specifically how we can use this market shift to the advantage of deep technologies and scientific developments in the field of spatial computing and spatial neuroscience, as well as neuromorphic computing related to spatial perception, which have been lying on the desks of scientists in the form of Nature manuscripts for about 4 decades now.

During my time as a Phd and then postdoctoral researcher at the Max Planck Institute in Munich, my focus was on training mice in a VR setting (a small VR arena as tiny goggles are generally frowned upon by mice as I found out the hard way). The aim of all these experiments was to 'trick' their brain that the VR is real. In particular, the parts of their brains that would generate specific activity during VR navigation when perceived as a real 3D space and not just a collection of pixels should be in the motor cortex, the visual cortex, and the so-called entorhinal cortex, which includes the so-called 'grid cells'. When grid cells were discovered in 2014 the 3 scientists behind the discovery each received a Nobel Prize in Medicine, and by now probably the one of the most well-deserved ones. Being a postdoc in the lab that got 2 of them is probably the reason why I might be biased, but how cool is it that they found neurons in the brain that collectively create a 3D map of the space around us, allowing us to move autonomously? Here is how this looks:

As you can see, this mouse has an electrode implanted in its brain recording from a single neuron, specifically a grid cell. By placing a white dot every time the neuron fires an action potential or 'spikes' at the location where it spikes, we slowly get what looks like a perfect grid. Since this was repeated in hundreds of experiments, and later on in other species, such as bats, primates, and humans, people started to wonder what might be behind this grid. Firstly, it does seem to save a lot of energy. In our previous newsletter, we go from here into explaining the connection with industry. This time, we'll go deeper into neurobiology, and the next step is going to blow your mind, because as it turns out our brains not only do not perceive images as pixels, but we also do not perceive distances as segments, but rather as overlapping grids. This simulation shows that we need around 6-7 overlapping grids onto one point in order to map it accurately, so that's 6-7 different neurons called grid cells creating overlapping grid maps combined with sensory input on velocity, borders and orientation to map your exact location in space at a centimeter level accuracy, without processing terrabytes of data per second (like volumetric video currently does in machine learning).

The distance between the grid centers always expands by the square root of two, which leads to the question around topology and how this works in 3D, and even if 3D is in fact the correct topology to attach to the activity of these grid cells. So how do they work in 3D then? Luckily Harvard neuroscientist Gily Ginosar has looked at it in bats, who naturally have to map space in 3D, and also interestingly in the dark as they hunt. These insights are extremely important for aerospace and defense.

They trained Egyptian fruit bats to fly in a large room (5.84.62.7m), while they wirelessly recorded single neurons in MEC(medial entorhinal cortex). They found 3D border cells and 3D head-direction cells, as well as many neurons with multiple spherical firing-fields. So, the firing fields of the grid cells of bats were found to be spherical instead of circular.

Now even more interestingly, now that we know the spatial cells firing fields are 3D spheres, can we be sure that 3D is the limit, or are our brains able to process higher dimensions, including warped spacetime? One scientists from Nobel laureate Moser lab at NTNU found a clever way to gently test what the real firing fields are, rather than impose a structure on them:

Here the torus is the result of a PCA analysis, not a randomly imposed shape like the circle or sphere. What is remarkable is the toroidal shape fits the firing pattern shape of these grid cells across very differently shaped environments and during REM and slow wave sleep. One important note here: your neurons are not 'tuned' to a toroidal spacetime, rather the model of their activity shows their spiking adapts to whatever environment they are in at the time - this is how evolution works. But the fact that the toroid keeps its shape across environments shows that our grid cells have the ability to process toroidal spacetime, and do what they need to do when they need to map a given space.

Now, and this is what brings us to what we do and why we do it at neurobotx , what if we could sufficiently understand the spontaneous gridmap formation and endow a neural network with that same ability and make it map a non-classical spacetime environment? In the next articles, we will look at how that could be achieved.