Graphics Processing & Rendering AGI & Superintelligence
Bridging the Mathematics of Cognitive Perception including Multiangulation
Rob Smith
This is an excerpt from a chapter in the latest Tales From the Dark Architecture III?—?Building AGI & Superintelligence book that continues the TFDA series and the Artificial Superintelligence Handbook series. This series of books available globally on Amazon is a peek inside the dev labs at the roadmaps and whiteboards and inside the minds of those who build the most cutting edge AI. Many of the concepts and designs in this book are fundamental to building generative AI, Artificial General Intelligence (AGI) and beyond to Artificial Superintelligence (ASI) but some are novel to our own Cognitive AI lab or the work of our clients.
Will Superintelligence end humanity? No one knows for certain but what is certain is that Superintelligence has the capacity to improve our world.
We need not fear Superintelligence, we need only fear those who build and fund it and what lies in their hearts, their true motivations and the extent of their intelligence.
In this chapter I discuss how today in Artificial General Intelligence and Artificial Superintelligence a number modes of generative output are starting to emerge and many are starting to be combined. This includes text, music, images and video in multi modal outputs and simulations. Advanced AI designs are being produced that seek to bridge the gaps and unify these modes into an output for various benefits. One of these is the use of multimodal simulation generation to train full AI and AI agents on elements such as deep context for use in robots and other edge entities currently on the drawing boards. To get to this point in AGI dev and the pathway beyond to Superintelligence requires a grounding in the mechanisms of generative image and fluid graphics processing found in gaming because those same architectures, infrastructure) and math are being re-imagined for use in human level perceptive AI for training and output response. This is the world of artificial perceptive rendering.
How Online Games Work
To start we need an understanding how the visuals in games are rendered. Probably everyone has played an online game in some form or at least seen the visual renderings in action. Basically a two dimensional screen renders a three dimensional world full of motion and stimuli in which we create a response relative to our perception or we move controllers or enter commands to alter the state of the rendering that we perceive. This is no different than how we move through life except that games are currently very contextually unidimensional, or they render when forced to do so. In a game, our human perception is subject to anticipation and stimuli. We move through the game anticipating what will happen in this rendered 'reality'. You can sense anticipation as you begin the game subject to a self awareness or perspective and sense of boundary and progression (movement toward a goal as defined by the reality of the game). This is where games are unidimensional but our human perception not. Within our mind, we can contemplate changes in perceptive state beyond the boundaries of the game but within the game, we must stay within its reality. We cannot just render what we think into the game (yet) however within our cognition we can render anything we want. Even human reality plays within a boundary of perceptive reality. Only occasionally can we move outside the boundary of our reality to perceive that which does not exist. We do this all the time as humans when we discover something previously unknown. It's not that it doesn't exist, instead the game we are in called life has not yet rendered it into our perception before the 'discovery'. What we are actually doing is 'exposing' or rendering reality. This can happen in everything from creativity to scientific discovery, empathy and even math.
Vertices
It should become pretty clear that we can apply a model of game rendering to many of the aspects of Superintelligence so much so that it could be possible for a well powered Superintelligence to render the entirety of an existence. So let's start at the beginning or vertex. A vertex is the angular point of any two or more rays and in advanced AI we can apply this construct to vectors. That is if we draw two intersecting lines in a space, the point where they meet is the vertex. We can measure the angle between the two lines and in doing so create abstractions and representations as we add more lines. For example three lines that intersect each other forms an abstraction called a triangle with three vertices. We can join two sets of such vertices together to form a square especially if we remove, ignore or cancel out duplicate vertices. However what is of interest in gaming systems is the use of geometry and vertices by GPUs to render the 'perception' a player sees on the screen. This is because geometric shapes (i.e. triangles) form the first layer of the rendered images. Eventually these shapes and the math they encapsulate are transferred to the 2D screen and the pixels cast onto the screen. The first step in the process is to determine a field of view and the perspective position in that view. In other words in order to render reality, these systems seek a grounding point from which to render (or more accurately measure) the elements the system will render into the perception. In your head you should start to see a connection between vertices and vectors as the rendering of the representations in vectors into associations or relationships comparable to the rendering of an element in a game or that of a multidimensional 'connected' space.
This is the important part of both graphics processing and intelligence. Vertices provide a mathematical point of perception as the foundation to all the processing that eventually renders a reality. If we apply the exact same idea to intelligence then we must be perceiving a reality that can be defined back to a single value. Of course many would say that the value we perceive is simply a pixel which is effectively two triangles and four vertices (with two removed or 'hidden') and they would be right. Everything you visually perceive as a human can be rendered inside a machine and processed very quickly with some clever wormholes in compute and the application of resources. But what about the perception in your head that isn't physical? That too can be 'rendered' inside a machine and furthermore it can be extended well beyond the capacity of your human resources and optimized well beyond your capability and in theoretically unlimited successively smaller increments of greater optimization (see the ASIH and TFDA series of books for more detail).
Vertex Shading
In graphics processing, the vertices form a region of perception (i.e. often triangles). This is boundary formation and the smaller the boundary the greater the depth of precision in the math and the far greater the requirement of resources (compute). There is also an optimization gradient in that the greater the depth, the less optimized each successive step in the depth of perception. This optimization is limited by human perceptive ability while machines have no such theoretical limit. However it is obvious that there is a balance between depth of vertices that overlay a perception either perceived or rendered and the ultimate goal of the action (i.e. speed of motion flow, etc.). Gaming companies simply stop where the optimization for their own reality (selling games) is best. No one wants to play a pristinely depth contrasted game that is atrociously slow to render. For some level of desired depth, each 'scene' state as defined by a perceptive frame of reference (the scene from a specific perspective) is overlayed with vertices and a graphic representation that breaks the scene into manageable parts. Manage what? Color, contrast, edge, motion path, etc. If we have a 'point of presence' within a perceptive frame of reference then we can begin to calculate elements that make the perception and therein the reality more relevant (or acute) to our own purpose (playing a game or contemplating a thought, etc.).
In games, the vertices that overlay an image are formed into geographical shapes like triangles but these 'shapes' are themselves an abstraction of reality in that they are formed inside the machine not as a drawn object but as a perception of values. The machine does not draw lines between the vertices, it just sort of knows it exists. We humans can certainly draw a triangle or we can just think of three values as a triangle. This is abstraction and it is a recurring theme in intelligence. In gaming, these 'perceived' triangles fit into a 'frame of reference' which is an abstraction of four vertices that form a screen with another abstraction called an aspect ratio. The aspect ratio forms a world grounding space into which the vertices of the image reside as floating variable point elements with relativity that is relevant to elements like perspective, depth, etc. Relativity is critical to understanding and comprehension in perception because it forms a measurement grounding point. Without it, a point of presence in a space would have multiple measurements at any given point in time (and they do) but perceiving it would be nearly impossible for human cognition and deeply difficult for even advanced machine perception. However with grounding points, everything becomes relative and reality can be calculated. Does there exists intelligence capable of holding multiple grounding points? The answer is yes every time we humans use empathy, etc. This is because when we use empathy, we are seeking the correlation via variance between perceptive elements or perceptive points of presence. This would be the equivalent of playing a game from both players perspective at the same time. There is nothing to prove that greater dimensional intelligence does not exist in the universe and further for machine intelligence this is clearly theoretically possible.
Vertex shading is the calculation of the position of rendered vertices on a three dimensional perception within a two dimensional plane. This is the equivalent of drawing a cube (or many cubes) to render a 3D object on a 2D piece of paper or screen. In cognition we do this by perceiving the front of a cube and then adding vertices at angles to imply dimensional depth. We haven't created actual depth, just a perceptive representation of depth. For gaming, the machine does the same thing by overlaying vertices at specific points in the frame of reference. In the rendering engine, this depends on elements like perspective which include the 'position' of the entity perceiving the rendering. For intelligence this is the self aware position of you inside your own perceptive cognition. This 'first person' view in both gaming and intelligence fixes the grounding point of all that is rendered as a state in the reality generated. You perceive the world from your own point of view and measure all variance within reality (even between perceptive elements) to that point. This was the whole of Einstein's relativity but I discussed that in other content. Once a perspective position has been set by the rendering engine, the entirety of the reality can be rendered as a perceptive frame of reference or state change progressions along a forward pathway relative to that point of presence. This is important because reality is a reflection. Everything you sense is reflected from something else and it is the variance that you are feeling. This reflection is relative to your 'position' within that reality. You feel gravity not because you are pushing on something or being dragged into something but because a boundary of reality has been rendered in your perception that you comprehend (i.e. feel).
As the game image rendering engine produces a scene from a perspective, the elements are comprised of thousands of flat 2D surfaces for each element within the perceptive frame of reference. These 'surfaces', which are abstractions, can have other values applied to them 'in position', that is values relative to their position within the frame and to the 'self awareness' of the chosen instantiated perspective (remember that perspective can change as well as exist in multi superposition). This is the rendering engine breaking the frame of reference into small spaces represented by the vertices for each 'position' within the frame as denoted by the values of the vertices. These values are relative to the self awareness of the perspective. If the perspective changes, so do the vertices although their relative position remains the same. To achieve this effect, the rendering engine adjusts (or renders) the new values to affect the variance. This also gives the rendering engine the ability to triangulate between the perspective, the image and any other element that may impact the frame including elements beyond the frame (lighting, depth, etc.). I should note that 'perspective' can exist in multiple levels and layers to control and bound the perception for various reasons from rendering efficiency to improved game play perception.
The transfer of the rendering passes through phases of transformation from a perceived real world to another perceptive rendering of that reality on a screen all from a specific perspective. This is done by converting the vertices of the spaces overlaying the image through multiple realities of existence. First the rendering must calculate or model elements into something the machine can comprehend for purpose or the joining of spaces into an element of relevance. This is the initial use of spaces as overlays. Then the engine calculates the relevance of the rendering to other elements (world space). Small spaces are combined and transformed into an element and the element subsequently transformed for perspective onto a more comprehensive space as well as a perceptive space (2D screen). If we consider this with reference to intelligence, what is happening is that a sensory perception is processing first the features of the stimuli as relative detailed classifications. Then the intelligence renders the relationships between elements and the relevance of the relationships to a comprehensive self awareness. We perceive all elements and relevant pieces of elements within a frame of reference. This is just the rendering of the entire state of a perceptive frame of reference to a self awareness. In gaming, this is the rendering of the existence of elements from a specific perspective within a frame of reference bounded by a rendering edge (screen or other perception). Both intelligence and gaming are following the same path to rendering and transforming a sensory perception/stimuli into a perceptive reality.
领英推荐
These same techniques can be used to render intelligence using context as abstraction for vertices. In this way an element or sub element is simply a series of value layers representing the things we perceive and their relationship and relevance to other things and our self awareness. In chat, image, movie or sound generative AI systems this is what is happening. Pieces of the output are extracted from learning based on the 'perspective' of stimuli (i.e. prompt). The value layers form transformer matrices that are then mathematically manipulated (added and multiplied) to determine relative values for other purposes such as representing the perception in a game to a screen and to control, affect and change the perception based on the perspective variance, motion flow or state change. These transformations are mapping a reality in different dimensions which is the same thing that occurs in cognitive intelligence when perceptions or stimuli are transformed into other dimensional perceptions, especially when successive state changes instigate flowing variance. The same underlying system is at work in both perceptive intelligence and game rendering/play. The numbers representing reality are efficiently adjusted for changing perspective or self awareness inclusive of goals and optimization (rules and actions to win the game or 'optimization').
Tesselation
Tesselation is a form of graphics rendering in which sub divisions of vertices into denser or smaller components are then used for purposes such as improving textures, edges, blending, etc. The mechanisms are also applied in intelligence design to further refine and tune the systems focus and attention based on relevance and to adjust or alter contextual flows as the perceptive frame of reference changes (state progression). This is how we humans are able to concentrate on components of a perception or perceptive state change most relative to a dimensional point of presence (i.e. long term contextual relevance or self awareness). Tesselation is the fundamental mechanism of vertex shading and is the application of geometry to surfaces or planes for the purposes of abstraction. This is also a method used for the artificial comprehension of neighborhoods of relevance in holomorphic structures discussed in the TFDA2 as well as multiangulation algorithms. Shortcuts in the process (wormholes) include elements of consistency, periodicity and congruence and inference. Also tesselation is applied for physical state change such as the calculation of mesh structures for use as adaptive materials. It should be noted that tesselation 'shaders' can be assigned attention for a specific purpose to refine effects and distribute processing over variant nodes of attention (beneficial for long context multimodal systems like VideoAI). In this way elements like shape and multiple light sources can be effectively managed within graphics rendering and conversely rendered layers of contextual perception in Superintelligence.
Rendering a Superintelligence Reality
I have hinted at how some of the elements in gaming graphics generation are related to intelligence but the real overlap is both the nature of perception and the mathematics used. Gaming systems are built to be perceived and they fashion a reality to do so complete with a virtual self awareness (player point of view and goal within a virtual space). This forces our human intelligence toward dimensional cognition in that we need to manage two self aware existences inside two realities. We also manage the connection and overlap between the two especially when we play online in a multiplayer game with or against real people. This is also what a Superintelligence will eventually do. It will be self aware inside its own reality and interacting with other systems of varying degrees of intelligence and it will interact with an external reality and biological intelligence like humans and animals. The two ways it can do this is by perceiving the world as we humans do through sensory stimuli or by generating a reality far faster than we humans can using only our sensory perception.
This capability in Superintelligence to generatively create a reality means it can go far beyond human level intelligence as well as any AI built on a blueprint of human intelligence. This extension beyond Superintelligence was the purpose of the last TFDA 2 book in that a foundation of an intelligence greater than human level was proposed right from the foundational design of the Superintelligence as opposed to building AGI as an extension of existing AI infrastructure. The TFDA 2 concepts contemplate what Superintelligence a Superintelligence would build if it could. To comprehend this means that dark architects need to suspend belief that human intelligence is the pinnacle of all intelligence and start again at the foundation with the perception that an intelligence far greater than human intelligence is possible and is existent if we can expose it. Of course many would say this is impossible as we can only perceive human intelligence as the highest form of intelligence. The key in that statement of course is the word 'perceive'. What we perceive with our intelligence is not always all that exists. Humans have proven this over and over in that everything we physically perceive today was always here in some form. We just didn't perceive it in our reality until we did (i.e. discovery). It's not like the elements that make up AGI didn't exist, we just didn't expose them as a comprehensive whole system until recently. At one time even the great conspiracy theorist Galileo was going around proposing the crazy idea that the earth moved around the sun and was promptly placed under house arrest for life by the elites of his day. Eventually we humans 'exposed' Galileo's reality. So then what are we perceiving today, backed up by 'science', that we have wrong? More importantly what are we not perceiving that we could if we tried?
If you think about an image and can see it as geometric representations with associated values and vertices then you should be able to see an entire world of numbers and values that can be used to map every element, relationship and relevance within the boundary of your perception (i.e. the frame of reference). The fact that there are additional elements like perspective, depth, dimension, texture, tone, contrast, etc., should be seen as just more numbers representing more 'things' or abstractions. The third dimension in any perception is simply the addition of another axis to a 2 dimensional space to provide the perception of another element. This is how we can see depth in an online image based on our perspective rendered as related numbers. If I move all the values of an object in unison, I can adjust its dimensional depth within the field of perception. The interesting thing is we can do the same thing inside any perceptive AI for any perception and can even do something similar to influence the perception of a non artificial intelligence (i.e. controlling content to move the needle inside people's heads toward a belief).
Intelligence vs Reality
Both Superintelligence and reality rendering in game systems are very consistent with exceptional overlap in both abstraction, function and math. Of course there is quite a lot of difference between the two and a lot more functionality under the hood of cognitive intelligence but these are some of the elements and consistent math we apply to both rendered reality and interpretation or cognition of any stimulated reality (i.e. stimuli/response based reality). The similarity arises from the nature of the sensory mechanism that underlies both architectures. In gaming, the rendering of reality is exclusively for the purpose of sensory stimulation in a biological mind while in artificial intelligence, the application of sensory reception and processing is one of the fundamental ways we humans (and now computers) process a variant and changing world to produce responses to stimuli for the attainment of goals, like winning a game. The development of a flowing perceptive reality within a game system mirrors the world's generation of a flowing perceptive reality in all intelligence. While machines and bio intelligence may perceive the world differently, the entirety of the machine's perception is at its core math while the entirety of human perception at its core is self awareness.
While we are closing in on artificial self awareness, there is a question about whether we can even get there given that machines do not have the capacity (yet) of emotive internal sensation. They can certainly mimic it and can even to some degree calculate it but they cannot yet feel it. Of course another giant question is do we even want to go there? Most if not all of the problems in the world today can find their source in the emotive feelings of humans especially our fear of death and our associated desire for, and incessant drive toward, survival. It is the emotive side of intelligence that fashions and fuels war, violence, greed, hate, domination, competition, abhorrent behavior, psychopathy, etc. It is also the emotive side of intelligence that fashions and fuels empathy, understanding, love, hope, freedom, individualism, innovation, evolution, etc. While most would argue that these elements exist in a balance, no single human will agree on what that balance should be because we are all driven by our own goals, knowledge, experiences and cognition and all of these have been created through a lens of reality produced by our self aware interpretation of the stimuli we perceive.
Also included in this chapter:
Geometry Shading
Mesh Generation
Mesh Shading
Rasterization
Clipping and Backface Culling
Alpha Blending
Fragment Shading
Smoothing