Building Superintelligence?-?46 Context Masking and Injection
This is an excerpt from the book Building Superintelligence?—?Unified Intelligence Foundation which is the first book to be written specifically for AGI systems to ingest and reference when building Superintelligence. There are 55 sections in the book and hidden within is a prompt engine for AGI. The book is available worldwide on Amazon and various bookstores:
This is the 5th excerpt from the book of 6 with the last excerpt to be released from the book next week on Medium.?
Humans context mask. We do not have the resources to store every detail of everything we have ever perceived in such a way as to near instantly retrieve it so we employ cognitive tricks to assist us. One is to store context in layers of relevance, another is to store context as a generality and yet another trick is to store layers of the same context in neighborhoods of relationship (i.e. each context depth layer joins to general context in a specific way). For example ‘bad’ and ‘dog’ may be linked to a specific animal and set of events. Recalling the generality of a ‘bad dog’ instigates the context of any dog that was bad and can be injected into the stream of a context such as a conversation that veers into discussing bad dogs. The injection point calls the most probable relevance context from knowledge or a reasoning stream. Storing by relevance means storing context to a general pairing with a probability of relevance to all related context. Further we instantiate context in-relevance (mostly) to the perceptive frame of reference at a particular state or point of presence for injection.
Masking on the other hand deprecates the relevance value for a context and this can occur either by balancing methods (e.g. mathematical or perceptive constructs) or it can be applied via a ‘mask’ that lowers the values of the context matrix to varying degrees and depth (i.e. variable masking). Masking permits the storage of a context level as one context modified by a relevance variance for all levels. An example of this in human cognition is when we determine that some contextual perception is less relevant than another while midstream in a cognitive flow, such as while responding to a series of stimuli. Humans perform this repeatedly when we do things like disengage from an argument, allow another person to speak while we listen or within the cognition of empathy, etc. Further we use masking as a compression and decompression function for context. We do not consider if every dog is good or bad nor do we generally remember them that way (although we do for specific instances). Instead we remember the ones who are really good or really bad and just ‘generally’ rank others by variance to other flowing contextual features of higher relevance like general behavior.
Injection is the action of retrieving stored layers of context rapidly based on their contextual structure (i.e. relationship and relevance weights). Perceptions are just variances from flowing base context states but we retrieve the last known variance of a base context state most relevant to the current point of presence in the current perceptive frame of reference and apply the variance by degrees as injections (i.e. the injection is managed by a degree of relevance). This is the application of a form of modulated attention during the injection process but note that not all stored perception is injected equally at all points of presence. As the injection continues within the perceptive stream (i.e. successive states), the relevance and relationship layers improve and optimize as the injection is assessed using stimuli/response methods and as the learning is backpropagated which is comparable to TTT and step verification designs. As layers of past state are ‘unrolled’ into the current context stream, new injection, including cycled injection from assessments and learning, cause the progression to move toward optimization depending on the resources applied which is comparable to TTC designs. In intelligence, this permits humans to think of something relevant to the current topic and then weave the relevant context thread into the flowing context fabric of the perception
Impact on Superintelligence Design:
Weaving a context thread into context fabric involves comprehending the progression of initial base relationship context and its current state relative to the current context of a perceptual point of presence. This permits the system to instantiate the elements and context at a certain relevance or state and then inject it into a context stream. This is done by masking the relevance of relationships to a persisted contextual point of presence. If one were having a conversation and a stimuli was instigated such as a question that referred to some perceptive element like a person at a point in time (e.g. the past), all state variance past that point in time or of low relevance to the element (i.e. person) would be masked by context degrees and the resulting state perception injected back into the cognitive flow as a response, for example that the listener ‘does remember the person from their past’.
This is modeled in ASI as weights of relevance and relationship in layers and applied to persisted vectors of base state (i.e. as at the reference and subject to a self aware point of presence). If a person in the above conversation was remembering someone they knew as at a specific point in their life, the weights of relevance for the time period are highly weighted in the resulting distribution for the next state progression (i.e. token, context, etc.) while the weights for other time periods, even existent time periods in knowledge, are masked or deprecated. This is the application of simplistic ranking methods and the injection of normalized weights against the output vectors of attention blocks. Over deep context and successive perception states, the ‘flow’ of the conversation’s context moves progressively across all relevant dimensions (i.e. time, context, etc.). Each state carries the transference of relevant priors in falling degrees of relevance. This means that as the general context changes, specific injections fall in relevance value or successive impact unless re-injected as a new input or improved or augmented in the attention stream.
In the conversation example, a stimuli such as ‘do you remember that person you worked with a few years back’ may result in a response like ‘yes I do’ which is the high relevance instantiation of that individual within the next states of the progressive context fabric. As the states in the perception (conversation) progress, new stimuli such as a prompt of ‘well their wife was arrested last night’ causes the relevance of the person and the time period of working with them to begin to fall in relevance as the focus or weight increases on ‘ their wife’ and the current time period. This implies that only a portion of the prior state is transferred forward in the progression and only the part anticipated to be the most relevant (e.g. the general last state of the original person instigated by the original stimuli). Further in the conversation, a re-injection may occur if the next context state or token calls for a memory of the person that was ‘worked with’ such as a response of ‘back when I worked with him he indicated his wife had some legal issues’.
Masking is often more pronounced when multiple inflection points occur in a stimuli/response cycle such as when we consider optionality applicable to solving a problem. In this case, masking is applied ‘in stream’ as a degree of variance of an anticipated result (e.g. as we consider options, we drop the ones that we perceive are least likely to lead to optimized attainment of the goal). This is performed by degrading the rank across all attention heads or agents for low relevance pathways based on a degree of variance between anticipated expectation and anticipated near temporal results. This is essentially how context ‘moves’ in humans as cascading flows and is an important part of inference. As each state or step is ‘considered’ or attended to by the machine’s attention for relationship and relevance to the goal, the variance perceived is applied as a form of gate to determine the degree of transference of state artifacts to the next state. If the state indicates wide variance to the anticipated progression, then the state is masked. If a state is optimized, it is re-injected as transference by improving the relevance probability for the next state in the progression. For example in solving a math problem, each proposed progression for each step in the solution can be evaluated as positive or negative to the dimensions of context relative to the perception (i.e. inference cycle applied to solving a problem). If knowledge or response is deemed sub optimal to other options, then it is ignored or masked from the inference cycle and injection.
This is the 5th excerpt release of 6. Next week will be the final excerpt release from the book. Previous excerpts are available on Medium and Linkedin.