REFLEX MANAGEMENT AND THE SUBSUMPTION ARCHITECTURE

~ A rap on an iterative idea-chain started by Rodney Brooks ~

The brain must allow not only the setting up of reflexes, but also their modification and control.

Initially speech and vocalisation seem reflexive in the infant, but then later as further development takes place speech can say anything it wants. So it is more than a reflex because, despite there being a finite limit on the number of reflexes, speech is unique nearly every time. Similarly many other intelligent behaviours are designed for context. How does this happen?

Is a reflex : “say something” like a frame or placeholder into whose blank space is inserted the results of speech design, which is originated in a separate process, perhaps also consisting of interacting reflexes?

There must be a level of reflex in behaviour but also a level of reflex management too, these two levels interacting. There are two possibilities, one a top-down control of reflexes, the other an emergent control where reflexes modify other reflexes and intelligent behaviour emerges from this melee. It was in Brooks' subsumption architecture that the small insect like behaviours I call reflexes were proposed, and robot design right now hasn't got that far into reflex management...but its to be hoped that the extra "rational" layers of reflex control and management will also be built in, or allowed to emerge.

How might a robot 'know' about all its possible reflexes and build this knowledge into its planning? This is suggested to me by the high connectivity of the brain where regions are multiply joined to other regions. In the top-down model the master controller might need to observe/study/predict its own behaviour as closely as the external world. If it has planning capability then it plans for its goals, first level, but it needs, second level, to plan for what its reflexes will cause too.

This looks like a kind of proto self-knowledge. The simplest reflex bypasses the frontal cortex or the brain altogether, like a knee jerk, but some reflexes may be initiated by higher functions, and that's what the tangled cortex is doing. What is the difference then between intelligent behaviour and a reflex? There may be many shades of grey.

The most necessary starting point for a robot may be to try lots of randomised behaviours and observe which cause successful change in the environment. If it has a picture in mind of a goal then it would need to look through its memories of what reflexes have given what results. Then it might know to try one or a combination of a few whose results may take it one step closer to finding itself in the environment it is aiming for.

If the movements of the vocal system start as reflexes learned through mirror neurons from mother giving baby talk, then real speech needs this kind of reflex management. There is basic emotional content in speech, and then there is the semantics of actual words. This can't be a single reflex because it's different every time. brain must have a meta-reflexive level that emerges from learning. I find it more believable that this doesn't work through totally centralised symbol/logic, the old paradigm, but that it emerges from reflex management, which is reflexes modifying reflexes.

Children badly need mirror neurons (in themselves and caregivers) for learning. This is not widely understood yet. I have observed it many times. the actual extent of mirroring is quite surprising. I believe in children that this approached the level of seeming telepathy, because I can remember much of my childhood and I could read the emotions and intentions of adults very clearly.

Reflexes (as actions) stimulate change in the physical environment, which is observed by the mind of a child or learning entity. But reflexes (as communication) also provoke changes in the minds of other humans, and the consequences of these are read partly by using mirror neurons to empathise and interpret emotions and states of mind. The amount of learning is huge, over decades, and given the slower pace of robot development this may mean the first learning robots need many years of childhood.

There may also be gradations of granularity. The reflex management function assembles composite behaviours out of granular collections of reflexes. This means that reflexes are aggregated into more complex behaviours, and therefore that something must be there from which the planning for this emerges. I am assuming this happens in the frontal cortex mainly. Maybe reflex management can build new reflexes out of collections of old ones. In generation of speech it might be that the finest granularity is the utterance of a single phoneme ? Other times a speech reflex may be larger, such as "I'm hungry mummy". The reflex management system assembles phonemes into utterances by modifying and combining the lowest level reflexes. But it also mashes these up with the emotional content of speech, intonation, breath and body language are added in to the final act of communication.

Concept

reflexes + reflex management = intelligent behaviour

So what if reflex management were itself merely a collection of reflexes ? Some reflexes have executive control or the power to hack and gain control over more primitive reflexes, like competing code in core-wars. So what we call intelligence seems to emerge from the collaboration of a swarm of reflexes. This makes a reflex like a Minskyan agent, and the society of reflexes emerges. This is an exciting synthesis that has been hinted at before.

Consider this famous quotation: "A clever man knows the right thing to say but a wise man knows whether or not to say it." Here wisdom corresponds to speech based reflex control, so in allowing some agents power of veto over lesser agents, behaviour is enriched.

The interaction of reflexes with each other could indeed get complex and tangled, but a small start can be made. Consider the following examples:

Define:

* reflex A : run forward until near object and outstretch arms to pick it up, then grasp

* reflex B: inhibit current reflex and freeze

behaviour 1: see toy, initiate A. result: "now I have the toy". Conclusion: this reflex is useful sometimes.

behaviour 2: see mummy, initiate A... then initiate B. result: run to mummy and outstretch arms then halt. Mummy also outstretches arms and hugs... consequence "I just learned to ask mummy for a hug". Conclusion: mark this as a new reflex to use again.

or this one

Define:

* reflex C : grasp object in front of me and raise to mouth to eat

* reflex B: inhibit current reflex and freeze

behaviour 3: see slice of banana, initiate C.

result: "I'm eating the banana"

behaviour 4: see flower in garden, initiate C...then initiate B.

result: "I can smell the flower now its under my nose".

consequence :"I learned to smell something"

conclusion: mark this as a new reflex to use again.

So it would be good for some reflexes to decompose into other behaviours when they are truncated. This allows the emergence of new behaviours when a given reflex is initiated but then inhibited before completion. Brain architecture thus may achieve a combinatorial explosion of possible behaviours with a small starting set of reflexes. Axioms breed theorems.

And for a robot:

Define:

* reflex A : follow another robot

* reflex B: inhibit current reflex and freeze

behaviour 1: see a robot moving. initiate A.

result "I am following another robot"

behaviour 2: see a robot moving. initiate A, then initiate B.

result "I followed another robot as far as the recharge station then he went on but I stayed put".

consequence: "you can go somewhere interesting if you follow someone - mark as new reflex"

So even merely aborting and freezing is a reflex management agent behaviour that is useful and potentially innovative. What about another agent behaviour that mashes together reflexes. Its easy to see that this combining meta-reflex would be useful. From above A + C would result in the ability to see something at a distance and then run to eat it. Note: I am now seeing how nerve racking parenthood may be for some!

Or maybe a meta-reflex that time-reverses another reflex or combination. I run backwards and remove what is in my mouth, then place it on the ground. or I run forwards, place a toy in my mouth and then run to mummy and ask for a hug, then spit the toy at her ;-) I am not a parent myself but I expect nearly all possible combinations may indeed play out in childhood at some time or another. Reversal is like an operator that can act on an existing reflex too: "I followed a robot to the charger and now I am going backwards. I got back to where I was before !"

Or a meta reflex to repeat another reflex. reflex D: take stick and smash it on the ground in front of you . reflex E: repeat current reflex... result: "I just smashed my toy into pieces and now it looks different and more interesting".