UserPreferences

SigmaNetwork

1. Sigma Network

This network uses a summation of the activations of the output in a type of population encoding with a type of reinforcement learning, such as Complementary Reinforcement Back Propagation (CRBP).

Here is XOR with the sum of the output layer activation (probabilistically determined) taken to be the answer: 

from pyrobot.brain.conx import *
class SigmaNetwork(Network):
    """
    Uses CRBP to train a population encoded summation on output layers. 
    """
    def __init__(self, name = 'Sigma Network', verbosity = 0):
        Network.__init__(self, name, verbosity)
        self.sigmaCorrect = 0
        self.times = 0
        self.outputLayers = None
        self.sigmaCorrect = 0        
    def initialize(self):
        Network.initialize(self)
        self.sigmaCorrect = 0        
    def preSweep(self):
        self.sigmaCorrect = 0
    def preBackprop(self, **dict):
        # could use a probabilistic version of round:
        # def prob(v): return int(random.random() < v)
        def mutate(v, p):
            for i in range(int(round(len(v) * p))):
                g = int(len(v) * random.random())
                v[g] = not v[g]
            return v
        for name in dict:
            layer = self[name]
            if layer.kind == "Output":
                vector = map(round, self[name].activation)
                score = abs(sum(vector)/float(self[name].size) - dict[name][0])
                if score > self.tolerance:
                    vector = mutate(vector, score)
                else:
                    self.sigmaCorrect += 1
                dict[name] = vector
        return dict
    def doWhile(self, *args):
        if self.outputLayers == None:
            self.outputLayers = 0
            for layer in self.layers:
                if layer.kind == "Output":
                    self.outputLayers += len(self.inputs)
        return self.sigmaCorrect != self.outputLayers

inputs = [[0, 0], [0, 1], [1, 0], [1, 1]]
targets= [[0], [1], [1], [0]]

net = SigmaNetwork()
net.addLayers(2, 5, 7) # make the hidden and output as large as you like
net.setInputs( inputs )
net.setTargets( targets)
net.train()

This works by overloading the Network.preBackprop() method. This method can construct an input or target vector. For each output layer, it sums a rounded version of the activations to see if it is equal to the first target value supplied in the dict (and ignores the rest, if any). If the answer is correct, then the rounded version is set to be the target. If it is not correct, then the vector is mutated and is given to be the target. If we had instead used [(1 -n) for n in vector] that would be Complementary Reinforcement Back Propagation (CRBP). However, this mutate method seems to work better, at least for XOR.

(!) The Sigma Network is now a part of conx, and need not have to be defined. Here is a full program: 

net = SigmaNetwork()
net.addLayers(2, 5, 5)
net.setInputs( inputs )
net.setTargets( targets)
net.tolerance = 0.4
net.reportRate = 100
sweeps = 500
cont = 0
while not net.complete:
    net.train(sweeps, cont=cont)
    cont = 1
net.interactive = 1
net.sweep()

In neuroscience, this type of representation is called [WWW]"population encoding". This gives the network much flexibility in representing the output because it is the sum of the outputs of many units.

What are some good parameters with which to train the network? Let find out: 

fp = open("sumXor.dat", "w")
net.resetEpoch = 500
for epsilon in [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]:
    for momentum in [0.0, 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]:
        net.setEpsilon(epsilon)
        net.setMomentum(momentum)
        net.initialize()
        net.sumCorrect = 0
        net.train()
        print >> fp, epsilon, momentum, net.epoch - 1
        fp.flush()
fp.close()

Change this to run a few trials of each combination. What other variables might you want to examine? Add those in another loop. You can also run these experiments in parallel on a Beowulf cluster using the Message Passing Interface (MPI).