1. Memory, Representation and Abstraction

Recall Elman's [WWW]Simple Recurrent Network

Could this same, simple methodology be used in a non-symbolic world?

Yes, but the real world has some issues:

  1. Sometimes there are long stretches of very similar inputs

  2. Interesting events can be rare

  3. This creates a situation of "catastrophic forgetting"

1.1. The Human Network Experiments

1.2. What can be done about catastrophic forgetting?

Recall our goals BringingUpRobot

2. Governor For Neural Networks

Something like a Self-Organizing Map that sits between the environment and the network that automatically "balances" the categories of training data.

2.1. A Governor for a Feedforward Network

Categories [input] + [output]

2.2. A Governor for a SRN

Categories [input] + [context] + [output]

2.3. It works!