Amdahl's Law

Amdahl's Law states that when moving a serial computation to a parallel one, you can never get linear speedup. That is, if you add N processors, it won't run N times faster. This is shown in the following picture.

This occurs because any communication needed must be added to the time of computation, and subtracted from possible speedup. Therefore, adding more computers won't speed up the processing at all after a point.

However, there are some emergent computations that appear to be super linear: they speed up a computation more than N times faster. How can that be? Because there is additional power added from the interaction. Therfore, Amdahl's Law only holds for problems broken down in a particular, independent fashion.

Of course, this emergent property isn't magic; it can be explained by information transmission: the more places that information can be exchanged, the more powerful a compuation can be.