Marcos Guillen: Human Neural Network Simulation

Marcos Guillen

Marcos Guillen:
Artificial Development, Inc., CCortex
Human Neural Network Simulation

at the Accelerating Change Conference 2003
Stanford University, September 12-14, 2003
Edited by Peter Y. Chou
WisdomPortal.com

Marcos Guillen, founder, president & CEO of Artificial Development, Inc. gave a powerpoint presentation on "Human Neural Network Simulation". Previously, Marcos was co-founder and CEO of Red Internauta, Spain's largest independent dial-up ISP (1999-2002). Marcos apologized to the packed audience in the Cypress Room that this is his first presentation in English, so he's a bit nervous. But he appeared confident once his talk began. 28 slides were shown followed by a Q&A session. Here are my notes augmented by Guillen's PowerPoint Presentation which may be found and downloaded from his web site, www.ad.com.

(1)	CCortex: Building a 20-billion neuron emulation of the human cortex. WWW.AD.COM Artificial Development, Inc. 800 West El Camino Real Mountain View, CA 94040
(2)	CCortex: Building a 20-billion neuron emulation of the human cortex. 1. What is CCortex? 2. Why a brain emulation? 3. Is it feasible? 4. How does it Works? 5. Applications
(3)	What is CCortex? A software emulation of the brain • 20 billion neurons • 20 trillion connections • Powerful, fast, spiking neuron engine • Distributed parallel architecture • Tons of data • Very large three-dimensional neural networks architecture
(4)	Why an Emulation? • Scientists can only partially describe how the brain works. • A number of brain functions are not yet understand. Most mind theories are, so far, untested. • A few groups have tried partial, two-dimensional emulations of small portions of the brain. • Small neural networks are unable to test current mind theories.
(5)	Why an Emulation? • Current mind theories are based on three-dimensional interactions between very large neuron populations. • Theories describing chaotic interactions between Competing, spiking neural networks are, so far, untested. • Here comes the engineering approach: modelling the brain.
(6)	Why an Emulation? • CCortex use a Teraflops class supercomputer to model a complete version of the human cortex. • CCortex will support three-dimensional modelling of the most complex, challenging mind theories. • CCortex holds the promise of unlocking the solution to problems that has been eluding us so far.
(7)	Brain Emulation Constraints • Computer power. • Lack of detail in our knowledge of neural structures and circuitry. • Researchers have come forward with a number of memory algorithms, and... • Consciousness theories have not been successfully tested so far.
(8)	The CCortex Approach • Minimize computer power needs, thanks to highly optimized neural algorithms. • Maintain a live database of neural structures, incorporating past & new discoveries into the model. • Collaborate with researches to test different memory and consciousness theories.
(9)	Is a strong AI feasible now? The brain in numbers. • Average human brain contains about 85 billion neurons. • Of these, only 12 to 15 billion are telencephalic neurons. (Shariff 1953) [J. Comp. Neurol. 98:381-400] • 70 billion are cerebellar granule cells. (Lange 1975) • Fewer than 1 billion are brainstem and spinal neurons. • Some neurons can fire at up to 200 times per second. • We assume that most neurons fire 40-60 times per second.
(10)	Is a strong AI feasible now? No cerebellum? • CCortex will not implement a full version of the cerebellum. • The cerebellum contains up to 70% of all brain neurons. • Traditionally considered the lesser brain, it is just a motor brain to control fine movements. • New studies indicate that it may be involved in a wider variety of tasks, but we know that removing the cerebellum from young individuals often causes few obvious behavioral difficulties, suggesting that the rest of the brain can learn to function without it. (Bower, Parsons 2003)
(11)	Is a strong AI feasible now? The emulation in numbers • Neurons: 20,000,0000,000 • Connections: 1,000 • Average updates per second: 50 • Maximum theoretical computer power needed to emulate the brain: 1,000 Teraflops (8-bit), or 250 Teraflops (32-bit) • Roughly the power of 62,500 Intel Pentium 4 chips
(12)	Is a strong AI feasible now? The emulation in numbers • Actually, 250 Teraflops (32-bit) is not that much for such a conservative Maximum theoretical. • The proponents* of such a conservative Maximum Theoretical do not intend those measurements as predictions of when the strong AI will be possible, but of when it would be inevitable. • Next, we are going to compare it with a similarly conservative Pentium 4 emulation. *Kurzweil, Moravec
(13)	Is strong AI feasible now ? The emulation in Numbers Pentium 4 emulation: • Transistors: 100 Million • Clock Speed: 4 Ghz • Average updates per second: 50 • Maximum theoretical computer power needed to emulate a Pentium 4: 400,000 Teraflops (1-bit) 12,500 Teraflops (32-bit) • Roughly the power of 3 million Intel Pentium 4
(14)	Is strong AI feasible now ? The emulation in Numbers • It "may" take up to 62,500 Intel Pentium 4's to emulate the human cortex. — Suggested alternative: do not update the complete system 50 times per second, update only the part of the system that is changing. • It "may" take up to 3 million Pentium 4's to emulate a single Pentium 4 — Suggested alternative: buy an AMD processor!
(15)	Why these conservative estimates are not realistic predictions • While emulating the brain, we are performing very simple, 8-bit operations. • Most structures in our brain are redundant, and can be greatly simplified. • Chunks of post-synaptic weights data can be easily compressed. • Simple algorithms allow us to operate over compressed data • We don't need to calculate the state of every single neuron. Only neurons receiving outside input or presently connected to an active neuron needs to be updated.
(16)	CCortex: How does it work? • Massive computer power. • 500 dual processor cluster nodes with 1.000 processors. • Up to 4 Teraflops. • Terabytes of RAM.
(17)	CCortex: How does it work? • Neuron database. • Post-synaptic weight database. • Circuitry database. • Neuron-type database.
(18)	CCortex: How does it work? • Neuron database: — Each neuron has an entry on the database: • Neuron type • Neuron position • Neuron firing status • Number of connections
(19)	CCortex: How does it work? • Post-synaptic weight database: — Each pair of connected neurons has an 8-bit value entry — Values hold information about the post-synaptic weight, vector, and its positioning relative to the cell body
(20)	CCortex: How does it work? • Circuitry database: — It holds shortcuts and connections between different neural networks — Controls CCortex input-output and the way different brain structures communicate with each other
(21)	CCortex: How does it work? • Neuron-type database: Properties of different neuron types and populations. Some fields are: — Adaptation properties — Firing rate/shape — Number of connections — Positioning markers
(22)	CCortex: How does it work? Spiking Neuron Engine (SNE): The neuron as an algorithm. • Only neurons receiving input or recently stimulated are updated. • Once all the pertinent neurons has been updated, it starts again. • Easy to update. Do we have new theories to test? Just re-configure the algorithm. • Each cluster node is running several SNE, updating its own chunk of data, and communicating with each others (to maximize multithreading capabilities, dual processors, and memory reads). • The cluster has thousands of SNE running in parallel at any given time. Some SNE oversees (and sometimes overrules) the others.
(23)	CCortex: How does it work? The 'universal neuron' algo uploads its properties from a database. • Interpolates action potential shape. • Calculates if the firing potential has been reached. • Fires accordingly, choosing the right spiking rate and action potential rate. • Updates neuron status data, and, if needed, post-synaptic weights data.
(24)	CCortex: How does it work? • Data compression — Most data is compressed. — Database index and cached data are stored in RAM. • Low cost of mathematical operations Basic operations are relatively simple for a modern processor. — Some operations are performed over compressed data. — Some operations are not performed at all: it is more cost — efficient to look then up on a database or in cached data.
(25)	Applications • CCortex Cluster is a research version — CCortex Cluster installation is a "test bed," not the final product. — By modelling the cortex we aim at improving our basic understanding of the human intelligence, and to develop new, patentable algorithms to implement it. • CCortex future versions: CCortex appliances and boards — The final CCortex product will more likely be implemented using appliance hardware, FPGA boards, or custom hardware. — After all, custom hardware has allowed 3D graphics chip manufacturers to develop 300 Gflops graphic processor retailing at $300. A custom chip may be CCortex future.
(26)	Applications Some basic lines of research we are pursuing with the CCortex Cluster are: • Cracking the neural memory algo. • Reproducing mammalian level decision capabilities. • Reproducing human level cognitive capabilities.
(27)	Applications • Cracking the neural memory algorithm commercial applications: — New, radically improved Linguistic User Interfaces (LUI) for computers, cell phones and vehicles. — Quantum leap for data retrieving technologies, including relational databases and Internet search engines. — Pattern recognition technology orders of magnitude more powerful than anything available now.
(28)	Applications • Reproducing mammalian level decision- making capabilities has innumerable commercial applications. It would simply reshape the computer industry. • Reproducing human level cognitive capabilities would have an unpredictable economic impact: Futurists like John Smart could loose their jobs! Marcos Guillen's talk began at 11:45 am and ended at 12:10 pm

**************************************************

Q&A:

Q: (inaudible)
A: You wouldn't move the mouse.
The mouse will move itself.

Q: Is yours an academic or commercial enterprise?
A: We're all computer guys— computer programmers.
We don't have neurologists on board.
There are tons of research out there.
No one has put them together.

Q: How is your company funded?
A: I was successful in dot-com in Spain.
and am funding myself right now.

Q: Brains can optimize their functions,
so neurons shorten their pathways.
A: (answer inaudible or too quick to jot down)

Q: On the topic of mimicking mammalian brains,
you need to consult neurologists.
The brain has 10²¹ operations/sec,
so your factors (20 x 10⁹) are way off.
A: We're just starting. We're not there yet.

Q: (Ben Goertzel): How will your brain computer model linguistics?
A: That problem is way ahead of us. Right now, we're trying
to get as close to the human brain as possible.