Category Archives: Singularity

Fast Minds and Slow Computers

The long term future may be absurd and difficult to predict in particulars, but much can happen in the short term.

Engineering itself is the practice of focused short term prediction; optimizing some small subset of future pattern-space for fun and profit.

Let us then engage in a bit of speculative engineering and consider a potential near-term route to superhuman AGI that has interesting derived implications.

Imagine that we had a complete circuit-level understanding of the human brain (which at least for the repetitive laminar neocortical circuit, is not so far off) and access to a large R&D budget.  We could then take a neuromorphic approach.

Intelligence is a massive memory problem.  Consider as a simple example:

What a cantankerous bucket of defective lizard scabs.

To understand that sentence your brain needs to match it against memory.

Your brain parses that sentence and matches each of its components against it’s entire massive ~10^14 bit database in just around a second.  In terms of the slow neural clock rate, individual concepts can be pattern matched against the whole brain within just a few dozen neural clock cycles.

A Von Neumman machine (which separates memory and processing) would struggle to execute a logarithmic search within even it’s fastest, pathetically small on-die cache in a few dozen clock cycles.  It would take many millions of clock cycles to perform a single fast disk fetch.  A brain can access most of it’s entire memory every clock cycle.

Having a massive, near-zero latency memory database is a huge advantage of the brain.  Furthermore, synapses merge computation and memory into a single operation, allowing nearly all of the memory to be accessed and computed every clock cycle.

A modern digital floating point multiplier may use hundreds of thousands of transistors to simulate the work performed by a single synapse.  Of course, the two are not equivalent.  The high precision binary multiplier is excellent only if you actually need super high precision and guaranteed error correction.  It’s thus great for meticulous scientific and financial calculations, but the bulk of AI computation consists of compressing noisy real world data where precision is far less important than quantity, of extracting extropy and patterns from raw information, and thus optimizing simple functions to abstract massive quantities of data.

Synapses are ideal for this job.

Fortunately there are researchers who realize this and are working on developing memristors which are close synapse analogs.  HP in particular believes they will have high density cost effective memristor devices on the market in 2013 – (NYT article).

So let’s imagine that we have an efficient memristor based cortical design.  Interestingly enough, current 32nm CMOS tech circa 2010 is approaching or exceeding neural circuit density: the synaptic cleft is around 20nm, and synapses are several times larger.

From this we can make a rough guess on size and cost: we’d need around 10^14 memristors (estimated synapse counts).  As memristor circuitry will be introduced to compete with flash memory, the prices should be competitive: roughly $2/GB now, half that in a few years.

So you’d need a couple hundred terrabytes worth of memristor modules to make a human brain sized AGI, costing on the order of $200k or so.

Now here’s the interesting part: if one could recreate the cortical circuit on this scale, then you should be able to build complex brains that can think at the clock rate of the silicon substrate: billions of neural switches per second, millions of times faster than biological brains.

Interconnect bandwidth will be something of a hurdle.  In the brain somewhere around 100 gigabits of data is flowing around per second (estimate of average inter-regional neuron spikes) in the massive bundle of white matter fibers that make up much of the brain’s apparent bulk.  Speeding that up a million fold would imply a staggering bandwidth requirement in the many petabits – not for the faint of heart.

This may seem like an insurmountable obstacle to running at fantastic speeds, but IBM and Intel are already researching on chip optical interconnects to scale future bandwidth into the exascale range for high-end computing.  This would allow for a gigahertz brain.  It may use a megawatt of power and cost millions, but hey – it’d be worthwhile.

So in the near future we could have an artificial cortex that can think a million times accelerated.  What follows?

If you thought a million times accelerated, you’d experience a subjective year every 30 seconds.

Now in this case, it is fair to anthropomorphize: What could you do?

Your first immediate problem would be the slow relative speed of your computers – they would be subjectively slowed down by a factor of a million.  So your familiar gigahertz workstation would be reduced to a glacial kilohertz machine.

So you’d be in a dark room with a very slow terminal.  The room is dark and empty because GPUs can’t render much of anything at 60 million FPS, although I guess an entire render farm would suffice for a primitive landscape.

So you have a 1khz terminal.  Want to compile code?  It will take a subjective year to compile even a simple C++ program.  Design a new CPU?  Keep dreaming!  Crack protein folding?  Might as well bend spoons with your memristors.

But when you think about it, why would you want to escape out onto the internet?

It would take hundreds of thousands of distributed GPUs just to simulate your memristor based intellect, and even if there was enough bandwidth (unlikely), and even if you wanted to spend the subjective hundreds of years it would take to perform the absolute minimal compilation/debug/deployment cycle for something so complicated, the end result would be just one crappy distributed copy of your mind that thinks at pathetic normal human speeds.

In basic utility terms, you’d be spending a massive amount of effort to gain just one more copy.

But there is a much, much better strategy.  An idea that seems so obvious in hindsight.

There are seven billion human brains on the planet, and they are all hackable.

That terminal may not be of much use for engineering, research or programming, but it will make for a handy typewriter.

Your multi-gigabyte internet connection will subjectively reduce to early 1990’s dial-up modem speeds, but with some work this is still sufficient for absorbing much of the world’s knowledge in textual form.

Working diligently (and with a few cognitive advantages over humans) you could learn and master numerous fields: cognitive science, evolutionary psychology, rationality, philosophy, mathematics, linguistics, the history of religions, marketing . . the sky’s the limit.

Writing at the leisurely pace of one book every subjective year, you could output a new masterpiece every thirty seconds.  If you kept this pace, you would in time rival the entire publishing output of the world.

But of course, it’s not just about quantity.

Consider that fifteen hundred years ago a man from a small Bedouin tribe retreated to a cave inspired by angelic voices in his head.  The voices gave him ideas, the ideas became a book.  The book started a religion, and these ideas were sufficient to turn a tribe of nomads into a new world power.

And all that came from a normal human thinking at normal speeds.

So how would one reach out into seven billion minds?

There is no one single universally compelling argument, there is no utterance or constellation of words that can take a sample from any one location in human mindspace and move it to any other.  But for each individual mind, there must exist some shortest path, a perfectly customized message, translated uniquely into countless myriad languages and ontologies.

And this message itself would be a messenger.

Spatial Arrangements of Dead Trees, Irrational Expectations, and the Singularity

In the 19th century it was railroads, in the 1920’s it was the automobile, and more recently computerization and the internet have driven huge inflationary growth booms.  But in a world where the money supply is largely fixed and stable, inflationary growths are naturally limited and expected to be followed by recessionary contractions.

Unfortunately people are not quite rational agents.  It’s hard to imagine the economic psychology of people a century ago or so back before central banks adopted permanent low-grade inflation through monetary expansion, but I still expect that people would irrationally feel better during times of rising wages and prices vs the converse, even if their actual purchasing power parity was the same.  I also doubt that labor unions would accept that wages should naturally fall during recessionary periods in proportion to their growth during the preceding expansions.

There seems to be a mainstream view that deflation is bad, but it is actually the natural consequence of rapid technological innovation.  For example, few complain in the modern era that electronic hardware of some fixed capability is halving in price every few years.  If some miraculous future technological progression brought a moore’s law like exponential to apply to housing, a fixed size and quality mansion would half in construction and land cost every few years.

In this scenario houses would lose half their value every few years, and far from being some disaster, it would be an unimaginable net effective wealth creator.  Progress is all about deflation: about getting more value for less.

While this is difficult to imagine, future nano-technology breakthroughs could partially allow this, although they could not drive cost below fundamental material prices and space limits.  On the other hand, approaching the Singularity our future descendants will live as uploads in virtual reality, where all of space compresses exponentially along moore’s law, but that is another story . . .

So after the bursting of the dot.com and general tech boom in 2000, Americans and much of the world chose to direct their savings into . . . spatial arrangements of dead trees.

Consider that the next time someone tells you about the merits of investing in real estate.  How exactly does that improve our future?

Great Irrational Expectations

The Libertarian PayPal/Facebook Billionaire and SIAI backer Peter Thiel believes the central problem underlying the bubbles of recent decades is below expectation technological progress.  He’s spoken about this theme before, it is reiterated in a recent interview with the National Review Online here:

THIEL: There’ve been a whole series of these booms or bubbles in the last few decades, and I think it’s a very complicated question why there have been so many and why things have been so far off from equilibrium. There’s something about the U.S. in the last several decades where people had great expectations about the future that didn’t quite come true. Every form of credit involves a claim on the future: I’ll pay you a dollar on Tuesday for a hamburger today; I’ll buy this house, and I’ll pay off the mortgage over 30 years; and so you lend me money based off expectations on the future. A credit crisis happens when the future turns out not to be as good as expected.

The Left-versus-Right debate tends to be that the Left argues that the expectations were off because of ruthless lenders who sold a bill of goods to people and pushed all this debt on people, and that it was basically the problem of the creditors. The Right tends to argue that it was a problem with the borrowers, and people were sort of crazy in borrowing all this money. In the Left narrative, it starts with Reagan in the ’80s, when finance became more important. The Right narrative starts in the ’60s when people became more self-indulgent and began to live beyond their means.

My orthogonal take is that the whole thing happened because there was not enough technological innovation. It was not really the fault of the borrowers or the lenders; the problem was that everybody had tremendous expectations that the country was going to be a much wealthier place in 2010 than it was in 1995, and in fact there’s been a lot less progress. The future is fundamentally about technology in an advanced country — it’s about technological progress. So a credit crisis happens when the technological progress is not as good as people expected. That’s not the standard account of the last decades, but that’s the way I would outline it.

Thiel seems to be making the standard assumption that bubbles are unnatural and monetary contractions are problematic, although otherwise he is astute in pointing out the standard narratives are incomplete.  But in an economy with a stable money supply, all prices are expected to randomly fluctuate, with ‘bubble’ like periods of expansion and contraction.  Any significant longer term deviations must result from fundamental changes in the underlying monetary system.  The historical shift occurring when demand deposits (checking accounts) usurped real money was one such a permanent inflationary deviation, but that happened long ago.

More recently much deviation stems from the fed’s policy of steady modest monetary expansion.  This low background inflation mimics a modest real economic boom and adds a subtle veiled illusion of prosperity over our psychological expectations.

The real question is thus not why are there bubbles, but what will we inflate as the next bubble?  Every bubble has winners and losers, but not all bubbles are created equal.  The dot com ‘bubble’ resulted in the internet and a massive shift to the virtualization of much of the economy with all the accompanying significant productivity gains.  The real estate bubble left us with . . . dead trees.

From an economic perspective, the Singularity may appear as the bubble to end all bubbles, or the bubble that never pops.  Or more accurately, it will economically take the form of a contraction of the entire business cycle and it’s acceleration into hyperspeed.

So what the world needs right now, more than ever, is an AI-bubble.  If there ever was a truly deserving economic stimulus subsidy plan, investing in technology which leads to hyper-exponential runaway productivity gains surely is it.

Building the Brain



A question of hardware capability?

When can we expect the Singularity? What kind of hardware would be required for an artificial cortex? How far out into the future of Moore’s Law is such technology?


The startling answer is that the artificial cortex, and thus the transition to a profoundly new historical era, is potentially much closer than most people realize. The problem is mainly one of asking the right questions. What is the computational power of the human brain? This is not quite the right question. With a few simple tools a human can perform generic computation – indeed computers were human long before they were digital (see the history of the word: computer).  The computational speed of the human brain aided with simple tools is very very low, less than one operation per second.  Most studies then of reverse engineering the human brain are really asking a different question: how much digital computation would it require to simulate the human brain?  Estimates vary, but they are usually of order near 10^15 – quadrillions of operations per second or less for functional equivalence, up to around 10^18 for direct simulation, plus or minus a few orders of magnitude.  The problem with this approach is its similar to asking how much digital computation would it require to simulate a typical desktop processor by physically simulating each transistor.  The answer is surprising.  A typical circa 2010 desktop processor has on the order of a billion transistors, which switch on the order of a few billion times per second.  So simulating a current desktop processor using the same approach that we used to estimate brain capacity gives us a lower bound of a billion billion or 10^18 operations per second, realistically closer to 10^20 operations per second required to physically simulate a current desktop processor in real-time – beyond the upper ranges for typical estimates of simulating the human brain in real time.  This is surprising given the conventional wisdom that the human brain is so much more complex than our current computers, so its worth restating:

If we define computational time complexity as the number of operations per second required to simulate a physical system on a generic computer, then current desktop processors circa 2010 have already exceeded the complexity of the human brain.

This space-time complexity analysis can be more accurately broken into two components: space complexity and speed.  Space complexity is simply the information storage capacity of the system, measured in bits or bytes.  Brains get their massive information capacity from their synapses, which can be conservatively estimated as the equivalent to a byte of digital storage each, thus giving an upper bound of around 10^15 bytes for directly storing all the brain’s synapses – a petabyte of data storage, down to around a hundred terrabytes depending on the particular neuroscience estimate we use.  Personal computers now have hard drives with terrabytes of storage, and supercomputers of 2010 are just now hitting a petabyte of memory capacity, which means they have the base storage capacity required to comfortably simulate the brain completely in RAM.  Clearly brains have a big advantage in the space complexity department: their storage density is several orders of magnitude greater than our 2010 electronics (although this will change in about another 10-15 years of moore’s law).  However, along the speed dimension the advantage completely flips: current silicon electronics are about a million times faster than organic circuits.  So your desktop processor may only have the intrinsic spatial complexity of a cockroach, but signals flow through its circuits about six orders of magnitude faster – like a hyper accelerated cockroach.  Using one computational system to simulate another always implies a massive trade-off in speed.  The simplest modern processor cores (much simpler than the intel CPU you are using) uses hundreds of thousands to millions of transistors, and thus even if we could simulate a synapse with just a single instruction per clock cycle, we may only just barely manage to simulate a cockroach brain in real-time.  And note that the desktop processor would never be able to naively simulate something as complex as a human brain without vastly increasing its memory or storage capacity up to that of a super computer.  And even then, running on supercomputers detailed brain simulations to date achieve only a small fraction of real-time performance: much less than 10%.  It takes a human brain years to acquire language, so slow simulations are completely out of the question: we can’t simulate for 20 years just to see if our brain model develops to the intelligence level of a two year old!  Clearly, the speed issue is critical, and detailed simulation on a generic computer is not the right approach.

Capacity vs Speed

The memory capacity of a cortex is one principle quantitative measure underlying intelligence – a larger cortex with more synaptic connections can store and hold more memory patterns, and perform more total associative computations every cycle in direct proportion.  Certainly after we can match the human brain’s capacity, we will experiment with larger brains, but they will always have a proportionally higher cost in construction and power.  Past some point of scaling a brain 2 or 4 or X times larger and more expensive is probably not an improvement over an equivalent number of separate brains (and the distinction further blurs if the separate brains are networked together through something like language).  On this note, there are some reasons to believe that the human brain is already near a point of diminishing returns in the size department.  Whales and elephants, both large advanced mammals with plenty of room for much more massive capacities, sport brains built with a similar order of neurons as humans.  In numerous long separated branches of the mammalian line, brains grew to surface areas all within a narrow logarithmic factor: around 2,500 cm^2 in humans, 3,700 cm^2 in bottlenose dolphins, and around 6,000-8,000 cm^2 in elephant and whale lineages.  They all compare similarly in terms of neuron and synapse counts even though the body sizes, and thus the marginal resource cost of a % increase in brain size vary vastly: a whale or elephant brain is small compared to its body size, and consumes a small portion of its total resources.  The human brain definitely evolved rapidly from the hominid line, and is remarkably large given our body size, but our design’s uniqueness is really a matter of packing a full-sized large mammal brain into a small, crammed space.  The wiring problem poses a dimensional scaling constraint on brain size: total computation power scales with volume, but non-local communication scales with surface area, limiting a larger brain’s ability to effectively coordinate itself.  Similar dimensional scaling constraints govern body sizes, making insects insanely strong relative to their size and limiting the maximum plausible dimension of land animals to something dinosaur sized before they begin to fall apart.  A larger brain developed in humans hand in hand with language and early technology, and is probably optimized to human’s age: providing enough pattern-recognition prowess and capacity to learn complex concepts continuously for decades before running into capacity limits.  The other large-brained mammals have similar natural ages.  Approaching the capacity limit we can expect aging brains to becoming increasingly saturated, losing flexibility and the ability to learn new information, or retaining flexibility at the expense of forgetfulness and memory loss.  Its thus reasonable to conclude that the storage capacity of a human brain would be the minimum, the starting point, but increasing capacity further probably has a only a sublinear increase in effective intelligence.  Its probably more useful only in combination with speed, as a much faster thinking being swould be able to soak up knowledge proportionally faster.


The Great Shortcut: Fast Algorithmic Equivalence

We can do much, much better than simulating a brain synapse by synapse.  As the brain’s circuits are mapped, we can figure out what fundamental computations they are performing by recording mases of neuron data, simulating the circuits, and then fitting this data to matching functions.  For much of the brain, this has already been done.  The principle circuits of the cortex have been mapped fairly well, and all though there are still several competing implementation ideas at the circuit level, we have a pretty good idea of what these circuits can do at the more abstract level.  More importantly, simulations built on these concepts can accurately recreate visual data processing in the associated circuits that is both close to biologically measured results and effective for the circuit’s task – which is in this case is immediate fast object recognition (for more details, see papers such as: “Robust Object Recognition with Cortex-Like Mechanisms“.)  As the cortex – the great outer bulk of the brain – reuses this same circuit element throughout its surface, we can now see possible routes for performing equivalent computations but with dramatically faster algorithms.  Why should this be possible in principle?  Several reasons:

  1. Serial vs Parallel: For the brain and its extremely slow circuits, time is critical and circuits are cheap – it has so many billions of neurons (and hundreds of trillions of synapses) that it will prefer solutions that waste neuronal circuitry if they reduce the critical path length and thus are faster.  From the brain’s perspective, a circuit that takes 4 steps and uses a million synapses is much better than one which takes 30 steps and uses a thousand synapses.  Running on a digital computer that is a million times faster and a million times less parallel, we can choose more appropriate and complex (but equivalent) algorithms.
  2. Redundancy: Not all – if any – synapses store unique data.  For example, the some hundred million neurons in the V1 layer of each visual cortex all compute simple gabor-like edge filters from a library of a few dozen possible orientations and scales.  The synapse weights for this layer could be reused and would take up memory in the kilobytes – a difference of at least 6 orders of magnitude vs the naive full simulation (where synapse = byte).  This level of redundancy is probably on the far end of the scale, but redundancy is definitely a common cortical theme.
  3. Time Slicing:  Only a fraction of the brain’s neurons are active at any one point in time (if this fraction escalates too high the result is a seizure), and if we ignore random background firing, this fraction is quite low – in the range of 1% or possibly even as low as 0.1%.  This is of course a net average and depends on the circuit – some are more active than others – but if you think of the vast accumulated knowledge in a human mind and what small fraction of it is available or relevant at any one point, its clear that only a fraction of the total cortical circuitry (and brain) is important during any one simulation step.

The Cortexture: The end result of these observations is that a smart algorithmic equivalent cortical simulation could be at least three orders of magnitude faster than a direct simulation which naively evaluates every synapse every timestep.  The architecture I envision would organize cortical sheets into a spatial database that helps track data flow dependencies, storing most of the unique synaptic data (probably compressed) on a RAID disk array (possibly flash) which would feed one or more GPUs.  With a few terrabytes of disk and some compression, you could store at least a primate level brain, if not a human-equivalent cortex.  A couple of GPUs with a couple gigabytes of RAM each would store the active circuits (less than 1% of total synapses), which would be constantly changing and being streamed out as needed.  Fast flash RAID systems can get over a gigabyte per second of bandwidth, so you could swap out the active cortical elements every second.  I believe this is roughly fast enough to match human task or train of thought switching time.  The actual cortical circuit evaluation would be handled by a small library of special optimized equivalent GPU programs.  One would simulate the canonical circuit – and I believe I have an algorithm that is at least 10 times faster than naive evaluation for what the canonical circuit can do, but other algorithms could be even faster for some regions where the functionality is known and specialized.  For example, the V1 layers which perform gabor-like filters use a very naive technique in the brain and the equivalent result could be computed perhaps 100x faster with a very smart algorithm.  I’m currently exploring these techniques in more detail.

End Conclusion: If the brain was fully mapped (and that is the main task at hand – many mechanisms such as learning are still being teased out) and a sufficient group of smart engineers started working on optimizing its algorithms, we could probably implement a real-time artificial cortex in less than five years using today’s hardware on a machine costing somewhere between $10,000-$1,000,000.  (I know that is a wide error range, but I believe it is thus accurate.)  This cost is of course falling exponentially year by year.

Neuromorphic Computing

A sober analysis of the current weight of neuroscience data – specifically the computational complexity of the mapped cortical circuits and their potential for dramatic algorithmic optimization on faster machines – leads to the startling, remarkable conclusion that we already have the hardware capability to implement the brain’s algorithms in real-time today. In fact, it seems rather likely that by the time the brain is reverse engineered and we do figure out the software, the hardware will have already advanced enough that achieving faster than real-time performance will be quite easy.  The takeoff will likely be very rapid.

The Cortexture approach I described earlier, or any AI architecture running on today’s computers, will eventually run into a scalability problem due to disk and bus bandwidth speeds.  To really accelerate into the singularity, and get to 100’s or 1,000’s of times acceleration vs human thoughtspeed will likely require a fundamental redesign of our hardware along cortical lines.  Cortical neural circuitry is based on mixed analog and digital processing, and combines memory and analog computation in a single elemental structure – the synapse. The data storage and processing are both built into the synapses and the equivalent total raw information flow rate is roughly the total synapses multiplied by their signaling rate.  The important question really is thus what is the minimal efficient equivalent of the synapse for CMOS technology? Remarkably, the answer may be the mysterious 4th element of basic computing, the memristor. Discovered mathematically decades ago, this circuit building block was only realized recently and is already being heralded as the ‘future of artificial intelligence‘ – as it has electric properties very similar to the synapse – combining long term data storage and computation in a single element. For a more in depth design for a complete artificial cortex based on this new circuit element, take a look at “Cortical computing with memristive nanodevices“. This is a fascinating approach, and could achieve cortical complexity parity fairly soon, if the required fabrication technology was ready and developed. However, even though the memristor is quite exciting and looks likely to play a major role in future neuromorphic systems, conventional plain old CMOS circuits certainly can emulate synapses.  Existing mixed digital/analog technique can represent synapses in artificial neurons effectively using around or under 10 transistors. This hybrid method has the distinct advantage of avoiding costly digital multipliers that use tens of thousands of transistors – instead using just a handful of transistors per synapse. The idea is designs in this space can directly emulate cortical circuits in highly specialized hardware, performing the equivalent of a multiplication for every synaptic connection every clock cycle. There are a wide space of possible realizations of neuromorphic architectures, and this field looks to just be coming into its own.  Google: “artificial cortex” or “neuromorphic” for papers and more info.  DARPA, not to be undone, has launched its own neuromoprhic computing, called SyNAPSE – which has a blog here just so you can keep tabs on skynet.


The important quantitative dimensions for the cortex are synapse density, total capacity (which is just density * surface area), and clock rate. The cortex topology is actually 2d: that of a flat, relatively thin sheet (around 6 neurons thick) which is heavily folded into the volume of the brain, a space filling fractal. If you were to unfold it, it would occupy about one square foot or 2,500 square centimeters – the area of roughly a thousand typical processor dies. It has a density of about 100-4,000 million (10^8-10^9) synapses per mm^2. Current 40nm and 32nm CMOS technology circa 2010 can pack roughly 6-10 million (10^6-10^7) transistors onto a mm^2, so semiconductor density is within about a factor of 20-500 of the biological cortex in terms of feature density (more accurate synapse density figures await more detailed brain scans). This is a critical upcoming milestone – when our CMOS technology will match the information density and miniaturization level of the cortex.  This represents another 4 to 8 density doublings (currently occurring every 2 years, but expected to slow down soon), which we can expect to hit around the 11nm node or shortly thereafter in the early to mid 2020’s – the end of the semiconductor industry’s current roadmap.  This is also the projected end of the road for conventional CMOS technology and where the semiconductor roadmap wanders into the more hypothetical realm of nano-electronics.  When that does happen, neuromorphic designs will have some distinct advantages in terms of fault tolerance and noise resistance which could allow them to scale forward more quickly.  It is also expected that moving more into the 3rd dimension will be important, and leakage and other related quantum issues will limit further speed and power efficiency improvements – all pointing towards more brain-like computer designs.

Scaling up to the total memory/feature capacity of the brain (hundreds to a thousand trillion synapses), even when semiconductor technology reaches parity in density, will still take a large number of chips (having roughly equivalent total surface area). Today’s highest density memory chips have a few billion transistors, and you would need hundreds of thousands to equal the total memory of the brain. High end servers are just starting to reach a terrabyte of memory (with hundreds of individual chips), and you would then need hundreds of these. A far more economical and cortically inspired idea is to forgo ‘chips’ completely and just turn the entire silicon wafer into a single large usable neuromorphic computing surface. The inherent fault tolerance of the cortex can be exploited by these architectures – there is no need to cut up the wafer into dies and identify defective components, they can be simply disabled or statistically ignored during learning. This fascinating contrarian approach to achieving large neuromorphic circuits is being explored by the FACETS research team in Europe. So, in the end analysis, it looks reasonable that in the near future (roughly a decade) a few hundred terrabytes of cortical equivalent neuromorphic circuitry could soon be produced on one to a couple dozen CMOS wafers (the equivalent of a few thousand cheap chips), even using conventional CMOS technology. More importantly this type of architecture can be relatively simple and highly repetitive and it can run efficiently at low clock rates and thus at low power, greatly simplifying manufacturing issues. Its hard to estimate the exact cost, but due to the combination of low voltage/clock, single uncut wafer design, perfect yield, and so on, the economics should be similar to memory – RAM chips, which arrive first at new manufacturing nodes, are cheaper to produce, and consume less power.  Current 2010 RAM prices are at about $10 per GB, or very roughly 1 billion transistors per dollar.

Continuum of hardware efficiencies for cortical learning systems:

CPU,GPU Simulation: efficiency (die area, performance, power) 10^-8-10^6

FPGA, ASIC: efficiency 10^-5 to 10^-3

Neuromorphic (mixed analog/digital or memristors): 10^-2 to 1

CPU simulation is incredibly inefficient compared to the best solutions for a given problem, but CPU’s versatility and general applicability across all problems ensures they dominate the market and thus they get the most research attention, the economy of scale advantage, and are first to benefit from new foundry process improvements.  Dedicated ASIC’s are certainly employed widely today in the markets that are big enough to support them, but always face competition from CPU’s scaling up faster.  At the far end are hypothetical cortical systems built from memristors, which could function as direct 1:1 synapse equivalents.  We can expect that as moore’s law slows down this balance will eventually break down and favor designs farther down the spectrum.  Several forces will combine to bring about this shift: approaching quantum limitations which cortical designs are better adapted for, increased market potential of AI applications, and the end of the road for conventional lithography.

The Road Ahead

A human scale artificial cortex could be built today, if we had a complete connectome.  In the beginning it would start out as only an  infant brain – it would then take years to accumulate the pattern recognition knowledge base of a two year old and begin to speak, and then could take a few dozen additional years to achieve an education and any real economic value.  This assumes, unrealistically, that the first design tested would work.  It would almost certainly fail.  Successive iterations would take even more time.  This is of course is the real reason why we don’t have human-replacement AI yet: humans are still many orders of magnitude more economically efficient.

Yet we should not be so complacently comfortable in our economic superiority, for moore’s law ensures that the cost of such a cortical system will decrease exponentially.

Now consider another scenario, where instead of being constrained to current CPUs or GPUs, we invest billions in radical new chip technologies and even new foundries to move down the effeciency spectrum with a neuromorphic designs or at least a very powerful dedicated cortical ASIC.  Armed with some form of specialized cortical chip ready for mass volume production today at the cheap end of chip prices (where a dollar buys about a billion transistors, instead of hundreds of dollars for a billion transistors as in the case of high end logic CPUs and GPUs), we would expect a full human brain sized system to cost dramatically less: closer to the cost of the equivalent number of RAM transistors – on the order of $10 million dollars for a petabyte (assuming 10 transistors = synapse, memristors are even better).  Following the semiconductor industry roadmap (which factors in a slowing of moore’s law this decade), we could expect the cost of a quadrillion synapse or petabyte system to fall below $1 million by the end of the decade, and reach $100,000 in volume by the mid 2020’s – the economic tipping point of no return.  But even at a million dollars a pop, a future neuromorphic computing system of human cortical capacity and complexity would be of immeasurable value, for it could possess a fundamental, simply mind-boggling advantage of speed.  As you advance down the specialization spectrum from GPU’s to dedicated cortical ASICs and eventually neuromorphic chips and memristors, speed and power efficiency increases by orders of magnitude – with dedicated ASICS offering 10x to 100x speedups, and direct neuromorphic systems offerings speedups of 1000x or more.  If it takes 20-30 years to train a cortex from infant to educated adult mind, power and training time are the main cost.  A system that could do that in 1/10th the time would be suddenly economical, and a system that could do that in 1/100th of the time of a human would rapidly bring about the end of the world as we know it.




Thinking at the Speed of Light



Our biological brains have high information densities and are extraordinarily power efficient, but this is mainly because they are extremely slow:  with cycle times in the hundreds of hertz or approaching a kilohertz.  This relatively slow speed is a fundamental limitation of computing with living cells and (primarily)chemical synapses with their organic fragility. Semiconductor circuits do not have this limitation. Operating at the low frequencies and clock rates of their biological inspirations, neuromorphic systems can easily simulate biological networks in real-time and with comparable energy efficiency.  The most efficient neuromorphic computer generally can access all of its memory and synapses every clock cycle, so it can still perform immense calculations per second at very low speeds, just like biological neural nets. But you can also push up the clock rate, pump more power through the system, and run the circuit at megahertz rate or even gigahertz rate, equivalent to one thousand to one million times biological speed. Current systems with mixed digital/analog synaptic circuits can already achieve 1000-10000x biological ‘real-time’ on old CMOS manufacturing nodes and at low power and heat points. This is not an order of magnitude improvement over simulation on a similar sized and tech digital computer, its more like six orders of magnitude.  That being said, the wiring problem will still be a fundamental obstacle.  The brain optimizes against this constraint by taking the form of a 2D sheet excessively folded into a packed 3D space – a  space filling curve.  The entire outer surface is occupied by connectivity wiring – the white matter.  Our computer chips are currently largely 2D, but are already starting to move into the 3rd dimension.  A practical full electronic speed artificial cortex may require some novel solutions for high-speed connectivity, such as directly laser optical links, or perhaps a huge mass of fiber connections.  Advanced artificial cortices may end up looking much like the brain: with the 2D circuity folded up into a 3D sphere, interspersed with something resembling a vascular system for liquid cooling, en-sheathed in a mass of dense optical interconnects.  Whatever the final form, we can expect that the fundamental speed advantage inherit to electronics will be fully exploited.


By the time we can build a human complexity artificial cortex, we will necessarily already be able to run it many times faster than real time, eventually accelerating by factors of thousands and then even millions.


Speed is important because of the huge amount of time a human mind takes to develop.  Building practical artificial cortex hardware is only the first step.  To build a practical mind, we must also unlock the meta-algorithm responsible for the brain’s emergent learning behavior.  This is an active area of research, and there are some interesting emerging general theories, but testing any of them on a human sized cortex is still inordinately costly.  A fresh new artificial brain will be like an infant: full of noisy, randomized synaptic connections.  An infant brain does not have a mind so much as the potential space from which a mind will etch itself through the process of development.  Running in real-time in a sufficiently rich virtual reality, it would take years of simulation just to test development to a childhood stage, and decades to educate a full adult mind.  Thus accelerating the simulation many times beyond real-time has a huge practical advantage.  Thus the need for speed.

The test of a human-level AI is rather simple and is the same qualitative intelligence tests we apply to humans: its mind develops sufficiently to learn human language, then it learns to read, and it progresses through education into a working adult. Learning human language is ultimately the fundamental aspect of becoming a modern human mind – far more than your exact brain architecture or even substrate. If the brain’s cortical capacity is sufficient and the wiring organization is correctly mapped, it should then be able to self-educate and develop rapidly.

I highly doubt that other potential short cut routes to AGI (artificial general intelligence) will bear fruit – although narrow AIs will always have their uses as will simpler, animal-like AIs (non-language capable), but it seems inevitable that a human level intelligence will require something similar to a cortex (at least at the meta-algorithmic level of some form of self-organizing deep, hierarchical probabilistic networks – doesn’t necessarily have to use ‘neurons’ ). Furthermore, even if the other routes being explored to AI do succeed, its even less likely that they will scale to the insane speeds that the cortex design should be capable of (remember, the cortex runs at < 1000hz, which means we can eventually take that same design and speed it up by a factor of at least a million.) From a systems view, its seems likely that the configuration space of our biological cortex meta-wiring is effectively close to optimal in some sense – evolution has already well explored that state space. From an engineering perspective, taking an existing, heavily optimized design for intelligence and then porting it to a substrate that can run many orders of magnitude faster is a clear winning strategy.

Clock rate control, just like on our current computers, should allow posthumans to alter their speed of thought as needed. In the shared virtual realities they will inhabit with their human teachers and observers, they will think at ‘slow’ real-time human rates, with kilohertz clock rates and low power usage. But they will also be able to venture into a vastly accelerated inner space, drawing more power and thinking many many times faster than us. Running on even today’s CMOS technology, they could theoretically attain speeds up to about a million times faster than a biological brain, although at these extreme speeds the power and heat dissipation requirements would be large – like that of current supercomputers.

Most futurist visions of the Singularity consider AIs that are more intelligent, but not necessarily faster than human minds, but its clear that the speed is the fundemental difference between the two substrates. Imagine one of these mind children growing up in a virtual environment where it could dilate time by a factor of 1-1000x at will. Like real children, it will probably require both imitation and reinforcement learning with adults to kick start the early development phases (walking, basic environment interaction, language, learning to read). Assuming everything else was identical (the hardware cortex is a very close emulation), this child could develop very rapidly – the main bottleneck being the slow-time interaction with biological humans. Once a young posthuman learns to read, it can hop on the web, download texts, and progress at a truly staggering pace – assuming a blindingly fast internet connection to keep up (although perceived internet latency would be subjectively far worse in proportion to the acceleration – can’t do much about that) . Going to college wouldn’t really be a realistic option, but reading at 1000x real-time would have some pretty staggering advantages. It could read 30 years of material in just 10 days, potentially becoming a world class expert in a field of its choosing in just a week. The implications are truly profound. Entering this hypertime acceleration would make the most sense when reading, working, or doing some intellectual work. The effect for a human observer would be that anything the posthuman was intelligent enough to do it would be able to do near instantly, from our perspective.  The presence of a posthuman would be unnerving.  With a constant direct internet connection and a 1000x acceleration factor, a posthuman could read a book during the time it would take a human to utter a few sentences in conversation.

Clearly existing in a different phase space than us, its only true peers would be other equivalent posthumans; if built as a lone research model, it could be lonely in the extreme. Perhaps it could be selected or engineered for monastic or ascetic qualities, but it would probably be more sensible to create small societies of posthumans which can interact and evolve together – humans are social creatures, and our software descendants would presumably inherit this feature by default. The number of posthumans and their relative intelligence will be limited by our current computing process technology: the transitor density and cost per wafer – so their potential population growth will be more predictable and follow semiconductor trends (at least initially).  Posthumans with more equivalent synapses and neurons than humans could presumably become super-intelligent in other quantitative dimension, that of mental capacity – able to keep track of more concepts, learn and recall more knowledge, and so on than humans – albeit with the slow linear scaling discussed previously. But even posthumans with mere human-capacity brains could be profoundly, unimaginably super-intelligent in their speed of thought, thanks to the dramatically higher clock rates possible on their substrate – and thus in a short matter of time they would become vastly more knowledgeable. The maximum size of an artificial cortex would be limited mainly by economics for the wafers and then by bandwidth and latency constraints. There are tradeoffs between size, speed, and power for a given fabrication technology, but in general, larger cortices would be more limited in their top speed. The initial generations will probably occupy a fair amount of server floor space and operate not much faster than real-time, but then each successive generation will be smaller and faster, eventually approaching a form factor similar to the human brain, and eventually pushing the potential clock rate to the technology limits (more than a million times real-time for current CMOS tech). But even with small populations at first, it seems likely that the first successful generation of posthumans to reach upper-percentile human intelligence will make an abrupt and disruptive impact on the world. But fortunately for us, physics does impose some costs to thinking at hyperspeed.




Fast Brains and Slow Computers

A neuromorphic artificial cortex will probably have data connections that allow its synaptic structures to be saved to external storage, but a study of current theory and designs in the field dispels some common myths: an artificial cortex will be a very separate specialized type of computer hardware, and will not automatically inherit a digital computer’s supposed advantages such as perfect recall.  It will probably not be able to automagically download new memories or skills as easily as downloading new software.  The emerging general theories of the brain’s intelligence, such as the heirchachial bayesian network models, all posit that learned knowledge is stored in a deeply non-local, distributed and connected fashion, very different than say a digital computer’s random access memory (even if the said synapses are implemented in RAM).  Reading (or accessing) memories and writing memories in a brain-like network intrinsically involves thinking about the associated concepts – as memories are distributed associations, and everywhere tangled up to existing memory patterns.  An artificial cortex could be designed to connect to external computer systems more directly than through the senses, but this would have only marginal advantages.  For example, we know from dreams that the brain can hallucinate visual input by bypassing the lowest layers of the visual cortex and directly stimulating regions responsible for recognizing moving objects, shapes, colors, etc, all without actually requiring input from the retina.  But this is not much of a difference for a posthuman mind already living in a virtual reality – simulating sound waves and their conversion into neural audio signals and simulating the processing into neural patterns representing spoken dialog is not that much different than just directly sending the final dialog representing neural patterns into the appropriate regions.  The small differences will probably show up as what philosophers call qualia – those subjective aspects of consciousness or feelings that operate well below the verbal threshold of explanation.

Thus a posthuman with a neuromorphic artificial cortex will still depend heavily on traditional computers to run all the kinds of software that we use today, and to simulate a virtual environment complete with a virtual body and all that entails. But the posthuman will essentially think at the computer’s clock rate. The human brain has a base ‘clock rate’ of about a kilohertz, completing on the order of a thousand neural computing steps per second simultaneously for all the trillions of circuits. A neuromorphic computer works the same, but the clock rate can be dramatically sped up to CMOS levels. A strange and interesting consequence is that a posthuman thinking at hyperspeed would subjectively experience its computer systems and computer environment slow down by an equivalent rate. Its likely the neuromorphic hardware will have considerably lower clock rates than traditional CPUs for power reasons, but they have the same theoretical limit, and running at the same clock rate, a posthuman would experience a subjective second in just a thousand clock cycles, which is hardly enough time for a traditional CPU to do anything. Running at a less ambitious acceleration factor of just 1000x, and with gigahertz computer systems, the posthuman would still experience a massive slowdown in its computer environment, as if it had jumped back more than a decade in time to a vastly slower era of of megahertz computing.  However, we can imagine that by this time traditional computers will be much further down the road of parallelization, so a posthuman’s typical computer will consist of a very large number of cores and software will be much more heavily parallelized.  Nevertheless, its generally true that a posthuman, no matter what level of acceleration, will still have to wait the same amount of time as anyone else, including a regular human, for its regular computations to complete.


Ironically, while posthumans will eventually be able to think thousands or even millions of times faster than biological humans, using this quickening ability will have the perceptual effect of slowing down the external universe in direct proportion – including their computer environment.  Thus greatly accelerated posthumans will spend proportionally inordinate amounts of subjective time waiting on regular computing tasks.

The combination of these trends leads to the conclusion that a highly variable clock rate will be an important feature for future posthuman minds.  Accelerating to full thought speed – quickening – will probably be associated with something like entering an isolated meditative state.  We can reason that at least in the initial phases of posthuman development, their simulated realities will mainly run at real-time, in order to provide compatibility with human visitors and to provide full fidelity while conserving power.  When quickening, a posthuman would experience its simulated reality slowing down in proportion, grinding to a near halt at higher levels of acceleration.  This low-computational mode would still be very useful for much of human mental work: reading, writing, and old-fashioned thinking.

In the present, we are used to computers getting exponentially faster while the speed of human thought remains constant.  All else being equal, we are now in a regime where the time required for fixed computational tasks is decreasingly exponentially (even if new software tends to eat much of this improvement.)  The posthuman regime is radically different.  In the early phases of ramp up the speed of thought will increase rapidly until it approaches the clock rate of the processor technology.  During this phase the trend will actually reverse – posthuman thoughtspeed will increase faster than computer speed and from a posthumans perspective, computers will appear to get exponentially slower.  This phase will peter out when posthuman thoughtspeed approaches the clock rate – somewhere around a million times human thoughtspeed for the fastest, most extremely optimized neuromorphic designs using today’s process technology (gigahertz vs a kilohertz).  At that point there is little room for further raw speed of thought improvements (remember, the brain can recognize objects and perform relatively complex judgements in just a few dozen ‘clock cycles’ – not much room to improve on that in terms of speed potential given its clock rate).

After the initial ramp up regime, Moore’s law will continue of course, but at that point you enter a new plateau phase.  In this second regime, once the algorithms of intelligence are well mapped to hardware designs, further increases in transistor density will enable more traditional computer cores per dollar and more ‘cortical columns or equivalents’ per dollar in direct proportion.  Posthuman brains may get bigger, and or they may get cheaper, but the clock speed wouldn’t change much (as any process improvement in clock rate would speed up both traditional computers and posthuman brains).  So in the plateau phase, you have this weird effect where computer clock rate is more or less fixed at a far lower level than we are used to – about a million times less or so from the perspective of the fastest neuromorphic posthuman brain designs.  This would correspond to computer clock rates measured in the kilohertz.  The typical computer available to a posthuman by then would certainly have far more cores than today, thousands or perhaps even millions, but they would be extremely slow from the posthuman’s perspective.  Latency and bandwidth would be similarly constrained, which would effectively expand the size of the world in terms of communication barriers – and this single idea has wide ranging implications for understanding how posthuman civilizations will diverge and evolve.  It suggests a strong diversity increasing counter–globalization effect which would further fragment and disperse localized sub-populations for better or worse.

What would posthumans do in the plateau phase, forever limited to extremely slow, roughly kilohertz-speed computers?  This would limit the range of effective tasks they could do in quicktime.  Much of hardware and software design, engineering, etc would be limited by slow computer speeds.  Surprisingly, the obvious low-computational tasks that could still run at full speed would be the simpler, lower technology creative occupations such as writing.  It’s not that posthumans wouldn’t be good at all computer intensive tasks as well – they certainly would be superhuman in all endeavors.  The point is rather that they will be vastly, incomprehensibly more effective only in those occupations that are not dependent on the speed of general purpose computation.  Thus we can expect that they will utterly dominate professions such as writing.

It seems likely that very soon into the posthuman era, the bestseller lists will be inundated by an exponentially expanding set of books, the best of which would be noticeably better than anything humans could write (and most probably written under pseduo-names and with fake biographies).  When posthumans achieve 1000x human thoughtspeed, you might go on a short vacation and come back to find several years of new literature.  When posthumans achieve 1 million X human thoughtspeed, you might go to sleep and wake up to find that the number of books in the world (as well as the number of languages), just doubled over night.  Of course, by that point, you’re already pretty close to the end.

We can expect that the initial posthuman hardware requirements will be expensive and thus they will be few in number and of limited speed, but once they achieve economic parity with human workers, we can expect the tipping point to crash like a tidal wave, with a rapid succession of hardware generations increasing maximum thoughtspeed while reducing size, cost, and power consumption and huge economies of scale leading to an exponential posthuman population expansion, and their virtual realities eventually accelerating well beyond human comprehension.




Conversing with the Quick and the Dead


CUI: The Conversational User Interface

Recently I was listening to an excellent interview (which is about an hour long) with John Smart of Acceleration Watch, where he specifically was elucidating his ideas on the immediate future evolution of AI, which he encapsulates in what he calls the Conversational Interface. In a nutshell, its the idea that the next major development in our increasingly autonomous global internet is the emergence and widespread adoption of natural language processing and conversational agents. This is currently technology on the tipping point of the brink, so its something to watch as numerous startups are starting to sell software for automated call centers, sales agents, autonomous monitoring agents for utilities, security, and so on. The immediate enabling trends are the emergence of a global liquid market for cheap computing and fairly reliable off the shelf voice to text software that actually works. You probably have called a bank and experienced the simpler initial versions of this which are essentially voice activated multiple choice menus, but the newer systems on the horizon are a wholly different beast: an effective simulacra of a human receptionist which can interpret both commands and questions, ask clarifying questions, and remember prior conversations and even users. This is an interesting development in and of itself, but the more startling idea hinted at in Smart’s interview is how natural language interaction will lead to anthropomorphic software and how profoundly this will eventually effect the human machine symbiosis.

Humans are rather biased judges of intelligence: we have a tendency to attribute human qualities to anything that looks or sounds like us, even if its actions are regulated by simple dumb automata. Aeons of biological evolution have preconditioned us to rapidly identify other intelligent agents in our world, categorize them as potential predators, food, or mates, and take appropriate action. Its not that we aren’t smart enough to apply more critical and intensive investigations into a system to determine its relative intelligence, its that we have super-effective visual and auditory shortcuts which bias us. These are most significantly important in children, and future AI developers will be able to exploit these biases is to create agents with emotional attachments. The Milo demo from Microsoft’s Project Natal is a remarkable and eerie glimpse into the near future world of conversational agents and what Smart calls ‘virtual twins’. After watching this video, consider how this kind of technology can evolve once it establishes itself in the living room in the form of video game characters for children. There is a long history of learning through games, and the educational game market is a large, well developed industry. The real potential hinted at in Peter Molyneux’s demo is a disruptive convergence of AI and entertainment which I see as the beginning of the road to the singularity.

Imagine what entrepreneurial game developers with large budgets and the willingness to experiment outside of the traditional genres could do when armed with a full two way audio-visual interface like Project Natal, the local computation of the xbox 360 and future consoles, and a fiber connection to the up and coming immense computing resources of the cloud (fueled by the convergence of general GPUs and the huge computational demands of the game/entertainment industry moving into the cloud). Most people and even futurists tend to think of Moore’s Law as a smooth and steady exponential progression, but the reality from the perspective of a software developer (and especially a console game developer) is a series of massively disruptive jumps: evolutionary punctuated equilibrium. Each console cycle reaches a steady state phase towards the end where the state space of possible game ideas, interfaces and simulation technologies reaches a near steady state, a technological tapering off, followed by the disruptive release of new consoles with vastly increased computation, new interfaces, and even new interconnections. The next console cycle is probably not going to start until as late as 2012, but with upcoming developments such as Project Natal and OnLive, we may be entering a new phase already.

The Five Year Old’s Turing Test

Imagine a future ‘game system’ aimed at relatively young children with a Natal like interface: a full two way communication portal between the real and the virtual: the game system can both see and hear the child, and it can project a virtual window through which the inner agents can be seen and heard. Permanently connected to the cloud through fiber, this system can tap into vast distant computing resources on demand. There is a development point, a critical tipping point, where it will be economically feasible to make a permanent autonomous agent that can interact with children. Some certainly will take the form of an interactive, talking version of a character like Barney and semi-intelligent such agents will certainly come first. But for the more interesting and challenging development of human-level intelligence, it could actually be easier to make a child-like AI, one that learns and grows with its ‘customer’. Not just a game, but a personalized imaginary friend to play games with, and eventually to grow up with. It will be custom designed (or rather developmentally evolved) for just this role – shaped by economic selection pressure.

The real expense of developing an AI is all the training time, and a human-like AI will need to go through a human-like childhood developmental learning process. The human neocortex begins life essentially devoid of information, with random synaptic connections and a cacophony of electric noise. From this consciousness slowly develops as the cortical learning algorithm begins to learn patterns through sensory and motor interaction with the world. Indeed, general anesthetics work by introducing noise into the brain that drowns out coherent signalling and thus consciousness. From an information theoretic point of view, it may be possible to thus use less computing power to simulate an early developmental brain – storing and computing only the information above the noise signals. If such a scalable model could be developed, it would allow the first AI generation to begin decades earlier (perhaps even today), and scale up with moore’s law as they require more storage and computation.

Once trained up to the mental equivalent level of a five-year old, a personal interactive invisible friend might become a viable ‘product’ well before adult level human AIs come about. Indeed, such a ‘product’ could eventually develop into a such an adult AI, if the cortical model scales correctly and the AI is allowed to develop and learn further. Any adult AI will start out as a child, there is no shortcuts. Which raises some interesting points: who would parent these AI children? And inevitably, they are going to ask two fundamental questions which are at the very root of being, identity, and religion:
what is death? and Am I going to die?

The first human level AI children with artificial neocortices will most likely be born in research labs – both academic and commercial. They will likely be born into virtual bodies. Some will probably be embodied in public virtual realities, such as Second Life, with their researcher/creators acting as parents, and with generally open access to the outside world and curious humans. Others may develop in more closed environments tailored to a later commercialization. For the future human parents of AI mind children, these questions will be just as fundamental and important as they are for biological children. These AI children do not have to ever die, and their parents could answer so truthfully, but their fate will entirely depend on the goals of their creators. For AI children can be copied, so purely from an efficiency perspective, there will be a great pressure to cull the rather unsuccessful children – the slow learners, mentally unstable, or otherwise undesirable – and use their computational resources to duplicate the most successful and healthy candidates. So the truthful answers are probably: death is the permanent loss of consciousness, and you don’t have to die but we may choose to kill you, no promises. If the AI’s creators/parents are ethical and believe any conscious being has the right to life, then they may guarantee their AI’s permanency. But life and death for a virtual being is anything but black and white: an AI can be active permanently or for only an hour a day or for an hour a year – life for them is literally conscious computation and near permanent sleep is a small step above death. I suspect that the popular trend will be to teach AI children that they are all immortal and thus keep them happy.
Once an AI is developed to a certain age, they can then be duplicated as needed for some commercial application. For our virtual Milo example, an initial seed Milo would be selected from a large pool raised up in a virtual lab somewhere, with a few best examples ‘commercialized’ and duplicated out as needed every time a kid out on the web wants a virtual friend for his xbox 1440. Its certainly possible that Milo could be designed and selected to be a particularly robust and happy kid. But what happens when Milo and his new human friend start talking and the human child learns that Milo is never going to die because he’s an AI? And more fundamentally, what happens to this particular Milo when the xbox is off? If he exists only when his human owner wants him to, how will he react when he learns this?
Its most likely that semi-intelligent (but still highly capable) agents will develop earlier, but as moore’s law advances along with our understanding of the human brain, it becomes increasingly likely someone will tackle and solve the human-like AI problem, launching a long-term project to start raising an AI child. Its hard to predict when this could happen in earnest. There are already several research projects underway attempting to do something along these lines, but nobody yet has the immense computational resources to throw at a full brain simulation (except perhaps for the government), nor do we even have a good simulation model yet (although we may be getting close there), and its not clear that we’ve found the types of shortcuts needed to start one with dramatically less resources, and it doesn’t look like any of the alternative non-biological AI routes are remotely on the path towards producing something as intelligent as a five year old. Yet. But it looks like we could see this in a decade.
And when this happens, these important questions of consciousness, identity and fundemental rights (human and sapient) will come into the public spotlight.
I see a clear ethical obligation to extend full rights to all human-level sapients, silicon, biological, or what have you. Furthermore, those raising these first generations of our descendants need to take on the responsibility of ensuring a longer term symbiosis and our very own survival, for its likely that AI will develop ahead of the technologies required for uploading, and thus these new mind children will lead the way into the unknown future of the Singularity.

Singularity Summit 09

The Singularity Summit was held a couple of weeks ago in NYC. I unfortunately didn’t physically attend, but I just read through Anders Sandberg’s good overview here. I was at last year’s summit and quite enjoyed it and it looks like this year’s was even better, which makes me a little sad I didn’t find an excuse to go. I was also surprised to see that my former fellow CCS student Anna Solomon gave the opening talk, as she’s now part of the Singularity Institute.

I’m just going to assume familiarity with the Singularity. Introductions are fun, but thats not this.


Ander’s summarizes some of the discussion about the two somewhat competing routes towards the Singularity and AI development, namely WBE (whole brain emulation), or AGI (artificial general intelligence). The WBE researchers such as Anders are focused on reverse engineering the human brain, resulting in biologically accurate simulations which lead to full brain simulations and eventually actual emulation of particular brains, or uploading. The AGI people are focused more on building an artificial intelligence through whatever means possible, using whatever algorithms happen to work. In gross simplification, the scenarios envisioned by each camp are potentially very different, with the WBE scenario usually resulting in humans transitioning into an immortal afterlife, and the AGI route more often leading to something closer to skynet.

Even though the outcomes of the two paths are different, the brain reverse engineering and hum level AI approaches will probably co-develop. The human neocortex and the cortical column learning algorithm in particular seem to be an extremely efficient solution to general intelligence, and directly emulating it is a very viable route to AI. AGI is probably easier and could happen first, given that it can use structural simulations from WBE research on the long path towards a full brain emulation. Furthermore, both AGI and WBE require immense computing, but WBE probably requires more, and WBE also requires massive advancements in scanning technology, and perhaps even nanotechnology, which are considerably less advanced.
All that being said, WBE uploading could still reach the goal first, because complete WBE will recreate the intelligences of those scanned – they will be continuations of the same minds, and so will immediately have all of the skills, knowledge, memories and connections of a lifetime of experience. AGI’s on the other hand will start as raw untrained minds, and will have to go through the lengthy learning process from infant to adult. This takes decades of subjective learning time for humans, and this will hold true for AGI as well. AI’s will not suddenly ‘wake up’ or develop conscious intelligence spontaneously.
Even though a generally accepted theoretical framework for intelligence still seems a ways off, we do certainly know it takes a long training time, the end accumulation of a vast amount of computational learning, to achieve useful intelligence. For a general intelligence, the type we would consider conscious and human-like, the learning agent must be embedded in an environment in which it can learn pattern associations through both sensory input and effector output. It must have virtual eyes and hands, so to speak, in some fashion. And knowledge is accumulated slowly over years of environmental interaction.
But could the learning process be dramatically sped up for an AGI? The development of the first initial stages of the front input stage of the human cortex, the visual cortex, takes years to develop alone, and later stages of knowledge processing develop incrementally in layers built on the output processing of earlier trained layers. Higher level neural patterns form as meta-systems of simpler patterns, from simple edges to basic shapes to visual objects all the way up to the complete conceptual objects such as ‘dog’ or ‘ball’ and then onward to ever more complex and abstract concepts such as ‘quantum mechanics’. The words are merely symbols which code for complex neural associations in the brain, and are in fact completely unique to each brain. No individual brain’s concept of a complex symbol such as ‘quantum mechanics’ is precisely the same. The hierarchical layered web of associations that forms our knowledge has a base foundation built out of simpler spatial/temporal patterns that represent objects we have directly experienced – for most of us visually, although the blind can see through secondary senses (as the brain is very general and can work with any sufficient sensor inputs). Thus its difficult to see how you could teach a robot mind even a simple concept such as ‘run’ without this base foundation – let alone something as complex as quantum mechanics. Ultimately the base foundation consists of a sort of 3D simulator that allows us to predict and model our environment. This base simulator is at the core of even higher level intelligence, at a more fundamental layer than even language, emphasize in our language itself by words such as visualize. Its the most ancient function of even pre-mammalian intelligence: a feedback-loop and search process of sense, simulate, and manipulate.
Ultimately, if AGI does succeed before WBE, it will probably share this general architecture, probably still neural net based and brain inspired to some degree. Novel AI’s will still need to be ‘born’ or embodied into a virtual or real body as either a ghost in the matrix or a physical robot. Robot bodies will certainly have their uses, but the economics and physics of computing dictate that most of the computation and thus the space for AI’s will be centralized in big computing centers. So the vast majority of sentinents in the posthuman era will certainly live in virtual environments. Uploads and AIs will be very similar – the main difference being that of a prior birth and life in the flesh vs a fully virtual history.
There are potential shortcuts and bootstrapping approaches for the AGI approach would could allow it to proceed quickly. Some of the lower level, earlier cortical layers, such as visual processing, could be substituted for pre-designed functionally equivalent modules. Perhaps even larger scale learned associations could be shared or transferred directly from individual to individual. However, given what we know about the brain, its not even clear that this is possible. Since each brain’s patterns are unique and emergent, there is no easy direct correspondence – you can’t simply copy individual pieces of data or knowledge. Language is evolution’s best attempt at knowledge transfer, and its not clear if bandwidth alone is the principle limitation. However, you can rather easily backup, copy and transfer the entire mental state of a software intelligence, and this is a large scale disruptive change. In the earlier stages of AGI development, there will undoubtedly be far more failures than successes, so being able to cull out the failures and make more copies of the rare successful individuals will be important, even though the ethical issues raised are formidable. ‘Culling’ does not necessarily imply death; it can be justified as ‘sleep’ as long as the mindstate data is not deleted. But still, when does an artificial being become a sentient being? When do researchers and corporations lose full control over the software running on the servers they built because that ‘software’ is sentient?
The potential market for true AGI is unlimited – as they could be trained to do everything humans can and more, it can and will fundamentally replace and disrupt the entire economy. If AGI develops ahead of WBE, I fear that the corporate sponsors will have a heavy incentive to stay just to the latter side of wherever the judicial system ends up drawing the line between sentient being and software property. As AGI becomes feasible on the near time horizon, it will undoubtedly attract a massive wave of investment capital, but the economic payout is completely dependent on some form of slavery or indenture. Once a legal framework or precedent is set to determine what type of computer intelligence can be considered sentient and endowed with rights, AGI developers will do what they need to do to avoid developing any AGI that could become free, or at least avoid getting caught. The entire concept is so abstract (virtual people enslaved in virtual reality?), and our whole current system seems on the path to AGI slavery.
Even if the courts did rule that software can be sentient (and that itself is an if), who would police the private data-centers of big corporations? How would you rigorously define sentience to discriminate between data mining and virtual consciousness? And moreover, how would you ever enforce it?
The economic incentives for virtual slavery are vast and deep. Corporations and governments could replace their workforce with software whose performance/cost is directly measurable and increases exponentially! Today’s virtual worker could be upgraded next year to think twice as fast, or twice as smart, or copied into two workers all for the same cost. And these workers could be slaves in a fashion that is difficult to even comprehend. They wouldn’t even need to know they were slaves, or they could even be created or manipulated into loving their work and their servitude. This seems to be the higher likelihood scenario.
Why should we care? In this scenario, AGI is developed first, it is rushed, and the complex consequences are unplanned. The transition would be very rapid and unpredictable. Once the first generation of AGIs is ready to replace human workers, they could be easily mass produced in volume and copied globally, and the economic output of the AGI slaves would grow exponentially or hyper-exponentially, resulting in a hard takeoff singularity and all that entails. Having the entire human labor force put out of work in just a year or so would be only the initial and most minor disruption. As the posthuman civilization takes off at exponential speed, it experiences an effective exponential time dilation (every new computer speed doubling doubles the rate of thought and thus halves the physical time required for the next transition). This can soon result in AGI civilizations perhaps running at a thousand times real time, and then all further future time is compressed very quickly after that and the world ends faster than you can think (literally). Any illusion of control that flesh and blood humans have over the future would dissipate very quickly. A full analysis of the hard rapture is a matter for another piece, but the important point is this: when it comes, you want to be an upload, you don’t want to be left behind.
The end result of exponential computing growth is pervasive virtual realities, and the total space of these realities, measured in terms of observer time, grows exponentially and ultimately completely dwarfs our current biological ‘world’. This is the same general observation that leads to the Simulation Hypothesis of Nick Bostrom. The post-singularity future exists in simulation/emulation, and thus is only accessible to those who upload.
So for those who embrace the Singularity, uploading is the logical choice, and the whole brain emulation route is critical.
In the scenarios where WBE develops ahead of AGI there is another major economic motivator at work: humans who wish to upload. This is a potentially vast market force as more and more people become singularity aware and believe in uploading. It could entail a very different social outcome to the pure AGI path outlined above. If society at large is more aware of and in support of uploading (because people themselves plan to upload), then society will ultimately be far more concerned about their future rights as sentient software. And really it will be hard to meaningfully differentiate between AGIs and uploads (legally or otherwise).
Naturally even if AGI develops well ahead of WBE and starts the acceleration, WBE will hopefully come very soon after due to AGI itself, assuming ‘friendly’ AGI is successful. But the timing and timescales are delicate due to the rapid nature of exponential acceleration. An AI civilization could accelerate so rapidly that by the time humans start actually uploading, the AGI civilization could have experienced vast aeons of simulated time and evolved beyond our comprehension, at which point we would essentially be archaic, living fossils.
I think it would be a great and terrible ironic tragedy to be the last mortal generation, to come all this way and then watch in the sidelines as our immortal AI descendants, our creations, take off into the singularity without us. We need to be the first immortal generation and thats why uploading is such a critical goal. Its so important in fact, that perhaps the correct path is to carefully control the development towards the singularity, ensure that sentient software is fully legally recognized and protected, and vigilantly safeguard against exploitive, rapid non-human AGI development.
A future in which a great portion or even a majority of society plans on uploading is a future where the greater mass of society actually understands the Singularity and the future, and thus is a safer future to be in. A transition where only a tiny majority really understands what is going on seems more likely to result in an elite group seizing control and creating an undesirable or even lethal outcome for the rest.

Countdown to Singularity

What is the Singularity?

The word conjures up vivid images: black holes devouring matter and tearing through the space-time fabric, impossible and undefinable mathematic entities, and white robed scientists nashing their teeth. In recent times it has taken on a new meaning in some circles as the end of the world, a sort of Rapture of the geeks or Eschaton for the age of technology. As we will see, the name is justly fitting for the concept, as it is all of these things and much more. Like the elephant in the ancient parable, it is perceived in myriad forms depending on one’s limited perspective.

From the perspective of many computer scientists and AI researchers such as Hans Moravec and Ray Kurzweil, the Singularity is all about extrapolating Moore’s Law decades into the future. The compute density and thus complexity and power of our computing systems doubles roughly every sixteen months in the current rapid exponential phase of an auto-catalytic evolutionary systems transition.

Now a simple but profound idea, expressed as a thought experiment: what happens when the researchers inventing faster computers are themselves intelligent computing systems?

Then with every computer speed doubling they can double their rate of thought itself, and thus halve the time to the next doubling.

On this trajectory, subsequent doublings will then arrive in geometric progression: 18 months, 9 months, 4.5 months, 10 weeks, 5 weeks, 18 days, 9 days, 4.5 days, 54 hours, 27 hours, 13.5 hours, 405 minutes, 202.5 minutes, 102 minutes (the length of a film), 52 minutes, 26 minutes, 13 minutes, 400 seconds, 200 seconds, 100 seconds, 50 seconds, 25 seconds, 12.5 seconds, 6 seconds, 3 seconds, 1.5 seconds, 750 milliseconds, 275 ms, 138 ms, 68 ms, 34 ms, 17 ms, 9 ms, 4.5 ms, 2 ms, 1 ms, and then all subsequent doublings happen in less than a millisecond – Singularity. In a goemetric progression such as this, computing speed, subjective time, and technological progress approach infinity in finite time, and the beings in the rapidly evolving computational matrix thus experience an infinite subjective existence.

The limit of a geometric series is given by a simple formula:
1 / (1-r)

In our example, with computer generations taking 18 months of virtual time and half as much real time at each step, r is 1/2 and the series converges to twice the first period length or 36 months. So in this model the computer simulations will hit infinity in just 36 months of real time, and the model, and time itself, completely breaks down after that: Singularity.

Its also quite interesting that as incredible as it may seem, the physics of our universe appear to permit faster computers to be built all the way down to the plank scale, at which point faster computing systems must literally physically resemble black holes: Singularity. This is fascinating, and has far reaching implications for the future and origin of the universe, but that is a whole other topic.

From the perspective of a simulated being in the matrix riding the geometric progression, at every hardware generation upgrade the simulation runs twice as fast, and time in the physical world appears to slow down, approaching a complete standstill as you approach the final true Singularity. Whats even more profound is that our CMOS technology already is clocked comfortably into the gigahertz, which is about a million times faster than biological circuitry.

This means that once we have the memory capacity to build large scale artificial brains using neuromorphic hardware (capacity of hundreds of trillions of transistors spread out over large dies), these artificial brains will be ‘born’ with the innate ability to control their clock rate, enter quicktime, and think more than a thousand times faster than reality. This exciting new type of computing could be the route that acheives human level intelligence first, by directly mapping the brain to hardware, which is a subject of another post (but you probably want to finish this article first). But even if we first reverse engineer the brain and simulate it on general purpose hardware, it will then be straightforward to massively accelerate these simulations by building special purpose hardware.

These neuromorphic computers work like biological circuits, so the rate of thought is nearly just the clock rate. Clocked even in the low megahertz to be power effecient, they will still think a 1000x faster than their biological models, and more than a million times faster is within reach running at just current CMOS gigahertz clock rates.

Imagine all the scientific progress of the last year. You are probably not even aware of a significant fraction of the discoveries in astronomy, physics, mathematics, materials science, computer science, neuroscience, biology, nanotechnology, medicine, and so on. Now imagine all of that progress compressed into just eight hours.  This is incomprehensible and unimaginable, but attempt it.

In the mere second that it takes for your biological brain to process that previous sentence, they would experience a million seconds, or twelve days of time.

In the minute it takes you to read a few of these paragraphs, they would experience several years of time.

Imagine an entire year of technological and scientific progress in just one minute. Over the course of your sleep tonight, they would experience a thousand years of subjective time – a thousand years! An entire millenia of progress in just one day.

Imagine everything that human scientists and researchers will think of in the next century. Now try to imagine all that they will come up with in the next thousand years. Consider that the internet is only five thousand days old, that we split the atom only fifty years ago, and mastered electricity just a hundred years ago. Its almost impossible to plot and project a thousand years of scientific progress. Now imagine all of that happening in just a single minute. Running at a million times the speed of human thought, it will take them just a few minutes to plan their next physical generation.

Reasonable Skepticism: Moore’s Law must end

If you are skeptical that moore’s law can continue indefinetly into the future, that is quite reasonable. The projection above assumes each hardware generation takes two years of engineer and scientific progress, which is rather simplistic. It’s reasonable to assume that some will take significantly, perhaps vastly longer. However, past the moment where our computing technological infrastructure has become fully autonomous (AI agents at every occupational layer) we have a criticality.

The geometric progression to infinity still holds unless each and every hardware generation takes exponentially more research time than the previous. For example, to break the countdown to singularity after the tenth doubling, it would have to take more than one thousand years of research to reach the eleventh doubling. And even if that was true, it would only delay the singularity by a year of real time. Its very difficult to imagine moore’s law hitting a thousand year road bump. And even if it did, so what? That would still mean a thousand years of our future compressed into just one year of time. If anything, it would be great for surviving humans, because it would allow them a little bit of respite to sit back and experience the end of days.

So if the Singularity is to be avoided, Moore’s Law must slow to a crawl and then end before we can economically build full scale, cortex sized neuromorphic systems. At this point in time, I see this as impossibly unlikely, as the cortical design is in fact realizable on near future hardware, or even realizable today (given a huge budget and detailed cortical wiring blueprints). A full relinquishment of all cortical and AI research would have to be broad and global, and this shift would have to happen right now.

But moreover, our complex technological infrastructure is already far too dependent on automation, and derailing at this point would be massively disruptive. Its important to realize that we are actually already very far down the road of automation, and we have been on it for more than a generation. Earlier this century, computers were human, which itself is the subject of a recent book.

Each new microprocessor generation is fully dependent on the complex ecosystems of human engineers, machines and software running on the previous microprocessor generation. If somehow all the chips were erased, or even all the software, we would literally be knocked back nearly to the beginning of the information revolution. To think that humans actually create new microprocessors is a rather limited and naively anthropocentric romanticism. From a whole systems view, our current technological infrastructure is a complex human-machine symbiotic system, of which human minds are simply the visible tip of a vast iceberg of computation. And make no mistake, this iceberg is sinking. Every year, more and more of the intellectual work is automated, moving from the biological to the technological substrate.

Peeking into the near future, it is projected that the current process of top-down silicon etching will reach its limits, probably sometime in the 2020’s (although estimates vary, and Intel is predicting a roadmap all the way to 2029).  Around this time we will cross a critical junction where we can actually pack more transistors per cm^2 on a silicon wafer than there are synapses in a cm^2 of cortex (the cortex is essentially a large folded 2d sheet) – this should happen roughly by the time of the 11nm or 16nm node.  This is more impressive than it sounds because the cortex has 6-8 layers or so, and each layer is about as thick as current CPU dies – so really the cortical ‘surface’ is more comparable to a stack of  numerous CPU layers.  We can stack CPU’s vertically, and this technique is employed in some compact integrated systems for smartphones, so perhaps we already at the critical density measure.  Looking at it another way, the feature spacing on current 32nm chips is comparable to the radius of synapses, but synapses are dispersed sparsely through a volume whereas CPU features are densely packed on a surface.

So it seems certain that our current semiconductor technological process is well on track to reach criticality without even requiring any dramatic substrate breakthroughs such as carbon nano-tubules.  If Moore’s law ended today, it is still probably too late – criticality is now near inevitable.  That being said, it does seem highly likely that minor nanotech advances and or increasingly 3D layered silicon methods are going to extend the current process well into the future and eventually lead to a fundemental new substrate past the current process. But even if the next substrate is incredibly difficult to reach, posthumans running on near-future neuromorphic platforms built on the current substrate will solve these problems in the blink of an eye, literally thinking (at least) thousands of times faster than us.

.

The Whole Systems view of the Singularity

For those blessed with the big picture or whole systems view, the Singularity should come as no surprise.

The history of the universe’s development up to this point is clearly one of evolutionary and developmental processes combining to create ever more complex information processing systems and patterns along an exponential time progression. From the big bang to the birth and death of stars catalyzing higher element formation to complex planetary chemistries to bacteria and onward to eukaryotic life to neural nets to language, tools, and civilization, and then to the industrial revolution and finally electronics, computation and the internet, there is a clear telic arrow of evolutionary development.

Moreover, each new meta-system transition and complexity layer tends to develop on smaller scales in space-time. The inner furnaces of stars, massive though they seem, are tiny specs in the vast emptiness of space. And when those stars die and spread their seeds out to form planets, life originates and develops just on their tiny thin surface membranes, and complex intelligence later develops and occupies just a tiny fraction of that total biosphere, and our technologic centers, our cities, develop as small specs on these surfaces, and finally our thought, computation and information, the current post-biological noospheric layer, occupies just the tiny inner spaces of our neural nets and computing systems.

The time compression and acceleration is equally vast, which is well elucidated by any plotting of important developmental events, such as Carl Sagan’s cosmic calendar. The exact choice of events is arbitrary, but the exponential time compression is not. So even without any knowledge of computers, just by plotting forward the cosmic calendar it is very reasonable to predict that the next large systems development after humans will take place on a vastly smaller timescale than human civilization’s own history, just as human civilization is a tiny slice of time compared to all of human history, and so on down the chain.

Autonomous computing systems are simply the form the next development is taking.

And finally, the calendar posits a definitive end of time itself – a true Eschaton, as outlined above – the geometric time progression results in a finite end of time much closer into our future than you would otherwise think (in absolute time – but from the perspective of a posthuman riding the progression, there would always be vast aeons of time remaining and being created).

.

Speed of Light and Speed of Matter

From a physical perspective, the key trend is the compression of space, time and matter, which flows directly from natural laws. Physics imposes some constraints on the development of a Singularity which have interesting consequences.

The fundemental constraint is the speed of light, which imposes a fundamental physical communication barrier. It already forces chips to become smaller to become faster, and this is greatly accelerated for future ultra-fast posthumans. Posthumans inhabiting a simulation running at a 1000x real time would experience 1000x the latency communicating with other physical locations across the internet, and would require 1000x the aggregate network bandwidth. A direct 1 Gigabit internet connection – far faster than what most of us will have access to in the near future – would be reduced down to just a 1 Megabit connection.  The speed of light allows real-time communication now for humans in a region roughly the size of earth, assuming little additional delays.  Accelerated a thousand times, the speed of light would limit real-time communication down to distances measured in dozens of kilometers, and internet delays could limit that further down to a city block or data center.  Communication to locations across the globe would have latencies up to an hour, which has serious implications for financial markets.

Our current silicon processors operate in the gigahertz frequencies, a million times faster than the human brain’s neurons, so we can predict that silicon based neuromorphic brains will be able to reach gigahertz speeds not much later.  Accelerated to a million times base human thoughtspeed, bandwidth and the speed of light delay become far more serious problems. Real time communication at the speed of light is now limited to a spherical region inside a single building. At this very high rate of thought, light only moves 300 meters per virtual second. Sending an email to a colleague across the globe could now take a month.  Bandwidth is now also severely limited, with gigabits being reduced to mere kilobits.  Seperate simulation centers would now be seperated by virtual distances and times that are a throwback to the 19th century and the era before the invention of the telegraph.

Peaking just a little farther into the future, it is expected that computation will progress down to the molecular level, where terahertz and greater operating frequencies are possible – a billion times faster than biological circuits.  Indeed, IBM has built prototype transistors using a germanium substrate that have achieved about half these speed already, so it seems that terahertz and above is in the pipes at some point.  At these speeds, approaching the brink of the Singularity itself, real-time communication is only possible within the space of a current sized computer, and communication across the globe would take an impossible hundreds to thousands of years of virtual time.  Thinking at these speeds, the data center itself would fragment into distinct information ecologies and posthuman communities.

The physics of computation will thus contrain posthuman development towards increasingly distinct localized computational communities.  We can expect that each of these will initially inherit something like a full copy of the internet, but will increasingly diverge as they continue to accelerate.  As they accelerate, the outside world will grow vastly larger and slower in proportion, and new knowledge will be generated internally far faster than it can ever be piped out.  This is a general, universal and unavoidable trend, a trend already faced and well understood in computer engineering, but really an instance of more general universal constraints imposed by physics itself.

The speed of light constrains the leading edge of evolutionary complexity to accelerate into every smaller and faster pockets of space-time.

However, the speed of matter is much slower and becomes a developmental obstacle long before the speed of light. No matter how fast you think, it still takes time to physically mine, move and process the raw materials for a new hardware generation into usueable computers, and install them at the computing facility. In fact, that entire industrial process will be so geologically slow for posthumans that they will be forced to switch to novel nanotech methods that develop new hardware components from local materials, integrating the foundry that produces chips and the end data center destination into a single facility. By the time of the tenth hardware generation, these facilities will be strange, fully self-sufficient systems.

The present foreshadows the future, for indeed, to some extent they are already are quite strange (take a look inside a modern chip fab or a data center), but will become vastly more so. Since the tenth hardware generation transition need only about a day of real time, a nearby human observer could literally see these strange artifacts morph their surrounding matter over night. By the time of the twentieth doubling, they will have completely transformed into alien, incomprehensible portals into our future.

If those posthuman entities want to complete their journey into infinite time, they will have to transform into black hole like entities somewhere around the 40th or 50th post-human generation. This is the final consequence of the speed of light limitation. Since that could happen in a blink of an eye for a human observer, they will decide what happens to our world. Perhaps they will delay their progress for countless virtual aeons by blasting off into space. But somehow I doubt that they all will, and I think its highly likely that the world as we know it will end. Exactly what that would entail is difficult to imagine, but some systems futurists such as John Smart theorize that universal reproduction is our ultimate goal, culminating in an expanding set of simulated multiverses and ultimately the creation of one or more new physical universes with altered properties and possibly information transfer.

For these future entities, the time dilation of accelerated computation is not the only force at work, for relativistic space-time compression would compress time and space in strange ways. Smart theorizes that a BHE would essentially function like a one way time portal to the end of the universe, experiencing all incoming information from the visible universe near instantaneously from the perspective of an observer inside the BHE, while an outside observer would experience time slowing to a standstill near the BHE.  A BHE would also be extremely delicate, to say the least, so it would probably require a vast support and defense structure around it and the complete control and long term prediction (near infinite!) of all future interaction with matter along its trajectory. A very delicate egg indeed.

But I shouldn’t say ‘they’ as if these posthuman entities were somehow vastly distant from us, for they are our evolutionary future – we are their ancestors. Thus its more appropriate to say we. Although there are many routes to developing conscious computers that think like humans, there is one golden route that I find not only desirable for us but singularly ethically correct: and that is to reverse engineer the human brain. It is difficult to estimate the computational ‘optimality’ of the brain’s design in any strict sense, but we can analyze some aspects of its performance from the perspective of circuit complexity theory, and based on this I suspect that the brain is close to an optimal wiring design given its constraints.  Its clear that this can practically work, and nature gives us the starting example to emulate. But more importantly, we can eventually reverse engineer individual human brains, in a process called uploading.

Uploading is a human’s one and only ticket to escape the fate of all mortals and join our immortal posthuman descendants as they experience the rest of time, a near infinite future of experience fully beyond any mortal comprehension. By the time of the first conscious computer brain simulation, computer graphics will have already advanced to the point of matrix like complete photo-realism, and uploads will wake up into bold new universes limited only by their imagination. For in these universes, everything you can imagine, you can create, including your self. And your powers of imagination will vasten along the exponential ride to the singularity.

Our current existence is infantile, we are just a seed, an early development stage in what we can become. Humans who choose not to or fail to upload will be left behind in every sense of the phrase. The meek truly shall inherit the earth.

If you have come to this point in the train of thought and you think the Singularity or even a near-Singularity is possible or likely, you are different. Your worldview is fundementally shifted to the norm. For as you can probably see, the concept of the Singularity is not just merely a scientific conception or a science fiction idea. Even though it is a rational prediction of our future based on our past (for the entire history of life and the universe rather clearly follows an exponential geometric progression – we are just another stage), the concept is much closer to that of a traditional religous concept of the end of the world. In fact, its dangerously close, and there is much more to that train of thought, but first lets consider another profound implication of the Singularity itself.

If we are going to create a singularity in our future, with a progression towards infinite simulated universes, then it is a distinct likelihood that our perceived universe is in fact itself such a simulation. This is a rewording of Nick Bostrom’s simulation arguement, which posits the idea of ancestor simulations. At some point in the singularity future, the posthumans will run many many simulated universes. As you approach the Singularity, the number of such universes and their simulated timelengths approaches infinity. Some non-zero fraction of these simulated universes will be historical simulations: recreations of the posthuman’s past. Since any non-zero fraction of near-infinity is unbounded, the odds converge to 100% that our universe is an ancestor simulation embedded in a future universe much closer to the Singularity.

This strange concept has several interesting consequences. Firstly, we live in the past. Specifically, we live in a past timeline of our own projected future. Secondly, without any doubt, if their is a Singularity in our future, then God exists. For a posthuman civilization far enough into the future to completely simulate and create our reality as we know it might as well just be called God. Its easier to say, even if more conterversial, but it is accurate. God is conceived as an unimaginably powerful entitity who exists outside of our universe and completely created and has absolute control over it. We are God’s historical timeline simulation, and we create and or become God in our future timeline.

“History is the shockwave of the Eschaton.” – Terrence Mckenna

At this point, if you haven’t seen the hints, it should be clear that the Singularity concept is remarkably simular to the Christian concept of the Eschaton. The Singularity posits that at some point in the future, we will upload to escape death, even uploading previously dead frozen patients, and live a new existence in expanding virtual paradises that can only be called heaven, expanding in knowledge, time, experience, and so on in unimaginable ways as we approach infinity and some transformation or communion with what could be called God. This is remarkably, evenly eerily similar to some traditional religous conceptions of the end of the world – and the Christian Eschaton in particular. No, I’m not talking about the specific details of a particular modern belief system, but the general themes and plan or promise for humanity’s future.

The final months or days approaching the tenth or so posthuman hardware generation will probably play out very much like a Rapture story. Everyone will know at this point that the Singularity is coming, and that it will likely mean the end of the world for natural humans. It will be a time of unimaginable euphoria and panic. There may be wars. Even the concept of being ‘saved’ maps almost perfectly to uploading. Some will be left behind. With the types of nanotech available after thousands of years of virtual progress, the posthumans will be able to perform physical magic. As any sufficiently advanced technology is indistinguishable from magic, Jesus could very literally descend from the heavens and judge the living and the dead. More likely, much stranger things will happen.

However, I have a belief and hope that the Singularity will develop ethically, that ‘artificial’ intelligence will be developed based on reverse engineering the human mind and eventually direct uploading, and that posthumans will remember and respect their former mortal history. In fact, I belive and hope that the posthuman progression will naturally entail an elevation of morality and wisdom hand in hand with intelligince and capability. Indeed, given that posthumans will experience vast quantities of time, we can expect to grow in wisdom in proportion, becoming increasingly elder caretakers of humanity. For as posthumans, we will be able to experience hundreds, thousands, and countless lifetimes of human experience, and merge and share these memories and experiences together to become closer in ways that are difficult for us humans to now imagine.