Investing in Computing Architecture – Part II

Part II
Read Part I 

The Graphcore approach Graphcore, a UK based company, which has raised USD 110 million from names like Amadeus, Draper Esprit, Robert Bosch VC or Sequoia, is a flagship company in semiconductor innovation. It covers the whole spectrum from its own graph processor – the so called IPU (Intelligent Processing Unit) – to the software stack – POPLAR: libraries, compiler. With the Graphcore solution both training and inference can be handled at a 10- 100x efficiency compared to other solutions according to the website.

To simplify how a deep neural network works, the Graphcore explanation and visualisation is very helpful because it resembles a human brain, however, the working mechanism is not the same. There are the so called:

• VERTICES, which would be the neurons in a human brain, and these represent the
  probabilities of various features; and the
• EDGES, which would be the equivalent of synapses that describe the causation
  between different features.

These vertices and edges make GRAPHs in a way that edges connect vertices. The extra computational power required is coming from this more complex structure (vs. a scalar CPU or a vector GPU). The below picture depicts this structure: clusters show layers in the neural network. Within clusters (layers) communication is very intense, while it is lighter between layers.

(source: Grapchcore)

Why to bother with all of these details? Because it leads us to the key architecture question, namely, how to build an AI chip? A Google paper estimated the cost of a training hardware to be USD 13 thousand. If the key challenges can be addressed, this cost can be significantly reduced, which would accelerate the prevalence of artificial intelligence.

• Parallelism: graphs are large, there are often thousands or millions of vertices.
• Sparsity: most vertices are connected to only a small subset of the other vertices,
  which has a major impact on memory access patterns and communication patterns
  between processors.
• Low precision: this model requires very high-performance computing on very low-
  precision data.
• Static: graph structures can be regarded as static, which enables the compiler to 1.
  balance computational work across processors; 2. partition and allocate memory; 3.
  schedule message traffic between processors.

Graphcore designed its IPU to provide parallelism along the above aspects. The architecture applies high-performance, independent processors with local memory. Distribution is ensure by the compiler, which divides the main programme into sub-programmes and allocate them to the many processors.

Other approaches
There are other startups that focus on the memory dilemma such as Mythic (USD 55.2 million, USA), Knowm (N/A, USA) or Innogrit (N/A, USA/China). Mythic’s key idea is placing the processors and the memory as close together as possible, so data do not need to be moved around the chip, which saves energy. Their analogue solution is targeting developers, who use edge computing to reduce latency and improve privacy especially, in the inference space.

Some teams try to mimic the brain to ensure high-performance and low-power. Koniku (USD 1.7 million), a neurochip company, aims to use actual biological neurons to e.g. build drones that can “smell” methane leaks in oil refineries. Brainchip (USD 28 million, USA) is a representative of neuromorphic chip design, which is still in its infancy. It states that it does not need large datasets to train neural networks.

REM (USD 2 million, USA) and Alpha ICs (USD 2.5 million, India) apply asynchronous processing compared to 99% of the processors being synchronous. It means that various tasks can run on the processor in their own pace taking exactly the time needed, which again translates into energy savings.

Other futuristic approaches are brought by Lightmatter (USD 11 million, USA) and Lightelligence (USD 10 million, USA), which create photonic chips: “running a beam of light through a gauntlet of tiny, configurable lenses (…) and sensors. By creating and
tracking tiny changes in the phase or path of the light, the solution is found as fast as the light can get from one end of the chip to the other.” This technology is over the R&D phase and it promises an affordable solution for hardcore AI developers.

With the eyes of an investor
As pointed out earlier, the AI chip market is gaining popularity again. Companies like Cambricon Technologies, Cerebras Systems, Graphcore, Horizon Robotics and Wave Computing, which raised USD 100(+) million capital each, are enhancing the credibility of the space. Most of the early-stage companies that are not in stealth mode are based in China or the USA as well as the leading investors in this space. The founders are not freshly graduated innovators, but experts with decades of experience. This is also true for many institutional investors. They typically commit to a deal if they have in-house knowledge on the semiconductor industry. Based on my background research, there are recurring names both among financial investors (e.g. Sequoia, Draper) and more strategic investors (e.g. Intel, Robert Bosch VC, Samsung).

What does this mean to early-stage investors with less relevant legacy? Personally, I am not afraid of the end of Moore’s law. This is a very exciting space with niches still before the hype. Vigilant VCs that are ready to dedicate resources to learn and better understand the space, can probably spot lucrative investment opportunities early on and add value to the business. Several hardware architecture solutions are over the R&D phase, while software layers orchestrating underlying platforms can be a good find.

There is much more to talk about when addressing the computing power problem. I am planning to look more into other topics such as serverless computing in my future blog posts. Meanwhile, if you are a founder or a team member in the space, feel free to reach out to start a conversation and building a community.

By Eva Rez