These are some notes i made that may be of interest. They are based predominantly on a video put out by Ark looking at investment opportunities in AI
The Ark interview with James Wang, referenced earlier (
https://youtu.be/V6WL4X6pmCY) discusses many aspects of AI and gives the impression that Cerebras (James Wang employer) offer a AI tool that is far superior to anything else on the market, much more capable than the GPU systems offered by Nvidia. For context one ought to remember that Nvida has a capitalisation of $708B whereas Cerebras is private.
The primary difference between Nvida and Cerebras regarding AI is that the former is based on its graphical processor chips that are combined together in cluster whereas Cerebras uses one large 300 mm diameter wafer (cut to be square). The advantage of Cerebras is that there is none of the interconnect overhead, but if there are errors the whole wafer has to discarded.
According to Wang the Cerebras system is essentially turnkey. He notes that a user can get a result with a small language model of 1.3 billion parameter in 24 hours, less time than it takes to order a bunch of GPU from Nvidia. For larger models there are extensive overheads with Nivida systems that lead to complications and require a lot of skilled ninja programmers to get around.
There are 3 approaches to AI discussed
1 The Nvidia model, using existing gpu to train general systems and limited by the interconnect architecture
2 Tesla dojo that is tailored entirely to solving self driving with huge vertical integration time scale savings
3 Cerebras general model with essentially turn key approaches that is now available over the net.
As far as I understand it, the Nvdia approach is by far the most common method and currently nvidia looks the most “picks and shovels” way to invest, but in the by and by Cerebras and several other AI business are likely to transition from private to public listings.
The customer references on
http://www.cerebras.net from various users indicate very impressive performance, in some cases far exceeding by factors of multiple hundreds, the performance of super computers.
Wang makes the point that Ai is the first technology that offers far more than is advertised with new applications emerging with use, unlike say an iPhone that can do a number of things very well, but does not extend beyond that.
He also notes that shortly the entire knowledge of humanity will be placed on chips in smart phones such that this edge computing will give everything available on the net without needing an internet connection; even faster than 5G!, but with out current updates.
Wang has no current concerns about the dangers of AI, so long as air gaps are maintained between Ai and critical systems, but should AI become robots there is then more risk.
Several papers are discussed. The development of quantitive understandings of the size of the data base needed to give specific results are described as analogues to the laws of physics and allow the tailoring of systems and compute to specific problems.
Meanwhile Google apparently argue, in a leaked document, that they and openAI are being lapped by open source:
https://www.semianalysis.com/p/google-w ... nd-neitherMy instinct is that this leak must be a spoof given the size of the parameter space needed for these large language models. How could open source do this? But perhaps I am wrong and I welcome correction if so.
In general AI looks to be the most exciting scientific and commercial development ever created, analogous, according to Buffett, to the Manhattan project with far reaching implications. It is fascinating that Buffett has changed from avoiding tech to arguing that Apple (in the last shareholder meeting) is the best investment Berkshire owns:
https://www.cnbc.com/video/2023/05/06/w ... e-own.htmlThings do indeed change.
Regards,