女士们，先生们，尊敬的教职员工，尊贵的客人，骄傲的父母们，最重要的是，加州理工学院 2024 届的毕业生。
今天对你们来说真的是快乐的一天，你们应该表现的兴奋一些。你们即将从加州理工学院毕业，这是伟大的理查德·费曼、莱纳斯·鲍林，还有一位在我们的行业中对我影响很大的卡弗·米德曾经毕业的学校。
是的，毕业是一件非常重要的事情。今天是充满自豪和喜悦的一天。这对你们所有人来说是一个梦想成真的一天，不只是你们，因为你们的父母和家人为了见证你们达到这个里程碑做出了无数的牺牲。所以，我们应该抓住这个时刻去感谢家人，让他们知道你爱他们。请你不要忘记这一点，因为你不知道你将在家里生活多久。今天你会感恩家人吗？作为父母，我真的很希望我的孩子没有搬出去生活，每天都能看到他们真的是太棒了。但现在他们搬走了，这让我很难过。希望你们可以花一些时间和父母在一起。
你们在这里的经历证明了你的品格、你的决心以及为你的梦想做出牺牲的意愿，你应该感到自豪。
学会做出牺牲、忍受痛苦和苦难，你在生活中需要这些品质。
我们之间有一些交集。首先，英伟达的两位首席科学家都来自加州理工学院。我今天发表演讲的原因之一是因为我们正在招聘。所以我想说的是，英伟达是一家非常伟大的公司。我会是一位非常好的老板，会得到大家喜爱。
来英伟达工作吧，我们都热爱科学和工程。尽管我们年龄相差大约 40 岁，但我们都处于职业生涯的巅峰期。对于所有关注英伟达和我的人来说，你能明白我的意思吗？对你们而言，你们会有很多很多巅峰要去，我只希望今天不是我的巅峰。所以，我一如既往地努力工作，确保我前面还有更多的高峰。
去年，我在台大的毕业典礼上发表演讲，分享了几个有关英伟达的发展故事以及我们学到的可能对毕业生有价值的经验教训。我不得不承认的是，我不喜欢给别人提建议，尤其是给别人的孩子。
因此，我今天的建议将在很大程度上伪装成我喜欢的一些故事和一些我喜欢的生活经历。
我相信我是当今世界上任职时间最长的科技公司 CEO。在过去 31 年的时间里，我没有破产，没有感到厌倦，没有被解雇。
我很庆幸享受了很多生活经历，从一无所有到创造英伟达到今天的成就。回想创建 CUDA 的漫长道路，CUDA 是我们花了 20 多年时间才发明出来的一种技术，它在当今的计算领域掀起了一场革命，正在彻底改变计算。
说到一个被公开取消的世嘉游戏机项目，我知道理查德·费曼对这个事情非常关心并经常提到，知识诚实和谦卑拯救了我们的公司，而撤退，战略性撤退是我们最好的策略之一。
所有这些都是我在毕业典礼上谈到的违反直觉的教训，但我鼓励毕业生参与 AI，这是我们这个时代最重要的技术。关于这一点，我稍后会再多说一些。
你们都了解 AI，很难不沉浸其中，被 AI 包围，并对此进行大量讨论。当然，我希望你们所有人都在使用它，并以令人感到神奇、惊讶的方式来玩弄它。你们要享受它，就必须参与其中，因为它进展得如此之快。AI 是我所知道的唯一一项同时在多个指数上取得进展的技术。技术变化非常非常快，因此，我建议台大的学生们在 AI 道路上不要走路，要跑起来，并参与 AI 革命。
然而一年后，令人难以置信的是发生了如此大的变化。所以今天，我想与你们分享我的观点，从我的角度来看你们即将毕业的一些重要事情。这些都是正在发生的非凡事情，你们应该有直觉理解，因为这对你来说很重要。这将对整个行业产生影响。希望你能抓住眼前的机会。
计算机行业正在从基础设施开始转变，从字面上看就是从螺栓开始，一切都在改变，每个行业也将发生变革。原因很明显，因为今天的计算机是唯一最重要的知识工具，是每个行业的基础。如果我们正在如此深刻地改变计算机，它当然会对每个行业都产生影响。我稍后会谈论这个问题。当你进入 AI 行业时，了解正在发生的事情是很重要的。
现代计算可以追溯到 IBM System 360，这是我学习架构的手册。从那以后，很多关于计算机和架构的文档和描述被提出来。但 System 360 在当时非常重要，该系统的思想、架构和原则至今仍主导着计算机行业。
我是第一代学习标志性教材 Mead–Conway VLSI 芯片设计的工程师之一，它革命性地改变了集成电路设计。正是这本书使我们这一代人能够设计超大规模的芯片，最终开发出 CPU。我不确定这本书现在是否还在这里教授，但它应该被继续使用。
CPU 推动了计算性能的指数级增长，摩尔定律这种性能、令人惊叹的技术进步，推动了信息技术革命。我们所参与的工业革命，我们这一代人所参与的工业革命，见证了世界上前所未见的大规模生产。大规模生产那些看不见、易于复制的东西。软件的大规模生产，催生出了一个价值 3 万亿美元的产业。当我还没毕业时，IT 行业微不足道，通过销售软件赚钱只是幻想。然而今天，软件是 IT 行业生产的最重要的商品、最重要的技术和产品创造之一。然而，NARD 缩放、晶体管制程和指令级并行性都削减了 CPU 的性能。但 CPU 性能提升放缓是在计算需求继续呈指数级增长的时候发生的。
这种需求与计算机能力之间呈指数级增长的差距，如果不加以解决，计算能源消耗和成本膨胀最终将扼杀每一个行业。正如我们所见，算力已经很紧张了。在英伟达推进 CUDA 技术二十年后，英伟达为加速计算提供了一条前进的道路。这是我有资格回来演讲的原因。
在计算通胀发生数十年后，通过将耗时的算法卸载到专门进行并行处理的 GPU 上，加速计算实现了难以置信效果。我们通常能够实现 10 倍、100 倍，有时甚至是 1000 倍的加速，节省了金钱、成本和能源。我们现在加速的应用领域覆盖计算机图形学、射线追踪，当然还有基因测序、科学计算、天文学、量子电路模拟、SQL 数据处理，甚至是 PANDAS、数据科学 。
加速计算已达到一个临界点。这是英伟达对计算机行业的第一大贡献，也是对社会的第一大贡献。如今，它为我们提供了一条永续计算的道路，在计算需求不断增加的情况下，成本、时间、能源方面都将持续下降一百倍。
加速计算的开启，必然会在其他领域引领新发展。起初我们不知道是哪些领域，直到深度学习走进我们的视野。一个全新的计算世界出现了。Geoffrey Hinton、Alex Krzyzewski 和 Ilya Sutskever 使用英伟达 CUDA GPU 训练 AlexNet，赢得了 2012 ImageNet 挑战赛，震惊了计算机视觉界。这是一个重大的时刻，是深度学习的大爆炸，是一个标志着 AI 革命开始的关键时刻。
在 AlexNet 后，我们的决策改变了英伟达，可能还改变了其他的一切。
我们看到了深度学习的潜力，并通过第一性原则，相信通过我们自己对深度学习可扩展性的分析，我们相信这种方法可以学习其他有价值的功能。也许深度学习可能是一个通用的函数学习器。同时，有些问题难以或无法用基本的第一性原则来解释。所以当我们看到这点时，深度学习是一项我们真正需要关注的技术，因为它的局限性可能只受到模型和数据规模的限制。
然而，当时也存在挑战。这是在 2012 年刚过不久时发生的事。我们如何无需建造这些巨大的 GPU 集群，就能探索深度学习的极限？在当时，英伟达是一家很小的公司。建造这些巨大的 GPU 集群可能要花费数亿美元。即使我们不这么做，也不能保证我们扩大规模后会有效。然而，没有人知道深度学习能够扩展到多大规模。如果我们不建造它，我们永远不会知道这个问题的答案。我们思考的一点是，如果你建造它，你能探索到深度学习的极限吗？我们的逻辑是，如果我们不建造巨大的 GPU 集群，那种未来永远不会到来。
因此，基于英伟达最初的原则，和当时的分析，我们认为当时已经到达了关键点，建造巨大的 GPU 集群将非常有效。当公司相信这件事情时，我们就应该对此采取行动。
因此，英伟达深耕深度学习，并在接下来的十年中，系统地重新发明了一切。我们重新发明了每一个计算层，从 GPU 开始。后来开发的 GPU，与过去我们开发的 GPU 非常不同。我们继续创设了关于计算的其他方面，包括互连、系统、网络，当然还有软件。
英伟达向未知领域投入了数十亿美元。数千名工程师花了十数年时间，致力于深度学习以及深度学习的拓展和推进。实际上，开始时，我们并不知道这项技术能发展到什么程度。我们就投资了数十亿美元。我们设计并制造了超级计算机，用以探索深度学习和 AI 学习的极限。
然后在 2016 年，我们发布了 DGX-1，英伟达的第一台 AI 超级计算机。我将第一台 DGX-1 交付给了旧金山的一家初创公司。那是一家无人知晓的初创公司，我的一群朋友正在开发人工智能，那家公司叫 OpenAI。
到 2022 年，也就是 AlexNet 诞生 10 年后，计算能力增长了约 100 万倍。想象一下，如果你的笔记本电脑比以前更强大 100 万倍，会变成什么样子？在算力增长了 100 万倍后，OpenAI 推出了 ChatGPT，AI 成为主流。
在这个十年间，Nvidia 从一个许多人最初认识的图形公司，专注于构建 GPU，转变为现在的 AI 公司，构建大规模数据中心级超级计算机。我们彻底改变了我们的公司。同时，我们也彻底改变了计算领域。
当今计算方式已经发生了根本性的变化。现在的计算堆栈使用 GPU 来处理在超级计算机上训练的大型语言模型，而不是使用 CPU 处理程序员编写的指令。我们正在创建人类无法编写的软件。我们正在创建的软件可以做一些十年前人类也无法想象的事情。计算机现在是意图驱动的，而不是指令驱动的。告诉计算机你想要什么，它就会想出办法来实现。像人类一样，AI 应用程序也会理解任务、推理、计划并协调一组大型语言模型来执行任务。未来的应用程序将以非常类似于我们做事的方式执行任务。
组建专家团队，使用工具，进行推理和计划，执行我们的任务软件。软件的能力已经彻底变化，我们的行业也在发生变化和转型，创造了一个前所未有的新行业。
一个新的行业正在我们眼前形成。AI 的输入和输出数据就像是工程师们手中的关键 token。这些浮点数中包含了智能信息 。公司现在正在建立一种新的数据中心，这种数据中心以前不存在。它专门生产智能 token，本质上是 AI 工厂，就像尼古拉？特斯拉过去在工业革命中发明的交流发电机一样。我们现在有了 AI token 生成器，它们将成为新工业革命的工厂。过去有生产能源和电力的大型行业，现在我们有了生产软件这种无形物的大型行业。在不久的将来，我们将拥有生产智能 token 的行业，AI 生成器。一种新的计算模式已经出现，一个新的行业已经形成，这一切都是因为我们从第一性原理出发，形成了对未来的信念，并付诸行动。
人工智能的下一个浪潮是机器人技术。在这个领域，人工智能不仅有语言模型，还有物理世界模型。我们与数百家公司合作制造机器人，包括机器人车辆、取放机械臂、人形机器人，甚至是整个巨大的机器人仓库。但与我们的 AI 工厂战略和经验不同，我们的机器人之旅实际上是通过一系列挫折形成的，而不是通过推理和深思熟虑的行动 。
正如你所知，英伟达发明了（ 首个专为图形处理设计 的）GPU。那是在我们发明人工智能工厂之前。 我们对计算机行业的第一个重大贡献是通过引入可编程着色器，重新定义了计算机图形学 。 我们在 2000 年推出了 GPU 和可编程着色技术 ，希望将 GPU 集成到每台计算机中。因此，我们开始将 GPU 与主板芯片结合，并为 AMD 的 CPU 推出了一款出色的集成图形芯片，我们的芯片组业务一举成功。我记得它几乎在一夜之间从零增长到十亿美元。但突然之间，AMD 想要控制 PC 中的所有技术，而我们希望保持独立。因此他们收购了 ATI，不再需要我们了。于是我们转向英特尔。
这也许不是个好主意，但我们还是求助于英特尔。通过谈判，我们获得了连接英特尔 CPU 的许可。苹果公司对我们的产品非常感兴趣，邀请我们与他们合作开发一款新电脑，这就是第一代 MacBook Air。英特尔看到发生的一切后，决定不再让我们这样做了。于是他们终止了我们的协议。
当我们再次转向时，这次我们获得了 ARM 公司的授权，并开发了低功耗 SoC、移动 SoC，实际上是世界上首个完整的操作计算机的 SoC。非常了不起，我们的芯片被用在谷歌设备中。他们邀请我们合作开发一款新设备，这就是安卓移动设备。然而，高通不希望我们这样做，所以他们不希望我们连接到他们的调制解调器上。而在没有调制解调器的情况下，很难制造移动设备。而且当时也没有其他的 LTE 调制解调器公司，所以我们不得不退出移动设备市场。
这种情况几乎每年都会发生。我们会建造一些东西。它会非常成功，带来数月的兴奋。
一年后，我们被踢出了这些市场。好吧，没有市场可转了， 我们决定进入一个确信没有客户的领域。 因为你可以肯定的是，在没有客户的地方，也就没有竞争对手，没有人会关心你。因此，我们选择了一个没有客户的市场，一个价值为 0 的市场。这就是机器人技术。我们建立了世界上第一个机器人计算机处理系统，投身一种当时无人理解的算法，它被称为深度学习。
这已经是 10 多年前的事了。10 年后的今天，我对我们的成就和创造下一波人工智能浪潮的机会感到无比高兴。更重要的是，我们培养了灵活应变和坚韧不拔的文化。
一次又一次的挫折，我们都克服了，并抓住了下一个机会。每次我们都获得新的技能，增强了我们的品格，也增强了公司的品格。我们的公司很难被分散注意力，也很难被打击。如今，没有任何挫折看起来不像是一个机会。具有讽刺意味的是，我们今天构建的机器人计算机甚至不需要图形处理，这也是我们最初旅程的起点。因此，今天的我们告诉并教会了我们一些东西。正如理查德·费曼所说，世界是不确定的，世界也可能是不公平的，会给你带来严峻的挑战，但要迅速克服它。
除此之外还有另一个机会，让我再告诉你们一个故事。
当我的孩子们还是青少年时，每年夏天我都会在我们的一个国际站点工作一个月。我们在日本度过了一个夏天。在一个周末，我们参观了京都和银阁寺。如果你还没有机会去，你一定要去。它以其精致的苔藓花园而闻名。
我们参观的那天是典型的京都夏日，令人窒息的炎热和潮湿，地面散发出炙热的气息。空气闷热而凝滞，我们和其他游客一样漫步在精心打理的苔藓花园中，我注意到了一位孤独的园丁。
这是银阁寺，苔藓花园十分大，大小与这个庭院差不多，而且它有大量的藏品，几乎涵盖了世界上所有种类的苔藓，而且它被维护得非常精美。
我注意到那个园丁正蹲着，用竹镊子小心翼翼地夹着苔藓，把它放进竹篮里。这里只有这一个园丁。篮子看起来是空的。有那么一会儿，我还以为他在把想象中的苔藓变成一堆想象中的死苔藓。
于是我走上前去问他，你在做什么？他用英语回答说：「我在摘死苔藓，照顾我的花园。」
我则疑惑不解地回应道：「可你的花园这么大。」
园丁则回答道他已经照顾花园 25 年了，他有充足的时间。那是我一生中最深刻的学习经历之一，它真的教会了我一些东西。这个园丁已经把自己奉献给了他的手艺，做着他一生的工作。当你找到了自己可奉献的、值得一生为之努力的事物时，你就有了足够的时间。
我之后开始每天早上都以完全相同的方式，首先做最优先的工作。我有一个非常清晰的优先级列表，在开始工作之前，会先从最高优先级的工作开始。我认为我的一天已经很成功了，因为我已经完成了最重要的工作，可以把整整一天奉献给帮助别人上。当人们为打扰我而道歉时，我总是说我有足够的时间。
2024 届毕业生，我很难想象会有人比你为未来做了更多的准备。你奉献自己，你努力工作。你从世界上最负盛名的学校之一获得了世界一流的教育。当你开始进入下一阶段时，借鉴我的经验，希望它们能帮助你开辟前进的道路。
我希望你相信一些非常规的、未经探索的东西，但你们需要让它是有根据的，是合理且符合理性的，然后全身心投入去实现它。你可能会找到你的「GPU」，你可能会找到你的「CUDA」，你可能会找到你的「生成式 AI」，你可能会找到你的「英伟达」。我希望你能将挫折视为新的机会，你经历的痛苦和苦难会增强你的性格、韧性和敏捷性，它们是我最珍视的终极超能力。而智力并不是这个「能力列表」中最重要的。
我忍受痛苦和折磨的能力，我长时间从事某件事情的能力，我处理挫折的能力，我看到机会的能力，我认为这是我的超能力，我希望它们也是你的。
我希望你能找到一种你可以为之奋斗的「手艺」。第一天就决定它并不重要，甚至短期内决定也不重要。但我希望你们确实找到一个你想毕生致力于完善并磨练技能的手艺，并让它成为你一生的工作。
最后，优先安排你的生活。有很多事情在发生，有很多事情要做，但是请优先安排你的生活，你就会有足够的时间去做重要的事情。
祝贺 2024 届的毕业生们，去追逐你们的梦想吧。
参考链接：
https：//www.reuters.com/markets/us/nvidia-becomes-worlds-most-valuable-company-2024-06-18/
https：//www.youtube.com/watch？v=Sc48ToLIQAY
来源：机器之心 2024-06-19
英文原文：
It really makes me cringe listening to all that.
Thank you for that kind introduction, but I hate hearing about myself.
And just as a show, well, maybe if you could just applaud.
How many of you know who NVIDIA is? And how many of you know what a GPU is? Okay, good, I don’t have to change my speech.
Ladies and gentlemen, President Rosenbaum, esteemed faculty members, distinguished guests, proud parents, and above all, the 2024 graduating class of Caltech.
This is a really happy day for you guys.
You got to look more excited.
You know you’re graduating from Caltech.
This is the school of the great Richard Feynman, Linus Pauling, and someone who’s very influential to me and our industry, Carver Mead.
Yeah, this is a very big deal.
Today is a day of immense pride and joy.
It is a dream come true for all of you, but not just for you because your parents and families have made countless sacrifices to see you reach this milestone.
So let’s take this moment and congratulate them, thank them, and let them know you love them.
You don’t want to forget that because you don’t know how long you’re going to be living at home.
You want to be super grateful today.
As a proud parent, I really loved it when my kids didn’t move out, and it was great to see them every day, but now they’ve moved out, it makes me sad.
So hopefully you guys get to spend some time with your parents.
Your journey here is a testament of your character, determination, willingness to make sacrifices for your dreams, and you should be proud.
The ability to make sacrifices, endure pain and suffering, you will need these qualities in life.
You and I share some things in common.
First, both chief scientists of NVIDIA were from Caltech.
And one of the reasons why I’m giving this speech today is because I’m recruiting.
And so I want to tell you that NVIDIA is a really great company, I’m a very nice boss, universally loved, come work at NVIDIA.
You and I share a passion for science and engineering, and although we’re separated by about 40 years, we are both at the peaks of our career.
For all of you who have been paying attention to NVIDIA and myself, you know what I mean.
It’s just that in your case, you’ll have many, many more peaks to go.
I just hope that today is not my peak.
Not the peak.
And so I’m working as hard as ever to make sure that I have many, many more peaks ahead.
Last year, I was honored to give the commencement address at Taiwan University, and I shared several stories about NVIDIA’s journey and the lessons that we learned that might be valuable for graduates.
I have to admit that I don’t love giving advice, especially to other people’s children.
And so my advice today will largely be disguised in some stories that I’ve enjoyed, and some life experiences that I’ve enjoyed.
I’m the longest running tech CEO in the world today, I believe.
Over the course of 31 years, I managed not to go out of business, not get bored, and not get fired.
And so I have the great privilege of enjoying a lot of life’s experiences, starting from creating NVIDIA, from nothing to what it is today.
And so I spoke about the long road of creating CUDA, a programming model.
The programming model that we dedicated over 20 years to invent, and that is revolutionizing computing today.
I spoke about a very, quite public, canceled Sega game console project we worked on, and where intellectual honesty, something that I know Richard Feynman cares very deeply about and spoke quite often about, where intellectual honesty and humility saved our company.
And how a retreat, a strategic retreat, was one of our best strategies.
All of these are counterintuitive lessons that I spoke about at the commencement.
But I encouraged the graduates to engage with AI, the most consequential technology of our time.
And I’ll speak a little bit more about that later, but all of you know about AI.
It’s hard not to be immersed in it and surrounded by it, and a great deal of discussion about it.
And of course, I hope that all of you are using it and playing with it, with surprising results and some magical, some disappointing, and some surprising.
But you have to enjoy it, you have to engage it, because it’s advancing so quickly.
It is the only technology that I’ve known that is advancing on multiple exponentials at the same time.
And so the technology is changing very, very quickly.
So I advised the students at the Taiwan University to run, don’t walk, and engage the AI revolution.
And yet, one year later, it’s incredible how much it’s changed.
And so today, what I wanted to do is share with you my perspective, from my vantage point, of some of the important things that are happening that you’re graduating into.
And these are extraordinary things that are happening that you should have an intuitive understanding for, because it’s going to matter to you, it’s going to matter to the industry.
And hopefully, you take advantage of the opportunity ahead of you.
The computer industry is transforming from its foundations, literally from studs.
Everything is changing from studs on up.
And across every layer, and soon, every industry will also be transformed.
And the reason for that is quite obvious, because computers today are the single most important instrument of knowledge.
And it’s foundational to every single industry and every field of science.
If we are transforming the computer so profoundly, it will, of course, have implications in every industry.
And I’ll talk about that in just a little bit.
And as you enter industry, it’s important you know what’s happening.
Modern computing traces back to the IBM System 360.
That was the architecture manual that I learned from.
It’s an architecture manual that you don’t need to learn from.
A lot better documentation and better descriptions of computers and architecture has been presented since.
But the System 360 was incredibly important at its time, and in fact, the basic ideas of the System 360, the architecture of it, the principal ideas and architecture and strategy of the System 360 are still governing the computer industry today.
And it was introduced a year after my birth.
In the 80s, I was among the first generation of VLSI engineers who learned to design chips from Mead and Conway’s landmark textbook.
And I’m not sure if it’s still being taught here.
It should be in the introduction of VLSI systems.
Based on Carver Mead’s pioneering work here at Caltech on chip design methodologies and textbook that revolutionized IC design.
And it enabled our generation to design supergiant chips and ultimately the CPU.
The CPU led to exponential growth in computing.
The performance, the incredible technology advances, that’s called Moore’s Law, fueled the information technology revolution.
The industrial revolution that we are part of, that my generation was part of, saw the mass production of something the world had never seen before.
The mass production of something that was invisible, easy to copy, the mass production of software.
And it led to a $3 trillion industry.
When I sat where you sat, the IT industry was minuscule.
And the concept that you could make money selling software was a fantasy.
And yet today, it’s one of the most important commodities, most important technologies and product creations that our industry produces.
However, the limits of the NARD scaling, of transistor scaling, and instruction level parallelism have slowed CPU performance.
And the slowed CPU performance gains is happening at a time when computing demand continues to grow exponentially.
This exponentially growing gap between demand of computing and the capabilities of computers, if not addressed, computing energy consumption and cost, inflation, would eventually stifle every industry.
We see very clear signs of computing inflation as we speak.
And after two decades of advancing NVIDIA’s CUDA, NVIDIA’s accelerated computing offers a path forward.
That’s the reason why I’m here.
Because finally, the industry realized of the incredible effectiveness of accelerated computing at precisely the time that we’re witnessing computing inflation after several decades.
By offloading time-consuming algorithms to a GPU that specializes in parallel processing, we routinely achieve 10, 100, sometimes 1,000-fold speedups, saving money, cost, and energy.
We now accelerate application domains from computer graphics, ray tracing, of course, to gene sequencing, scientific computing, astronomy, quantum circuit simulations, SQL data processing, and even pandas, data science.
Accelerated computing has reached a tipping point.
That is our first great contribution to the computer industry, our first great contribution to society, accelerated computing.
It now gives us a path forward for sustainable computing where cost will continue to decline as computing requirement continues to grow.
A hundred-fold, a hundred-fold of anything in time or cost or energy savings that accelerated computing opened surely would trigger a new development somewhere else.
We just didn’t know what it was until deep learning came to our consciousness.
A whole new world of computing emerged.
Jeff Hinton, Alex Krzyzewski, and Ilya Sutskever used NVIDIA CUDA GPUs to train AlexNet and shocked the computer vision community by winning the 2012 ImageNet Challenge.
This was the big moment, the big bang of deep learning, a pivotal moment that marked the beginning of the AI revolution.
Our decisions after AlexNet transformed our company is something that’s worth taking note of.
Our decisions after AlexNet transformed our company and likely everything else.
We saw the potential of deep learning and believed, just believed through principle thinking, believed through our own analysis of the scalability of deep learning.
We believed the approach could learn other valuable functions.
That maybe deep learning is a universal function learner and how many problems are difficult or impossible to express using fundamental first principles.
And so when we saw this, when we saw this, we thought this is a technology we really have to pay attention to because its limits are potentially only limited by model and data scale.
However, there were challenges at the time.
This is 2012, shortly after 2012.
How could we explore the limits of deep learning without having to build these massive GPU clusters? At the time we were a rather small company, building these massive GPU clusters could cost hundreds and hundreds of millions of dollars.
And if we didn’t though, there was no assurance that it would be effective if we scaled.
However, no one knew how far deep learning could scale.
And if we didn’t build it, we’d never know.
This is one of those, if you build it, will they come? Our logic is if we don’t build it, they can’t come.
And so we dedicated ourselves based on our first principled beliefs and our analysis.
And we got ourselves to the point where we believed this was going to be so effective and when the company believes something, we should go act on it.
So we dove deep into deep learning, and over the next decade, systematically reinvented everything.
We reinvented every computing layer, starting with the GPU itself.
The invention of the modern GPU, which is very different than the GPU of the past that we invented in the first place, and we went on to invent just about every other aspect of computing, the interconnects, the systems, the networking, and of course, software.
We invested billions.
We invested billions into the unknown.
Thousands of engineers for a decade worked on deep learning and advancing and scaling deep learning without really knowing how far we could really take the technology.
We invested billions.
And we designed and built supercomputers to explore the limits of deep learning and AI.
And in 2016, we announced DGX-1, our first AI supercomputer, and I delivered the first one to a startup in San Francisco, a startup nobody knew anything about, a group of friends of mine who were working on artificial intelligence, a company called OpenAI.
In 2022, 10 years after AlexNet, and about a million-fold increase in computing later, a million-fold.
If you could just imagine, what would it be like if your laptop was a million-fold more capable? A million-fold later, OpenAI launched ChatGPT, and AI went mainstream.
During this decade, NVIDIA transformed ourselves from a graphics company that many of you probably first knew us as, that builds GPUs, to now an AI company that builds massive data center scale supercomputers.
We transformed our company completely.
We also transformed computing completely.
The fundamental way of doing computing today has been radically changed.
The computing stack now uses GPU to process large language models that are trained on supercomputers rather than CPUs that are processing instructions written by programmers.
We are now creating software that no humans can write.
We are now creating software that does things that no humans can imagine, even just 10 years ago.
Computers are now intention-driven rather than instruction-driven.
Tell a computer what you want, and it will figure out how.
And like humans, AI applications will understand the mission, reason, plan, and orchestrate a team of large language models to perform tasks.
Future applications will do and perform very similar to the way we do things, assemble teams of experts, use tools, reason and plan, and execute our mission.
Software and what software can do has been completely changed.
Even our industry, as it’s being changed and transformed, created yet another industry, an industry the world’s never seen before.
An industry is forming right in front of our eyes.
AI’s input and output are tokens.
For all the engineers in the room, you know what I mean.
These are floating-point numbers that embed intelligence.
Companies are now building a new type of data center that didn’t exist before that specialize in producing intelligence tokens.
Essentially, AI factories.
Like AC generators that Nikola Tesla invented of the past industrial revolution, we now have AI token generators, and they will be the factories of a new industrial revolution.
There’s large industries producing energy, electricity.
We now have a large industry producing something invisible called software.
In the future, in the very near future, we’ll have industries that are producing, manufacturing intelligence tokens, AI generators.
A new computing model has emerged, and a new industry has emerged, all because we reasoned from first principles, formed our belief about the future, and we acted on them.
The next wave of AI is robotics, where AI, in addition to a language model, also has a physical world model.
We work with hundreds of companies building robots, robotic vehicles, pick-and-place arms, humanoid robots, and even entire gigantic warehouses that are robotic.
But unlike our AI factory strategy and our experience there, which was really formed through reasoning and deliberate action, our robotics journey resulted from a series of setbacks.
As you know, NVIDIA invented the GPU.
This was before we invented AI factories.
Our first great contribution to the computer industry was reinventing computer graphics through programmable shaders. We invented the GPU and programmable shading in 2000.
We wanted to integrate GPUs into every computer, and so we started to combine our GPUs with motherboard chips, and we launched a fabulous integrated graphics chip at the time for AMD CPUs.
Our chipset business was an instant success.
I think it went from zero to a billion dollars practically overnight.
But then all of a sudden, AMD wanted to control all of the technology in the PC, and we wanted to stay independent, so they purchased ATI and no longer needed us.
We turned to Intel.
That probably wasn’t a great idea, but we turned to Intel and negotiated a license to connect to Intel CPUs.
Apple was excited by what we were building and asked us to work on a new computer with them, which became the first MacBook Air.
Well, Intel saw what happened and decided they didn’t want us to do that anymore, and so they terminated our agreement.
Well, we pivoted again, and this time we went and licensed ARM, and we built a low-power SoC, a mobile SoC, the world’s first SoC that was essentially a computer, a full operating computer, and it was incredible.
Our chip excited Google, and they asked us to work on a new device, which turned out to have been the Android mobile device.
Well, Qualcomm decided they didn’t want us to do that, and so they didn’t want us to connect to their modems, and it’s hard to build a mobile device without being connected to a modem.
And there were no other LTE modem companies, so we had to exit the mobile device market.
Well, this happened practically on a year rhythm, and we would build something, it would be incredibly successful, generate enormous amounts of excitement, and then one year later, we were kicked out of those markets.
Well, with no more markets to turn to, we decided to build something where we are sure there are no customers, because one of the things you can definitely guarantee is where there are no customers, there are also no competitors, and nobody cares about you.
And so we chose a market with no customers, a $0 billion market, and it was robotics.
We built the world’s first robotics computer processing an algorithm nobody understood at the time called deep learning.
This is over 10 years ago now.
Ten years later, I can’t be happier with what we’ve built and the opportunity to create the next wave of AI. More importantly, we developed agility and a culture of resilience.
One setback after another, we shook it off and skated to the next opportunity.
Each time, we gained skills and strengthened our character.
We strengthened our corporate character.
Our company is really hard to distract and really hard to discourage, and no setback that comes our way doesn’t look like an opportunity these days.
Ironically, the robotics computer that we built today doesn’t even need graphics, which is why our journey started in the first place.
So where we are today tells us something and teaches us something.
The world is uncertain, as Richard Feynman would say, and the world can be unfair and deal you with tough cards.
Swiftly, shake it off.
You’ve apparently been paying too much attention to your books.
Swiftly, shake it off.
Come on, that’s pretty clever.
I made myself laugh.
There’s another opportunity out there, or create one.
Let me tell you one more story.
I used to work from one of our international sites for one month each summer.
When our kids were in their teens, we spent a summer in Japan.
For a weekend, we visited Kyoto and the Silver Temple.
If you haven’t had a chance to go, you must.
It’s renowned for its exquisite moss garden.
The day we visited was quintessential Kyoto summer day, suffocatingly hot and humid, sticky.
Heat is radiating from the ground.
The air was thick, still.
Along with the other tourists, we wandered through the meticulously groomed moss garden.
And I noticed the lone gardener.
Now, remember, the moss garden, this is the Silver Temple, the moss garden is gigantic.
It’s about the size of this courtyard.
And it has the collection, the largest collection of just about apparently every species of moss in the world.
And it’s just exquisitely maintained.
I noticed the lone gardener squatting, carefully picking at the moss with a bamboo tweezer and putting it in the bamboo basket.
And you have to, it’s a bamboo tweezer, you know, and it’s just this one gardener.
And the basket looked empty.
Well, for a moment there, I thought he was picking imaginary moss into a pile of imaginary dead moss.
And so I walked up to him and I said, what are you doing? And in his English, he said, I’m picking dead moss.
I’m taking care of my garden.
And I said, but your garden is so big.
And he responded, I have cared for my garden for 25 years.
I have plenty of time.
Well, that was one of the most profound learnings in my life.
And it really taught me something.
This gardener has dedicated himself to his craft and doing his life’s work.
And when you do that, you have plenty of time.
I begin each morning, I do every single morning exactly the same way, I begin each morning by doing my highest priority work first.
I have a very clear priority list and I start from the highest priority work first.
Before I even get to work, my day is already a success.
I’ve already completed my most important work and can dedicate my day to helping others.
And when people apologize for interrupting me, I always say I have plenty of time and I do.
Graduates of the class of 2024, I can hardly imagine anyone more prepared for the future than you.
You dedicated yourself, you worked hard, you earned a world-class education from one of the most prestigious schools in the world.
And as you commence into the next stage, take my learnings and hopefully they’ll help you along the way.
I hope you believe in something, something unconventional, something unexplored, but let it be informed and let it be reasoned.
Then dedicate yourself to making it happen.
You may find your GPU, you may find your CUDA, you may find your generative AI, you may find your NVIDIA.
I hope you will see setbacks as new opportunities.
Your pain and suffering will strengthen your character, your resilience and agility, and they are the ultimate superpowers.
Of all of the things that I value most about my abilities, intelligence is not top of that list.
My ability to endure pain and suffering, my ability to work on something for a very, very long period of time, my ability to handle setbacks and see the opportunity just around the corner I consider to be my superpowers, and I hope they’re yours.
And I hope you find a craft.
I hope you find a craft.
It’s not important to decide on day one, it’s not even important to decide any time soon, but I hope you do find a craft, that you want to dedicate your lifetime to perfecting, to hone the skills of, and let it be your life’s work.
And then lastly, prioritize your life.
There’s so many things going on, there’s so many things to do, but prioritize your life and you will have plenty of time to do the important things.
Congratulations, class of 2024, go get them.