未来展望 What's Coming Next

讲完了模型是怎么训练出来的、它们究竟是什么,我们来看看一些未来的能力——大致「正在到来」的东西。这一章不是承诺,更像是几个方向上的预览。 Having covered how these models are trained and what they actually are, let's look at some future capabilities — roughly what's 'coming down the pipe.' This chapter isn't a promise; it's more a preview of a few directions.

你会注意到的第一件事,是模型会很快变得多模态(multimodal)。我们前面讲的一切都只关于文本,但很快就会有不只处理文本的 LLM:它们能原生地操作音频(于是能听、能说),也能操作图像(于是能看、能画)。这些能力的开端我们其实已经看到了,而关键在于——它会原生地发生在语言模型内部,从而支持更自然的对话。 The first thing you'll notice is that models will quickly become multimodal. Everything we've covered concerned text, but soon there will be LLMs that handle more than text: they'll operate natively over audio (so they can hear and speak) and over images (so they can see and paint). We're already seeing the beginnings of this, and the key is that it will happen natively inside the language model, enabling more natural conversation.

第二个方向是智能体(agents)。现在大多数用法,是我们把一个个单独的任务「端到盘子上」交给模型:「请帮我解决这个任务。」模型把这个小任务做完,但要把许多任务组织成连贯的执行、去完成一份真正的「工作」,仍然得靠我们自己。今天的模型还达不到那种能力——它们还无法长时间地、连贯且能纠错地把一连串任务串起来。但这一点在持续改善,未来我们会看到所谓 agent:它们随时间执行任务,你来监督、查看它们的工作,它们时不时回来汇报进展。任务不再只占几秒钟的响应,而是几十秒、几分钟甚至几小时。 The second direction is agents. Most usage today hands the model a single task on a silver platter: 'please solve this task for me.' The model does that little task, but stringing many tasks into a coherent execution to perform a real 'job' is still up to us. Today's models aren't capable of that yet — they can't coherently, error-correctingly chain tasks together over long stretches of time. But this keeps improving, and in the future we'll see agents: they perform tasks over time, you supervise and watch their work, and they check back now and then to report progress. Tasks will no longer take just a few seconds of response, but tens of seconds, minutes, or even hours.

但正如我们反复强调的,模型并非万无一失,所以这一切都需要监督。在工厂里,人们会谈论自动化的「人机比」(human-to-robot ratio);在数字世界里,我们大概也会谈论「人–智能体比」(human-to-agent ratio)——人会更多地变成 agent 任务的「监督者」,而不是每一步的执行者。 But as we've stressed repeatedly, models are not infallible, so all of this will require supervision. In factories, people talk about the 'human-to-robot ratio' for automation; in the digital world we'll likely talk about a 'human-to-agent ratio' — humans becoming more the supervisors of agent tasks, rather than the executors of every step.

第三,一切都会变得更普遍、更隐形——直接集成进各种工具里,无处不在。与之相关的,是模型开始能「使用计算机」:替你执行操作。今天的模型大多还不能代你采取行动,但已有早期例子(比如能接管键盘、鼠标、替你点击操作的功能)。这意味着模型不只是回答你,而是真的能在你的环境里动手做事。 Third, everything will become more pervasive and invisible — integrated directly into tools, everywhere. Related to this, models will begin to 'use the computer': taking actions on your behalf. Most models today still can't act for you, but there are early examples (features that can take over the keyboard and mouse and click through actions for you). This means a model that doesn't just answer you, but actually does things inside your environment.

最后是一句关于研究的总评:这个领域还有大量工作可做。一个例子是「测试时训练 / 学习」(test-time training)。回想我们讲过的两大阶段:先是训练阶段,调好参数让模型把任务做好;参数一旦确定就固定下来,部署做推理时模型不再改变,也不会从它正在做的事情里继续学习。它唯一可用的「测试时学习」,就是上下文窗口里的「上下文学习」(in-context learning)——靠那个可动态调整的上下文。 Finally, a general comment on research: there's still a lot of work to do in this field. One example is test-time training / learning. Recall the two big stages we covered: first the training stage, where we tune the parameters so the model does the task well; once the parameters are set, they're fixed, and at deployment / inference the model no longer changes or learns from what it's doing. Its only available 'test-time learning' is in-context learning inside the context window — relying on that dynamically adjustable context.