[{"content":"今天参加了SII和SJTU AI 组织的2026年强化学习暑期学校的开课典礼，听Richard Sutton讲了自己关于构造intelligence的一些思考，个人感觉很受启发，将演讲的几个takeaway记录如下：\nIntelligence has nothing to do with the domain knowledge, we need developing the algorithm to acquire the knowledge. Doing resarch from a big world perspective, don\u0026rsquo;t be distracted by applications, optimality, regret or something else non-related Intelligence. Be ambitious, keep eyes on Prize of understanding intelligence. 下午的时候，举行了一个圆桌研讨，Rich和Summer school的同学们进行了一些讨论，可惜本i人实在不争气，没有得到机会。但我大概从其他同学的提问以及Rich的回答里get几个小点：\n关于LLM： I think the path to true AI goes through reinforcement learning and not through large language models. I think large language models are also sort of a big distraction. They do not lead there. Language models, first of all, have no experience, they have no goals, and they have no way of telling right from wrong. So, hear me. Words matter. Okay, ideas matter. Only ideas matter. Words are just labels, not ideas. Words do not matter; only ideas matter. 关于学习的本质 The final result is a policy. Yes. All agents have policies. We need policies because we need to be able to respond quickly. We have to have precomputed ways of behaving, mappings from the situation to the action. It always ends up as a policy. The precise limits of a field can be a bit vague because they change over time. I do think we should try to have a definition. Artificial intelligence is famous for not having a definition of what its topic is. I do not really like that, but it is true. 关于Robot Learning The thing with robots and trial-and-error learning is that today’s robots are very fragile. These are words that we throw around. We use them a lot, but we do not really know what they mean. World model — what does it mean? Does anyone know what it means? Does Yann LeCun know what it means? I do not think so, actually. Anyway, words can mean many things. Words do not matter, but ideas matter. So what is the idea? Watch for this: if you notice many people using some words but they cannot define what they mean, this happens all the time. I think it is kind of obvious that robots need to learn the way babies learn: by trying things, by learning the consequences, by learning the laws of physics — or analogues of the laws of physics — so they can learn what and how they can achieve things, how they can grab an object, throw an object, fall over, walk. They have to learn all these things. But all the companies are showing and training the robots from human-collected data. I think this is exactly the wrong way. 对一些研究的吐槽 It should not involve deep learning. It should not involve replay buffers. I will tell you some more things: it should not involve stored data. All the data should be learned from online in some way.（关于project） I like neural networks. I have nothing against neural networks or the idea of learning in multilayer networks. I think that is actually the answer. But the current algorithms are not good at that. They have been made to work by using replay buffers and all kinds of hacky things. The state of the art is that we do not yet have good nonlinear learning methods. I hate those words. Test-time learning — what does this mean? What a strange thing to say. There is no training and testing, just learning. 具体到本次讲座的重要内容Alberta Plan, 我感觉 Rich 关于 Agent 架构的设计挺耐人寻味的，我现在对其认识还处于比较浅的层面，有很多问题想要和其进行探讨：\n我整理了如下两个问题，希望在接下来的时间里能够有机会跟Rich简单聊一下：\nWhat is the most important design among these four parts? Without which, the agent\u0026rsquo;s decision making performance will be affected mostly? How could these four parts cooperate in a correct way, is there any specific design to ensure that? 今天，还认识几位来自ZJU，SII，HKUST的几位朋友，期待在后续的交流中获得一些新的insight。最后show一下和Rich的合照，这或许是我人生中距离图灵奖最近的一次了，hhhhhh~\n","date":"2026-07-07","externalUrl":null,"permalink":"/docs/2026-rl-summer-school-summary-day-1/","section":"文档","summary":"","title":"2026 Rl Summer School Summary: Day 1","type":"docs"},{"content":"","date":"2026-07-07","externalUrl":null,"permalink":"/tags/reinforcement-learning/","section":"标签","summary":"","title":"Reinforcement Learning","type":"tags"},{"content":"","date":"2026-07-07","externalUrl":null,"permalink":"/tags/","section":"标签","summary":"","title":"标签","type":"tags"},{"content":"欢迎来到我的个人空间。这里记录文档、笔记和一些持续更新的想法。\n","date":"2026-07-07","externalUrl":null,"permalink":"/","section":"首页","summary":"","title":"首页","type":"page"},{"content":"","date":"2026-07-07","externalUrl":null,"permalink":"/docs/","section":"文档","summary":"","title":"文档","type":"docs"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"这里是 JulyThirteenth 的个人站点，用来沉淀文档、项目记录和日常思考。\n","externalUrl":null,"permalink":"/about/","section":"关于","summary":"","title":"关于","type":"about"}]