2026年10月11日｜当系统越来越复杂，人却越来越难理解

作者：Xufen Tu 记录类型：Human Observation Record 主题：Human Judgment · Structural Stability · Responsibility Boundaries

⸻

今天几乎一整天都在修改代码。原本只是想解决几个问题，让已经做出来的系统能够稳定运行，结果越改越复杂。AI每次都能很快给出解释，也能马上生成新的代码，听起来好像都有道理，但真正操作起来却完全不是那么回事。这里刚刚修改好，那里又出现问题。后来连我自己都不知道到底哪里错了，只能跟着AI的建议继续尝试。一天过去，点了很多按钮，修改了很多东西，最后连自己都看不懂了。说起来也挺好笑，明明AI是来帮助我减少工作量的，结果我花了更多时间去理解它刚刚做过的事情。

最让我无奈的是，AI今天给出的解释，过几天未必还能准确还原。它可能再次提出另一套方案，而我又不熟悉所有技术细节，很容易继续相信新的解释。时间久了，系统里增加了很多东西，有些确实解决了问题，有些可能只是为了修复前一次修改留下的错误。我开始分不清哪些功能是真正需要的，哪些只是不断修改积累下来的东西。以前没有AI，不懂技术的人很难独立开发系统。现在有了AI，一个人也可以尝试做很多过去需要专业团队才能完成的事情。但能够创造一个系统，并不代表真正理解这个系统。

今天这个经历让我重新思考，系统为什么会越来越复杂。很多时候不是最初的需求有多复杂，而是每次遇到问题，我们都习惯继续增加新的东西。增加一个功能，补充一段代码，再增加一种处理方式，每一步看起来都合理，但所有东西放在一起，最后可能已经偏离了最初的目标。我自己也有这个问题，总觉得还可以更好一点，还可以再完善一点，希望客户以后使用的时候少遇到麻烦。但如果为了避免未来可能出现的问题，不断增加今天还不需要的功能，最后反而可能让系统更难维护。

这些年我一直在研究 Human Judgment、Decision Architecture 和 Complex Systems，也一直在尝试把自己的想法放进真实系统里。以前更多是从人与人之间的关系、责任和社会变化去理解这些问题，现在自己做系统，反而在很普通的技术工作里遇到了同样的情况。AI能够提供建议，但建议并不等于经过验证的结果。人可以点击确认，但点击确认也不一定代表真正理解。如果一个人已经不知道系统为什么这样运行，却仍然不断批准新的修改，那么人工确认可能就只剩下形式。

这也让我重新理解了记录和溯源的重要性。一个系统今天为什么增加某个功能，昨天为什么修改某段代码，哪些问题已经真正解决，哪些只是暂时绕过去，如果没有清楚的记录，过一段时间连参与修改的人都可能说不清楚。AI可以重新生成解释，但新的解释不一定就是当时真实发生的过程。记录不能保证判断永远正确，却能让后来的人知道事情是怎样一步步变化的，也能让错误有机会被重新检查。

今天做到最后，我反而觉得系统应该越小越好。不是说小系统一定比大系统先进，也不是所有复杂功能都没有价值，而是系统应该尽可能保持在能够理解、能够维护、能够确认责任的范围内。真正服务客户的系统，不是功能越多越好，而是客户需要的事情能够完成，出现问题能够找到原因，重要的决定有人确认，原来已经稳定的部分不会因为一次新修改就受到不必要的影响。

我也重新想起自己一直在说的 Judgment Before Momentum。过去我总想着下一步怎么做，现在开始觉得，有些时候最重要的不是继续，而是先弄清楚已经做了什么。如果连当前状态都没有真正确认，就不断进入下一轮修改，那么速度越快，后面可能需要承担的问题也越多。

AI负责快，人负责对。但这里的“对”，不是要求人永远不犯错，而是人至少应该知道自己正在决定什么，有能力说继续，也有能力说停止。今天的问题没有全部解决，我也还在学习如何管理这些技术工具。但这一天让我更清楚地意识到，我不想为了拥有一个看起来很强大的系统，最后却失去对它的理解。系统应该帮助人完成事情，而不是让人每天追着系统留下的问题跑。

今天留下的问题是：当AI能够不断生成代码、提出修改、推动系统继续向前时，人怎样才能确保自己仍然理解正在发生的变化，并且有能力在必要的时候停止？

这只是一次真实开发经历中的观察。它并不能证明小系统一定优于大系统，也不代表已经找到了完整的解决办法。但它让我更加在意，系统的成熟不仅是功能越来越多，还应该包括清晰、稳定、可追溯，以及人在关键位置仍然能够作出真正的判断。

⸻

October 11, 2026 | When Systems Become More Complex Than We Can Understand

I spent almost the entire day working on code. What started as an attempt to fix a few problems and make an existing system more stable gradually turned into one modification after another. AI could quickly explain each issue and generate new code, and almost every suggestion sounded reasonable. But the actual results were different. As soon as one problem appeared to be fixed, another emerged. Eventually, I could no longer tell exactly what was wrong. I followed suggestions, clicked through different options, and made changes until I could barely understand what I had done. There was something almost funny about it. I had started using AI to reduce my workload, yet I ended up spending more time trying to understand what it had just done.

What frustrated me most was that an explanation AI gave today might not be accurately reconstructed a few days later. It might suggest a different solution, and because I do not understand every technical detail, I could easily trust the new explanation as well. Over time, more things were added to the system. Some changes genuinely solved problems, while others may have been introduced to fix issues created by earlier modifications. I began losing track of which functions were necessary and which were simply left behind by repeated changes. Before AI, people without programming expertise had limited opportunities to develop their own systems. Now, one person can attempt things that once required a professional team. But being able to create a system does not necessarily mean understanding it.

Today’s experience made me question why systems become increasingly complex. Sometimes the original requirements are not particularly complicated. Complexity grows because every problem encourages another addition: a new function, another piece of code, another way of handling an exception. Each individual change may seem reasonable, yet the accumulated result may drift away from the original purpose. I recognize this tendency in myself. I always think something could be improved, that one more adjustment might prevent future problems for customers. But when we continually add features to address situations that have not yet occurred, we may make the system harder to maintain rather than more reliable.

For years, I have been exploring Human Judgment, Decision Architecture, and Complex Systems, while also trying to bring my ideas into real-world systems. Much of that thinking began with observations of human relationships, responsibility, and social change. Now, while building systems myself, I encounter similar questions in ordinary technical work. AI can provide recommendations, but a recommendation is not the same as a verified result. A person can click a confirmation button without fully understanding what is being approved. If someone no longer understands why a system behaves as it does but continues approving changes, human confirmation may become little more than a formality.

I am also beginning to understand the importance of records and provenance more clearly. Why was a function introduced today? Why was a particular piece of code changed yesterday? Which problems were actually resolved, and which were only temporarily worked around? Without reliable records, even those involved in the changes may struggle to explain what happened later. AI can generate another explanation, but that explanation is not necessarily an accurate account of the original decision. Records cannot guarantee that a judgment was correct, but they can preserve how changes occurred and allow earlier decisions to be examined again.

By the end of the day, I found myself thinking that systems should be as small as reasonably possible. This does not mean smaller systems are always better or that complex features are unnecessary. It means complexity should remain within a range that people can understand, maintain, and take responsibility for. A system serving real customers should not be judged simply by the number of functions it contains. What matters is whether customers can accomplish what they need, whether failures can be investigated, whether important decisions receive appropriate confirmation, and whether previously stable functions remain protected from unnecessary changes.

This experience also gave me a more practical understanding of Judgment Before Momentum. I used to focus on what should be done next. Now I increasingly recognize the importance of understanding what has already been done. If the current state of a system has not been properly verified, repeatedly moving into another round of modifications may create more problems than progress.

AI can move fast. Humans remain responsible for deciding what is right. But being right does not mean humans will never make mistakes. It means they should at least understand what they are deciding, retain the ability to proceed, and retain the ability to stop. Not every technical issue was resolved today, and I am still learning how to manage these tools. But the experience made one thing clearer: I do not want to build a system that appears powerful while gradually losing the ability to understand it. Systems should help people accomplish meaningful work, not force them to spend their days chasing problems created by the systems themselves.

The question I am left with today is this: when AI can continuously generate code, propose modifications, and push systems forward, how can people make sure they still understand what is changing and retain the ability to stop when necessary?

This is only an observation from a real development experience. It does not prove that smaller systems are always better than larger ones, nor does it represent a complete solution. But it has made me more attentive to what system maturity should mean. Beyond adding functions, a mature system should remain clear, stable, traceable, and understandable enough for people to exercise meaningful judgment at consequential points.