AI Hooks and Bloat
How to Prevent Bloating Your AI Tools with Too Much Context and Instructions
With my previous article about instructions, I mentioned that it is important to keep the instructions file concise and to the point, because it is loaded during every interaction with the model.
It is a known fact that the more context you give to the model, the more it will lose its sharpness and start to forget important information.
Moreover, if you give the model skills, it does not guarantee that they will improve the final results.
Once again, it depends on the situation, model and the task, but it is important to keep in mind that more context does not always mean better results.
Instructions/Skills Bloat
Even in the Anthropic guide to skills (Source), they mention that it is important to keep the skills concise and to the point, because it is loaded during every interaction with the model.
They recommend to keep the number of skills at max 20-50. Otherwise you are bloating the model with too much context.
However, in terms of writing code, the model already knows how to write code. It sees your codebase and can see all the patterns inside of it.
Do not bloat the context with too much duplicate information.
There are more research papers, where they show that the more skills and knowledge you give to the model, its performance can actually decrease, because it is bloated with too much information and it starts to forget important information.
Here are the sources as here or here.
To summarize it:
- Based on the research above, the cost of inference can go up by 20% and the performance increases only by 4%, which is not a good trade-off.
- Models are good at digging out information—do not give them data that they can easily find out by themselves.
- Explain the bare minimum for the model to understand the context of the codebase, how it is structured, its gimmick, and how to run it.
- When it comes to skills, a skill should contain information that presents a gap in the model’s knowledge—something the model does not have the possibility to know by itself. This includes domain knowledge or specific information about the codebase.
- Smaller models are able to level with larger models if they are given proper skills.
- Human-crafted instructions have better performance than AI-generated instructions.
Definitely, read the research papers above, because they are really interesting and they show that more context does not always mean better results.
Hooks
Hooks are a tiny bit under the radar concept, but they can save you a lot of headaches and steer the model or improve your workflow in some manner.
You can run commands during the session based on the actions of the model.
The best example is when you do not know how to persuade the model to use your tool. For example, you use bun, but it still tries to use npm or yarn because it is more popular. You can create a hook that runs before the tooling and checks if the correct package manager is being used. If not, you can return an error and tell the model to use the correct one. This way you can steer the model to use the correct tools and avoid unnecessary errors without interrupting the session.
Couple of examples, where you can leverage hooks:
- logging of the interactions with the model
- staging/committing/pushing code to the repository
- running tests/linters/formatters at the end of session
- command assertion if correct tooling is used
- running scripts to automate repeated actions
Codex does not support hooks at the moment Github Issue, but you can already use it at Claude Code or Github Copilot.
You can define any cmd script, which will run during the session and it can be used to automate repeated actions.
Your Codebase
Your codebase is the ultimate source of information for the model. If your codebase is a mess, the model will also produce a mess.
Imagine you have a codebase that is all over the place as spaghetti code. One module has dependencies from 10 other modules, and it is not clear where to start.
The model will be confused as a person would be. Of course, it is able to handle it somewhat, and it is cheaper to give it a messy codebase than to write instructions about it, but it is still not ideal.
The ideal codebase has modules that disclose their behavior on the surface. In most cases, it is enough to read only this surface to understand how the module works. This makes the model’s life easier and it can produce better results, and it is also easier for you to understand the codebase and work with it.
Create modules, which disclose their behavior on the surface, so model does not need to dig into the module as whole and can safe its capacity. If you need to change module, it does not need to go anywhere else, but only to the module itself.
Conclusion
The AI gets more and more complicated, because well … it is not AGI. It is still another tool, which needs steering into correct direction.
We are getting better at using it, understanding its limitations and how to leverage it in the best way possible, but it is still a tool, which needs to be used with care and understanding of its limitations.
If it will be AGI, we will not be here talking about any of this.
Let’s see, how far we will get with them.
Socials
Thanks for reading this article!
For more content like this, follow me here or on X or LinkedIn.
