From MCP to Code Execution
Rethinking model tooling: flexibility, cost and security
MCP (Model Context Protocol) has been created by Anthropic to provide a way for AI models to call external tools/APIs in a standardized way.
A lot of communities and companies have created their implementations of MCP for their frameworks, products and services.
It is really great to try to make AI models more useful and let them do more.
But now we end up glue MCP servers to API endpoints of various services, databases or file storages. In other words, we create the next set of wrappers for our APIs, which wrap our business cases.
The problems with MCP
Non-deterministic nature of LLMs
The LLMs are already capable to calling external tools if they are properly prompted. However, their non-deterministic and probabilistic nature makes them hard to rely on in productions.
There are sill occasions, when you directly prompt the model to call an external tool, and it just does not work as expected.
Moreover, if you have so many tools at your disposal at your API, the model might get confused about what to call and when. In IDEs, you can sometimes be explicit about what tool to use, but in chatbots or other applications, you want the model to decide that on its own.
Context and Token Heaviness
Another problem is the token heaviness of the current implementations of MCP.
Every time, you want to call an external tool, the descriptions of the tools need to be included in the context of the model.
If you suddenly need to transfer large chunks of database or files, suddenly the context gets really heavy and processing eats up your tokens. You find yourself in a situation where you need to pay a lot of money for tokens just to call an external tool.
Moreover, it increases the risk of hitting the context window and losing the data you have already in the context.
Code execution as an alternative
If you use any sort of AI for your coding, you might have experience that models have become quite good at generating code snippets that do what you want (not great, but decent enough).
This opens up an interesting alternative to MCP - code execution.
Instead of defining a set of tools that the model can call, you can just let the model generate code that does what you want, and then execute that code in a safe environment.
This approach has advantages:
- Flexibility: The model can generate code that is tailored to the specific task at hand.
- Cost-effectiveness: You do not need to pay for tokens to include tool descriptions in the context or all the data you want to process. Everything is done in the code execution environment, which cost you compute power, but not tokens.
- Privacy: You can execute the code in a secure environment, without exposing sensitive data to external tools.
You can have descriptions of API endpoints, database schemas or file structures in the prompt, and let the model generate code that interacts with them and gets the job done.
The model generates code that does what you want, and you execute that code in a safe environment.
Safe Environment Challenge
The main challenge with this approach is to ensure that the code generated by the model is safe to execute.
You need to have a sandboxed environment, where the code can be executed without risking the security of your system.
The main flaw of the LLMs is that you can inject a malicious prompt that can lead to generation of harmful code. You can leak sensitive data, delete files or even take over the system.
The last thing you want is rough code generated by the model deleting your database or sending API keys to an external server.
Sturdy constraints and monitoring need to be in place to prevent such scenarios.
What you can do:
- limit internet connection to verified endpoints
- limit file system access to specific directories
- human intervention and code review before execution
- validation of the solutions generated by the model
Luckily, some providers offer such environments as a service, so you do not need to build it on your own.
For example, Daytona, e2b and others.
They take care of most of the security aspects, so you can focus on building your application.
Sandboxing and security is a complex topic, and you need to carefully evaluate the risks and benefits of this approach for your specific use case.
Ideas for the future
The MCP is a great step towards making AI models more useful and capable of calling external tools.
However, I believe that code execution is a more flexible and cost-effective approach to achieve the same goal.
With code execution, you can create custom solutions that are tailored to your specific needs, without being limited by the predefined set of tools.
Or you can take two API endpoints, describe them to the model, and let it generate code that connects them together, migrate data or do any other operation you need.
If you have agentic applications, you can let the model generate code that decides what to do next based on the current context and state of the application. It can combine data from multiple sources, process it and generate the desired output. Real example here.
The possibilities are endless and I want to give it a try in my future projects. The tooling might probably need to evolve further to make it more accessible and secure, but I believe that code execution is the way to go in the foreseeable future.
Is this the end of MCP?
No, I do not think so. Already existing MCP implementations can coexist with code execution approaches.
You can turn MCP calls into code interfaces that the model can use to generate code. Yes, it will take some context, but all the data processing can be done in the code execution environment.
So, was the MCP mistake? Not at all. I always consider such steps as evolutionary. It is better to start doing something and learn from it than to wait for the perfect solution.
You cannot think about the future, you have to build it step by step.
Resources
Socials
Thanks for reading this article!
For more content like this, follow me here or on X or LinkedIn.
