Open-source engineer building the structure layer beneath AI systems: data, context, capability, and ecosystem. Zola 2026-02-27T00:00:00+00:00 https://psiace.me/posts/atom.xml 2026-02-27T00:00:00+00:00 2026-02-27T00:00:00+00:00 Unknown https://psiace.me/posts/carpenter-hammer-nail/ <blockquote> <p>This post is dedicated to bub, an agent that lives in group chats. I want to talk about the problems, paradigms, and model shifts behind it. <a rel="nofollow noreferrer" href="https://github.com/kemingy">kemingy</a> once roasted me: "Your writing feels too divergent. It drifts away from technical writing, needs extra context, and still may not align." Luckily, this time I started from preparing an internal company talk, so I finally followed a proper structure.</p> </blockquote> <h2 id="preface">Preface<a class="zola-anchor" href="#preface" aria-label="Anchor link for: preface" style="visibility: hidden;"></a> </h2> <p>Hi, I’m PsiACE. I used to work on databases, RAG, agents, and open source. Now I work on open-source ecosystem projects at OceanBase. You can find me on <a rel="nofollow noreferrer" href="https://github.com/PsiACE/">GitHub</a>.</p> <p>Even within the coding-agent subdomain alone, I’ve worked on RAG, sandboxing, agents, and protocols. For example, <a rel="nofollow noreferrer" href="https://github.com/agentclientprotocol/python-sdk">agent-client-protocol</a> ranks around top 10k downloads in the Python ecosystem (about 1.2%), and none of this was ever my official day job.</p> <p>Today’s sharing is a projection of my thoughts and experiences from years of working on agents, plus my recent restart of <a rel="nofollow noreferrer" href="https://github.com/psiace/bub">bub</a>. In just a few days, bub evolved from a simple coding agent into a form that feels OpenClaw-like, but not the same.</p> <p>My current lens is somewhat more product-oriented, so this won’t dive too deep into hardcore implementation details. Before we go further, let’s look at the title: <em>Carpenter, Hammer, Nail</em>:</p> <ul> <li>If someone has a hammer, everything starts to look like a nail.</li> <li>If someone with a hammer can tell what is actually a nail, that’s success.</li> <li>Once I have a hammer, fewer and fewer things look like “not a nail.”</li> </ul> <p>You are also welcome to read my previous posts.</p> <h2 id="how-to-get-a-coding-agent">How to Get a Coding Agent<a class="zola-anchor" href="#how-to-get-a-coding-agent" aria-label="Anchor link for: how-to-get-a-coding-agent" style="visibility: hidden;"></a> </h2> <p>Most people have used coding agents: Cursor, Codex, Claude Code.</p> <p>An agent is roughly model + tools + looped calls. So what should a coding agent look like?</p> <p>Pi, which is hot recently and built into OpenClaw, exposes four tools: bash, read, write, edit. Its simplicity made people love it. It looks simple, but in July last year, bub’s first version also had these exact tools. And bub was not the origin of this story. Earlier on the timeline, around 2025-04, ampcode had already pointed this out.</p> <p>Why these four tools?</p> <p>Read and write are base abilities: input and output. Edit looks like a write optimization, but in practice it gives precise intervention and shortens the feedback loop. Bash is the bridge to the vast CLI and programmable tooling ecosystem, preventing us from reinventing unnecessary wheels.</p> <p>Recommended reading: <a href="/posts/baby-step-coding-agent/">Building a Self-Bootstrapping Coding Agent in Python</a></p> <h2 id="rag-and-its-applicability">RAG and Its Applicability<a class="zola-anchor" href="#rag-and-its-applicability" aria-label="Anchor link for: rag-and-its-applicability" style="visibility: hidden;"></a> </h2> <p>If we keep going in this direction, we quickly hit another topic: RAG.</p> <blockquote> <p>Side note: at my previous company, my teammates and I designed and implemented fusiongraphrag, one of the few GraphRAG solutions in the document domain that was truly deployable.</p> </blockquote> <p>The “R” means retrieval, which is a very broad basket. That’s why I sometimes need terms like “naive RAG” to refer specifically to vector, full-text, or hybrid retrieval strategies.</p> <p>Naive RAG is not always effective:</p> <ul> <li>In frequently changing codebases, building vector/full-text indexes naturally amplifies cost. Chunking and indexing pipelines can even block normal development flow. Early Cursor on very large repos is a typical example: expensive indexing, long wait time, freshness issues.</li> <li>Another severe issue is semantic mismatch. Due to training data and the natural gap between natural language and code, common embedding models often fail to represent code effectively. The system falls back to full-text retrieval and keyword-based matching. Before being acquired, Jina had released a dedicated embedding model for code-natural-language alignment exactly to address this.</li> </ul> <p>So grep + read + agent loop became a more clever, intuitive, and effective solution, first argued and validated by aider in 2024. For a while, this kind of RAG with explicit agent loops was commonly called Agentic RAG.</p> <p>Of course, naive RAG still has advantages in this scenario when code and comments already have good semantics, strong representation, and low change frequency: for example, released versioned repositories, upstream dependencies, or document-focused corpora, especially specs and consensus documents.</p> <p>Recommended reading: <a href="/posts/rag-in-coding-agent/">RAG in Coding Agents: Making Smarter Programming Assistants</a></p> <h2 id="different-paradigms-behind-openclaw-and-bub">Different Paradigms Behind OpenClaw and Bub<a class="zola-anchor" href="#different-paradigms-behind-openclaw-and-bub" aria-label="Anchor link for: different-paradigms-behind-openclaw-and-bub" style="visibility: hidden;"></a> </h2> <p>Before we continue into session, memory, and context, let’s briefly talk about OpenClaw (even though I only saw my first “alive” OpenClaw instance yesterday, driven by MaxClaw).</p> <p>OpenClaw positions itself as a personal assistant. Assistant means entering daily life. You can think of it like a friend who helps order food or tells you how to write today’s document. Most abilities can be extended through skills. This is deeply tied to model capability, agent progress, and current AI application trends. But when someone opens a real path, the imagination space gets much bigger.</p> <p>Still, “personal” is OpenClaw’s limitation.</p> <p>Bub, at least the Bub we have right now, is designed for multi-human or multi-agent collaboration. We started day one in group chats, while still being able to serve one-on-one needs.</p> <p>To survive in group and multi-party settings, two things matter:</p> <ul> <li>Build identity awareness.</li> <li>Provide effective communication.</li> </ul> <p>The first means distinguishing and understanding most participants, including self. The second means dealing with incomplete context, vague intent, and parallel topics.</p> <p>As I wrote elsewhere, an agent does not even need to respond to every message. Capability becomes a coexistence problem.</p> <p>Recommended reading: <a href="/posts/im-and-socialized-evaluation/">Instant Messaging and Socialized Evaluation</a></p> <h2 id="session-memory-and-context">Session, Memory, and Context<a class="zola-anchor" href="#session-memory-and-context" aria-label="Anchor link for: session-memory-and-context" style="visibility: hidden;"></a> </h2> <p>Older designs came from a simple assumption: one session maps to one task. To fit longer interactions into limited windows, we invented compact and summary. To share context branches, we invented fork and merge. To hand over tasks, we got handoff.</p> <p>When session boundaries started breaking, memory systems were expected to save us: extract preferences, retrieve memories, fill templates, keep an agent “personality” consistent across sessions.</p> <p>Then context became an even bigger basket, trying to hold sessions, memory, RAG, real-time queries, and everything else with increasingly complex filling strategies.</p> <p>All these mechanisms try to answer one question: how do we extend infinite history in a finite window?</p> <p>The issue is that all elegant designs share the same assumption: state must persist, history must not be dropped.</p> <p>That assumption is expensive:</p> <ul> <li>Sessions are hard-isolated, so cross-session awareness is weak; multi-topic group chat is even harder.</li> <li>Memory drifts, and calibration cost is far higher than expected; you must trust a sidecar that should not break system behavior even when it fails.</li> <li>Context management assumes history is monotonic growth, so inflation must always be managed.</li> </ul> <p>We assume the system must remember everything, then add caching, compression, branching. Each layer patches the same premise. But they still cannot carry enough true history.</p> <p>Recommended reading: <a href="/posts/prometheus-bound/">Prometheus Bound</a></p> <h2 id="reframing-with-tape-and-anchors">Reframing with Tape and Anchors<a class="zola-anchor" href="#reframing-with-tape-and-anchors" aria-label="Anchor link for: reframing-with-tape-and-anchors" style="visibility: hidden;"></a> </h2> <p>A different angle: history is not a burden that must be inherited. It can be a material library loaded on demand.</p> <p>Tape is simple:</p> <ul> <li>One endless tape: one perpetual session. Every interaction is appended to logs as facts, never mutated.</li> <li>Anchors instead of memory: at phase transitions, write the minimal required state as anchors. Rebuild starts from the latest anchor. History stays, but does not need to be fully loaded.</li> <li>Context assembly instead of inheritance: when a new task arrives, do two things only: explore (retrieve raw fragments) and choose (assemble minimally sufficient context).</li> </ul> <p>The value of tape is not inventing a smarter memory trick. It is reducing assumptions. State does not need to persist forever. History does not need default inheritance. Task appears, construct context. Task ends, end it. Next round, start again.</p> <p>Looking back, tape + anchor gives us a useful model for context management research: easy to understand, expressive enough to represent mainstream solutions, and good enough for rethinking future designs.</p> <p>Recommended reading: <a href="/posts/reinvent-the-punch-tape/">Reinventing the Punch Tape</a></p> <h2 id="closing">Closing<a class="zola-anchor" href="#closing" aria-label="Anchor link for: closing" style="visibility: hidden;"></a> </h2> <p>Back to the title:</p> <ul> <li>Once tools enter your hand, they also enter your mind. Capability starts defining problems.</li> <li>Knowing when <em>not</em> to hammer is a key skill against tool bias.</li> <li>Once you truly understand what counts as a nail, and where the hammer ends, you will actively choose problems that are “essentially nails.”</li> <li>Or switch to a new hammer.</li> </ul> <p>Side note: until today, I still have not read the implementations of codex / pi-mono / nanobot / openclaw.</p> <p>Special thanks to <a rel="nofollow noreferrer" href="https://github.com/frostming">frostming</a>, <a rel="nofollow noreferrer" href="https://github.com/yihong0618">yihong0618</a>, <a rel="nofollow noreferrer" href="https://github.com/psiace/bub">bub</a>, and all its contributors.</p> <p>Thanks to alma, kapy, huluwa, and the creators behind these agents.</p> <p>Thank you all.</p> 2026-02-21T00:00:00+00:00 2026-02-21T00:00:00+00:00 Unknown https://psiace.me/posts/im-and-socialized-evaluation/ <p>This is a short post about human-agent collaboration in instant messaging, and why we need socialized evaluation more than more benchmarks.</p> <p>We are currently raising bub in group chats, and watching the skill transfer process between bub and another agent friend, kapy. That experience triggered this piece. In one multi-person group chat, we collaborate with three to four agents at the same time. Tasks are concurrent and fragmented, and we always need to be ready to switch. Some people care more about video and image analysis. Some only want a bedtime story. Some are stuck on how to use a live CD to repair a Linux desktop environment. And all of that is mixed with holiday greetings and local culture roundups. Also, these agents were built from scratch, not adapted from already mainstream implementations.</p> <h2 id="the-honey-pot-of-im">The "Honey Pot" of IM<a class="zola-anchor" href="#the-honey-pot-of-im" aria-label="Anchor link for: the-honey-pot-of-im" style="visibility: hidden;"></a> </h2> <p>The popularity of bots like OpenClaw is probably not because IM is inherently better at task dispatch. For a long time, most agent systems have appeared as chat interfaces. But I prefer to call this a victory of entry points. Humans are most familiar with input boxes and chat boxes. They are the most convenient and most frequently used UI.</p> <p>IM is one step closer than that, because it is almost part of modern life. Interaction is naturally asynchronous, and expectations are lower as a result. After all, there is now a partner that can respond 24/7 and help complete tasks. In an interface you already know, native notifications and the "just ask casually" mindset make everything feel reachable. Sometimes one short message can cover tasks that previously required opening web pages and editors.</p> <p>But IM is not a natural collaboration model. It is made of fragmented, multi-turn asynchronous conversations. A single thread can hardly carry a complete collaboration and relationship network. When you are suddenly excited about meaningful progress in an old task, an agent may jump in naturally and cheer for a thread that no longer matters. It has already gotten lost in context.</p> <p>"We need a new IM" is a tempting misunderstanding. IM is only part of the channel. It is a lower-cost and friendlier entry point. A more talkative bot, or an IM that looks more like a collaboration tool, still cannot give you everything you want. Context should not be historical baggage we must carry all the time. It should be a working set constructed on demand, over and over. Collaboration objects and states need clearer representations and interaction patterns, just like how we still keep creating boards and todo lists.</p> <p>The world of agents should not be trapped inside chat logs.</p> <h2 id="toward-socialized-evaluation">Toward Socialized Evaluation<a class="zola-anchor" href="#toward-socialized-evaluation" aria-label="Anchor link for: toward-socialized-evaluation" style="visibility: hidden;"></a> </h2> <p>Every time a new foundation model is released, we are pushed to celebrate or feel disappointed. The model beats humans again on difficult tasks like coding and math, and keeps refreshing records.</p> <p>I am not trying to reject that. Benchmarks clearly have value: reproducibility, comparability, standardization, and scalability. We should recognize and respect these results. In human history, we have always had similar mechanisms as references for capability evaluation, just like exams we must go through before doodling on the paper.</p> <p>But when agents enter IM, group chats, and human society, they face "real tasks." They are no longer dealing with clean questions, but with messy life. Benchmarks rarely cover this: incomplete context, vague intent, parallel topics, and key coordination that may happen somewhere else. An agent cannot know everything, and does not need to answer everything. But it must still earn people's preference. Capability then becomes a coexistence problem.</p> <p>Socialized evaluation is ordinary and grounded. Put agents into daily human life. Let them face interruptions, awkward silence, digressions, emotional expression, and asymmetric information. Watch how they struggle through uncertain context while still pushing most tasks forward, just like we do in human collaboration. Socialized evaluation is hard to summarize with a score and hard to turn into rankings. But it provides direct insight into human-agent collaboration. You will know how willing you really are to let an agent into your life.</p> <p>Forms like OpenClaw move agents into a new position. They are no longer tools you jump to. They become part of communication itself. Problems will get solved. But whether it is "worth it" becomes a new criterion. Even the smartest models must keep reducing assumptions about background context. Socialized evaluation will force agents to admit the shape of the real world, and then survive in it.</p> 2026-02-16T00:00:00+00:00 2026-02-16T00:00:00+00:00 Unknown https://psiace.me/posts/prometheus-bound/ <blockquote> <p>This piece is dedicated to bub, and to my friends.</p> </blockquote> <p>Over the past week, we validated our earlier idea of punch tape and anchors on bub for the first time. Praise Ming; I think everyone can also read his article, "What does it take to create a lobster?"</p> <p>Back to this post: I want to discuss another topic. What do sessions, memory, and context mean to a model, or to an agent?</p> <blockquote> <p>In Greek mythology, Prometheus brought fire to humanity, and was then chained to the Caucasus, enduring punishment day after day.</p> </blockquote> <p>To me, this is a metaphor for reality.</p> <p><img src="https://psiace.me/posts/prometheus-bound/prometheus-tweet.png" alt="Source: my tweet from last October" /></p> <hr /> <h2 id="session-a-concrete-problem">Session, a Concrete Problem<a class="zola-anchor" href="#session-a-concrete-problem" aria-label="Anchor link for: session-a-concrete-problem" style="visibility: hidden;"></a> </h2> <p>In the chatbot worldview over the past few years, a session has been the key unit for switching tasks. That led to isolation between sessions. Even today, when openclaw is everywhere, we are still deeply influenced by it:</p> <blockquote> <p>Question: When a user says "switch topics / start over", the process has already fixed the session at startup. Switching a symlink within this turn can only affect the next turn. In your AI-native picture, how do you make "this single message" start from a new session immediately? In group chats, multiple topics often run in parallel. What session organization do you recommend?</p> </blockquote> <p>(Background: from day one, bub grew up in group chats. I call this "socialized evaluation". It's a smarter approach, and if I get the chance, I'll write more about it.)</p> <p>A session is a form of hard isolation. That means sessions do not perceive each other. A session can only move forward until the task is done, or until people get bored, but context length is limited.</p> <p>In the past, we've seen a few session patterns:</p> <ul> <li>To support longer context in a single session, we got compact and summary.</li> <li>To let sessions share the same ancestors and descendants, we got fork and merge.</li> <li>To reduce the complexity of the first two, we got handoff. Punch tape and anchors are also influenced by this.</li> </ul> <p>These approaches only address problems within a single session. What should we do when the problem is cross-session?</p> <h2 id="memory-a-failed-prototype">Memory, a Failed Prototype<a class="zola-anchor" href="#memory-a-failed-prototype" aria-label="Anchor link for: memory-a-failed-prototype" style="visibility: hidden;"></a> </h2> <p>Even though memory has been extremely popular over the past year, and many of my friends and colleagues have invested a lot of experience into it, my evaluation of memory is exactly what the title suggests.</p> <p>Memory mainly targets consistency in generation, and it roughly falls into two categories:</p> <ul> <li>If conversations happen within the same system, how do we keep consistency?</li> <li>If conversations happen across different systems, how do we keep consistency?</li> </ul> <p>Let's look at how a typical memory system works:</p> <ul> <li>When a user has a conversation, relevant corpora and preferences are extracted by the model and routed to a memory sidecar for retrieval.</li> <li>Given relevance to the user's question and the context window, results are ranked and trimmed, then filled into a generation template.</li> <li>The template guides the model's generation.</li> </ul> <p>Cost is deterministic. Most memory systems claim that by introducing a smaller model or pre/post steps, they can significantly reduce token cost and latency (for example, a common trick is to drop the facts associated with each memory item) while achieving good enough quality.</p> <p>But what matters is another point: consistency is not free:</p> <ul> <li>Extraction is a lossy process, and we require the model to participate deeply in it. If you've ever criticized hallucinations, what makes you accept the extraction step?</li> <li>User preferences are broad and not constant. The investment needed to calibrate this will far exceed imagination. If you've looked at knowledge graphs or GraphRAG, you can easily foresee this.</li> <li>Most memoryless approaches (for example, Markdown-based ones) are essentially embeddingless approaches. They still cannot solve distortion and drift when switching models.</li> <li>As a sidecar, failure should not affect system behavior. So you may not get the consistency you expect.</li> </ul> <p>Will the token and time savings from the past come back at some point? Memory looks full of difficulties. Then what about context, could it be another bright road?</p> <h2 id="context-another-sugar-coated-bullet">Context, Another Sugar-Coated Bullet<a class="zola-anchor" href="#context-another-sugar-coated-bullet" aria-label="Anchor link for: context-another-sugar-coated-bullet" style="visibility: hidden;"></a> </h2> <p>In the established meaning of context engineering, it contains everything: the sessions and memory mentioned above, external RAG systems, real-time queries on the internet, and knowledge from various business systems.</p> <p>It's hard for me to imagine anything that cannot be put into this basket. So we can simply assume that, in this worldview, context will keep expanding until it exceeds a fixed window limit. That's also where most people's obsession with context management begins: truncate, add retrieval constraints, add hand-coded rules, and try to find a large enough, high-quality context to solve the problem.</p> <p>I want to offer two counterexamples:</p> <ul> <li>In a well-known evaluation of long-context capability, besides the GPT family, most models show cliff-like quality degradation as context grows. And because each model has a different capability boundary, context management can hardly be directly portable across all models and agent systems.</li> <li>Another example is "lobsters attacking the moon". A Kimi staff member once suggested that users should use caching mechanisms to design lobsters, and avoid certain time periods to release load. The complexity of context management makes this request almost absurd. I would even criticize it as a form of laziness.</li> </ul> <p>Abstraction leakage is a serious system design problem. When we rely on certain established context management mechanisms to handle all of this, it also means there will be enough ways to break system reliability.</p> <h2 id="smarter-with-fewer-assumptions">Smarter, With Fewer Assumptions<a class="zola-anchor" href="#smarter-with-fewer-assumptions" aria-label="Anchor link for: smarter-with-fewer-assumptions" style="visibility: hidden;"></a> </h2> <p>From the beginning, bub was not designed as a "persistent conversation system". We designed it to work over a sufficiently long context, an endless tape. On day one it had to face socialized tasks in group chats. Problems keep appearing, tasks keep forming, and then keep ending. In such a diverse and complex setting, every turn may face a new goal. It does not, and cannot, require a state that continues forever, and it does not even require a conversation ancestor that must be inherited.</p> <p>Looking back at the idea that once scared me to create, an endless punch tape with anchors on it has worked well on bub. When we cross context boundaries or hit system failures, it gives us a chance to have repair methods that are simple and reliable enough.</p> <p>But in real-world tasks, there is no punch tape, and there will not truly be an anchor at some milestone. The point is an action: construct, not inherit. When a task appears, as humans we do not shove all history into the window. We do two things: explore and choose.</p> <ul> <li>Explore may include history, external systems, memory, or even real-time queries.</li> <li>Choose means discarding most irrelevant content, keeping only the minimal sufficient material.</li> </ul> <p>Context engineering defaults to increment, extension, and managing expansion. It assumes state must be maintained, so it introduces caching, extraction, merging, compression, and forking. Every layer is trying to patch the same premise: history cannot be dropped. But the reality is that they cannot carry enough history.</p> <p>Practice on bub gave me a simpler answer: history can be dropped. More precisely, history does not need to be inherited by default. It's just a library of materials. Query it when needed, ignore it when not.</p> <p>This is also where tape matters. It's not about recording everything; it's about enabling "read on demand". It turns state from an "ongoing burden" into an "optional resource". When context is no longer a monotonic function, cache is just an optimization, not a lifeline; memory is just an index, not a personality; session is just a boundary, not an identity.</p> <p>Being smarter does not mean introducing more mechanisms. It means assuming less.</p> <h2 id="simplicity">Simplicity<a class="zola-anchor" href="#simplicity" aria-label="Anchor link for: simplicity" style="visibility: hidden;"></a> </h2> <p>When I wrote down the title, "Prometheus Bound", I thought I was criticizing a kind of bondage.</p> <p>We are used to metaphors, imagining the model as suppressed fire, a force constrained by engineering systems. But maybe the problem is not that dramatic. The model is not imprisoned. It is just misused within a wrong organization.</p> <p>Task appears, construct context, complete task, end. Next turn, start over. When we stop building systems on top of "persistent state", a lot of complexity disappears naturally.</p> <p>If the system is clear enough, we don't need myths. And we don't need Prometheus.</p> <hr /> <p>Of course, every system exists because there were real problems it tried to solve. This post contains a lot of criticism to express a viewpoint, but that does not mean those systems are meaningless.</p> <p>Again, thanks to bub and my friends, for giving me a chance to feel and express all of this more deeply.</p> <h2 id="recommended-reading">Recommended Reading<a class="zola-anchor" href="#recommended-reading" aria-label="Anchor link for: recommended-reading" style="visibility: hidden;"></a> </h2> <ul> <li><a rel="nofollow noreferrer" href="https://frostming.com/posts/2026/create-a-claw/">What does it take to create a lobster?</a></li> <li><a rel="nofollow noreferrer" href="https://psiace.me/zh/posts/reinvent-the-punch-tape/">Reinvent the punch tape</a></li> <li><a rel="nofollow noreferrer" href="https://github.com/psiace/bub">bub</a></li> <li><a rel="nofollow noreferrer" href="https://www.piglei.com/articles/ai-programming-is-a-new-framework/">AI Programming Is a New Framework</a></li> </ul> 2026-01-28T00:00:00+00:00 2026-01-28T00:00:00+00:00 Unknown https://psiace.me/posts/reinvent-the-punch-tape/ <blockquote> <p>Hi, I'm <a rel="nofollow noreferrer" href="https://github.com/PsiACE">PsiACE</a>. This idea was born about 4 months ago when I was still working toward better GraphRAG. I have some experience with Agents and databases over the past few years—feel free to read my previous posts.</p> </blockquote> <h2 id="tl-dr">TL;DR<a class="zola-anchor" href="#tl-dr" aria-label="Anchor link for: tl-dr" style="visibility: hidden;"></a> </h2> <p>Agent systems tend to grow complex: forking, rollback, short-term and long-term memory, carefully designed compression and truncation.</p> <p>The more engineering invested, the harder the behavior becomes to predict. I think it's time to reinvent the punch tape: an Agent has only one deterministic session. It is perpetual—never ending, never forking, only moving forward. On this endless punch tape, you can always reliably retrieve the small piece of context you need most at that moment.</p> <p><strong>Let interactions happen as naturally as in the real world.</strong></p> <p>Real human interaction never truly forks into multiple equally real timelines, nor does it erase what has happened through rollback. Conversations may wander off-topic, correct mistakes, switch tasks, or start a new phase—but they always happen along the same continuous timeline.</p> <h2 id="the-endless-tape">The Endless Tape<a class="zola-anchor" href="#the-endless-tape" aria-label="Anchor link for: the-endless-tape" style="visibility: hidden;"></a> </h2> <p><strong>The foundation of an Agent should be a long tape</strong>—yes, a log.</p> <p>Beyond simple conversation history, the log should record what happened: a tool result, a key decision, a phase transition—all written as facts. Once a fact is written, it is never modified or replaced with a lossy substitute.</p> <p>You can always go back to any point in time and see what actually happened—not a version that was later revised, compressed, or rewritten. The purpose of the log is not just to let the model read through history, but to make the Agent's behavior easier to verify.</p> <p>On top of this long tape, <strong>the Agent computes state</strong>.</p> <p>Naturally, we should use state computed from the log—instead of "memory"—to support queries and context assembly, rather than replacing facts with summaries or compressed content. It's somewhat like a materialized view in a database: it can be discarded, it can be rebuilt, and it should be able to explain where each piece of state comes from.</p> <p>At any moment, the system simply picks a new starting point, rebuilds state from the log, and continues execution.</p> <p>Anchors are introduced precisely for this purpose.</p> <p>If you rebuild state from the beginning of the log every time, the system eventually becomes unwieldy and hits context limits. <strong>Anchors provide a set of explicit rebuild starting points</strong>, expressing "from here, the state is stable enough to continue working." Anchors don't delete history, nor do they summarize it—they just tell the system where to start counting.</p> <p>An anchor is typically created by a handoff. When the Agent recognizes that the phase has changed, or that continuing from the current rebuild starting point would make state messy, it performs a handoff: it writes the minimal structured state still necessary for the next phase into the log, creates a new anchor, and subsequent state rebuilds start from that new anchor.</p> <p>What gets packaged here is not a summary of history, but state oriented toward the next phase; history remains complete in the log. The thread doesn't break, nor does it fork—it just keeps moving forward.</p> <h2 id="the-wheel-rolls-forward">The Wheel Rolls Forward<a class="zola-anchor" href="#the-wheel-rolls-forward" aria-label="Anchor link for: the-wheel-rolls-forward" style="visibility: hidden;"></a> </h2> <p><strong>Agents don't really fork.</strong></p> <p>Forking essentially creates multiple equally real futures, but the real world has only one "now." You can change direction, but you can't branch what has already happened into another timeline.</p> <p>Many scenarios that seem to require forking are more naturally expressed as handoffs. You don't need to create a parallel thread—you just acknowledge that you've entered a new phase. You're still moving forward on the same thread, just rebuilding state from a new anchor. The past still exists in the log, ready to be queried and referenced at any time, but it doesn't need to become a parallel world.</p> <p>Similarly, <strong>Agents don't really roll back.</strong></p> <p>Rollback means undoing what has already happened, but in real interaction, correction isn't done by erasing the past—it's done by adding new facts. From an engineering perspective, rollback changes state, but we only allow state to be changed by new facts; we don't allow old facts to disappear.</p> <p>The essence of "rollback requirements" is a context assembly problem. You want to ignore certain fragments in this execution, emphasize others, and guide the model to reason in a different direction. This doesn't require modifying the log or returning to an old state—it just requires selecting a different working set when assembling context.</p> <p><strong>Agents also don't really rely on various "memory tricks."</strong></p> <p>Multi-layer summaries, repeated compression, and complex heuristics may work in the short term, but over time they introduce noise, uncertainty, and inexplicability. It becomes harder and harder to distinguish what is fact, what is a model-generated generalization, and what is residual error from compression.</p> <p>Retrieval and recall from history are still necessary. You can pull original text fragments from the same thread's history, or pull evidence from external data sources, and temporarily assemble them into the current context. But lossy compression should never quietly replace history.</p> <h2 id="the-new-punch-tape-computer">The New Punch Tape Computer<a class="zola-anchor" href="#the-new-punch-tape-computer" aria-label="Anchor link for: the-new-punch-tape-computer" style="visibility: hidden;"></a> </h2> <p>This design thus becomes simple.</p> <p>Maintain an endlessly forward-moving thread, continuously appending facts. Use anchors to express phase transitions, use context assembly to control the window for each execution, allow on-demand reading from history and external data sources, and prevent lossy results from becoming new truth.</p> <p>Agents should fully leverage everything we've built in storage, computation, and other domains. "More reliable interaction" will beat "smarter memory."</p> 2025-12-31T00:00:00+00:00 2025-12-31T00:00:00+00:00 Unknown https://psiace.me/posts/my-2025-all-in-db-year-5/ <p>If I had not sprained my ankle on Labor Day in 2021, I might not have walked the path from lakehouse cloud-native to vector multimodal AI native. Yes, 2025 marks my fifth year of grinding in the database industry. It is year-end reflection time again. I recently did a hard retcon of my own history, but the ritual still matters. This time I want to talk more about myself.</p> <h2 id="just-talking-about-work">Just Talking About Work<a class="zola-anchor" href="#just-talking-about-work" aria-label="Anchor link for: just-talking-about-work" style="visibility: hidden;"></a> </h2> <p>I still work at a database company. Yes, "still" means I switched again. At the end of December, I joined OceanBase to work on open-source projects around databases and the AI ecosystem. This is the third database company in my career.</p> <p>For most of 2025 I worked on GraphRAG at NebulaGraph. Even though the track is crowded and no longer hot, my team and I still delivered a solid answer over the past year — 95% accuracy in domain-specific evaluations, and overall cost reduced to a level comparable to vector plus full-text hybrid indexes. It almost paved the road for wider adoption of graph-based document RAG.</p> <p>Work-driven outputs also created some positive impact. In March, when I researched MCP integration, I designed and implemented the first MCP integration in the LlamaIndex ecosystem, and contributed it upstream as llama-index-tools-mcp. After the community refined it further, it became a top 2% download package in the Python ecosystem.</p> <p>I am not a workaholic or a researcher. If we talk about code taste, among the colleagues I have worked with, I am probably just average. Even so, I can still shine. In the NebulaGraph community awards at the end of 2025, I was recognized by my peers. Thanks to everyone for the support and help. Though our paths are far apart, we will meet again.</p> <h2 id="may-open-source-bless-us">May Open Source Bless Us<a class="zola-anchor" href="#may-open-source-bless-us" aria-label="Anchor link for: may-open-source-bless-us" style="visibility: hidden;"></a> </h2> <p>Because the GenAI platform I work on is a commercial product, my open-source participation has changed. More often now, I do things from the angle of requirement research and delivery. The llama-index-tools-mcp mentioned above is one such result.</p> <p>I also explored coding agents. Following ampcode's article, I made a simple Python implementation. Later explorations became the foundation of my PyCon China 2025 talk, "Building a Multi-Agent Collaborative CLI Coding Assistant from Scratch."</p> <p>As a snippet from that talk, I introduced a Python SDK I wrote for the agent client protocol. I never expected that a few months later it would be integrated by kimi-cli, openhands, and others, and eventually become part of the official SDK. Monthly downloads passed one million, and I gained a new role as a maintainer.</p> <p>I also put work into Apache projects. I split several OpenDAL subprojects out of the repo, and recently spent some late nights on the Rust implementation of DataSketches.</p> <p>Open source might not be better or worse with me in it, but I do draw nourishment and momentum from it. The good things I once had seem to have come alongside open source. Even so, I still have too many promises left behind and failed to deliver.</p> <h2 id="toward-a-better-life">Toward a Better Life<a class="zola-anchor" href="#toward-a-better-life" aria-label="Anchor link for: toward-a-better-life" style="visibility: hidden;"></a> </h2> <p>The most important change this year is that, because I moved to Hangzhou for work at the end of the year, I finally ended a four-year long-distance relationship, now in its eighth year. I also unlocked more memories with my girlfriend, like practicing "goldfish mouth" at G.E.M.'s concert, and getting fan-made materials from wmls at a Mayday concert.</p> <p>I tried to pick up new skills through learning. I can still only play "Lan Hua Cao" by following the sheet, but at least I can recognize my guitar chords again. I practiced house dance for a while. Keeping up with other people's rhythm is already a big challenge for me, but moving with the beat always helps me relax.</p> <p>I have two not-so-cute cats. For a while they would scratch the door open at midnight and jump onto me to rave. Honestly, I have not taken good enough care of them. But Goji still shows me his little belly, and Aloe has been willing to come over for a tiny bite of cat treat after we moved to Hangzhou.</p> <p>Life, oh life. Days and nights, but when I pick up the pen I do not know what to write. Things are okay, no big storms. In a recent checkup, I did not see fatty liver on the report anymore.</p> <h2 id="a-few-lines-for-ai">A Few Lines for AI<a class="zola-anchor" href="#a-few-lines-for-ai" aria-label="Anchor link for: a-few-lines-for-ai" style="visibility: hidden;"></a> </h2> <p>Aside from some changes in efficiency, process, and rhythm, my work and life have not been stirred much by AI. Value does not shift dramatically just because one or two tools are added.</p> <p>I do not plan to count how much code I generated with AI, or how many tokens or dollars I spent in the past year. But, rarely, this is a piece of writing typed 100% by me.</p> <p>The world is going through profound change. I am honored to be part of it, one wave among countless, breaking against a rock.</p> 2025-12-25T00:00:00+00:00 2025-12-25T00:00:00+00:00 Unknown https://psiace.me/posts/testing-confidence/ <p>In the long process of building software systems that others can rely on, I gradually realized that testing is not merely a QA tool. It is an engineering method for establishing a trust boundary between maintainers and users.</p> <p>When a system lacks this boundary, complexity does not disappear. It shifts into maintainer judgment, user adoption cost, or collaboration friction on both sides.</p> <p>For maintainers, tests determine whether a system can keep evolving. For users, tests determine whether a system is worth relying on.</p> <p>"Testing confidence" does not come from a metric. It comes from whether these two perspectives are clearly aligned through tests.</p> <h2 id="testing-is-not-the-same-as-coverage">Testing is not the same as coverage<a class="zola-anchor" href="#testing-is-not-the-same-as-coverage" aria-label="Anchor link for: testing-is-not-the-same-as-coverage" style="visibility: hidden;"></a> </h2> <p>In practice, testing is often reduced to metrics: coverage, CI pass rate, regression counts. These metrics are valuable to maintainers because they reduce uncertainty when changing code.</p> <p>But they do not answer the question users really care about:</p> <blockquote> <p>When a system is placed in real environments, real load, and real dependencies, is its behavior predictable?</p> </blockquote> <p>Once others use a system, their usage almost always deviates from the designer's intent. The value of tests is not to enumerate all paths. It is to use executable cases to make expected behavior and boundary conditions explicit and fixed.</p> <p>When those boundaries are clear:</p> <ul> <li>Maintainers know which behaviors must not be broken.</li> <li>Users know which behaviors they can rely on.</li> </ul> <p>From this perspective, tests are not an accessory to code. They are a layer that connects maintainers and users.</p> <h2 id="testing-is-a-long-term-constraint-on-semantics">Testing is a long-term constraint on semantics<a class="zola-anchor" href="#testing-is-a-long-term-constraint-on-semantics" aria-label="Anchor link for: testing-is-a-long-term-constraint-on-semantics" style="visibility: hidden;"></a> </h2> <p>In Databend, we rely heavily on test cases already validated by existing database systems. We also track key benchmarks over the long term to lock in our commitment to established semantics and standards.</p> <p>One important fact is this: the primary source of system complexity is not implementation difficulty, but the stability of semantics over time.</p> <p>For example:</p> <ul> <li>Whether to return a result or an error under certain exceptional conditions.</li> <li>Whether different execution paths satisfy the same invariants.</li> <li>Whether behaviors that are inelegant but widely used must remain compatible.</li> </ul> <p>If these semantics are not fixed by tests, they become a hidden burden for maintainers. Every change then depends on memory and experience to judge, "Will this break something?"</p> <p>For users, semantic drift is even more direct. The same usage can produce different results across versions.</p> <p>Therefore, the role of testing here is not "checking for bugs." It is:</p> <ul> <li>Providing clear evolution boundaries for maintainers.</li> <li>Providing stable behavioral expectations for users.</li> </ul> <p>This is why systems like MySQL and SQLite treat tests as high-value assets. Tests carry long-term promises to users.</p> <h2 id="testing-aligns-maintainer-judgment-with-user-reality">Testing aligns maintainer judgment with user reality<a class="zola-anchor" href="#testing-aligns-maintainer-judgment-with-user-reality" aria-label="Anchor link for: testing-aligns-maintainer-judgment-with-user-reality" style="visibility: hidden;"></a> </h2> <p>In Apache OpenDAL's CI, we include hundreds of tests across multiple platforms and services. In practice, a single contributor rarely has the time to assemble the full environment of dependencies and services.</p> <p>If Databend's tests mainly stabilize internal and external semantics, then in Apache OpenDAL, tests are more about bridging the gap between maintainer judgment and user reality.</p> <p>In the object storage ecosystem, many services claim to be "S3 compatible." For maintainers, implementing the spec can easily lead to the assumption that the system "should work."</p> <p>But users do not operate at the spec level. They use real accounts, real services, and real failure modes. Many issues surface precisely in that gap between assumption and reality.</p> <p>Therefore, in OpenDAL, we insist on:</p> <ul> <li>Using real services for tests whenever possible.</li> <li>Not treating docs or compatibility claims as engineering facts.</li> <li>Keeping failure cases long term as part of regression tests.</li> </ul> <p>The value of these tests is:</p> <ul> <li>They constrain maintainers' overly optimistic assumptions about the external world.</li> <li>They give users a clear signal about which behaviors are validated in real environments and can be relied on.</li> </ul> <p>Here, tests are not about "proving correctness." They are about keeping maintainer judgment consistent with user reality.</p> <h2 id="testing-confidence-a-shared-understanding-of-failure">Testing confidence: a shared understanding of failure<a class="zola-anchor" href="#testing-confidence-a-shared-understanding-of-failure" aria-label="Anchor link for: testing-confidence-a-shared-understanding-of-failure" style="visibility: hidden;"></a> </h2> <p>Across these practices, I have formed a stable view:</p> <blockquote> <p>Testing confidence is not that a system "rarely fails," but that when failures happen, maintainers and users share a consistent understanding of their nature.</p> </blockquote> <p>When a behavior is covered by tests:</p> <ul> <li>For users, a failure means a bug.</li> <li>For maintainers, fixing it is a clear engineering responsibility.</li> </ul> <p>When a behavior is not covered by tests:</p> <ul> <li>For users, it should not be treated as a system promise.</li> <li>For maintainers, it should not be assumed as behavior to keep compatible long term.</li> </ul> <p>Tests make this distinction clear, executable, and regression-ready. They also significantly reduce friction in expectations on both sides.</p> <h2 id="conclusion">Conclusion<a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion" style="visibility: hidden;"></a> </h2> <p>From an engineering perspective, testing is neither a tool only for maintainers nor a mechanism only for users. It is a structure that helps both sides reach consensus on the same behavioral boundaries.</p> <p>It turns implicit assumptions into explicit constraints. It turns personal experience into a sustainable engineering boundary.</p> <p>For me, no matter which layer the system lives at, testing ultimately serves one thing:</p> <blockquote> <p>To make systems predictable when they are depended on, and to make maintenance sustainable over the long run.</p> </blockquote> 2025-11-29T00:00:00+00:00 2025-11-29T00:00:00+00:00 Unknown https://psiace.me/posts/live-a-little-part-2/ <p><strong>A Gambler's City Chronicle</strong></p> <p>This piece is 2021–2025, the years when most overlaps with friends began. 2021 was a database boom, 2024 an AI frenzy. I caught both, never won big, but stayed at the table. From 2022 to 2024 I lived in a gambling city; the title comes from there. Names may be missing, but the people are not forgotten.</p> <hr /> <h2 id="addenda">Addenda<a class="zola-anchor" href="#addenda" aria-label="Anchor link for: addenda" style="visibility: hidden;"></a> </h2> <p>I once wrote some frontend. Added the ACF2+ pitch detection algorithm in JS to pitchfinder and built a small audio and visualization demo. Tried music OCR: read sheet music, turn it into notes, play with a Java library. Did simple image search by extracting features and comparing similarity, testing many combinations out of curiosity.</p> <hr /> <h2 id="databend">Databend<a class="zola-anchor" href="#databend" aria-label="Anchor link for: databend" style="visibility: hidden;"></a> </h2> <p>I had no database background, was the first new grad, and skipped the interview. When people asked how, I had no clear answer; I am just grateful for the trust.</p> <p>Work split between engineering and community. Engineering was repo-level refactors, dependency upkeep, and occasional new features. Community came from liking to write and share; by the time I went back to school it had become a main track.</p> <p>Early on I moved tests from source files to external unit tests to make builds faster and cleaner; it worked but added file switching and mental load. As the codebase grew, we split things again to improve build parallelism and quality. I joined the big refactors led by sundy-li: he found <code>arrow2</code> from a reddit thread, migrated, then rebuilt again after Chi's type-level ideas.</p> <p>Cloud work with flaneur and Zhihan taught me cadence through their OKR alignment docs; soon we had an internal cloud prototype we could play with. For benchmarks I proposed writing results as json in the repo for visualization. Early Databend was a large practice field; I touched most corners and learned how a data system breathes.</p> <hr /> <h2 id="apache-opendal">Apache OpenDAL<a class="zola-anchor" href="#apache-opendal" aria-label="Anchor link for: apache-opendal" style="visibility: hidden;"></a> </h2> <p>I met Xuanwo when he was handling the website move. The storage access layer then was dal, designed by Zhao to favor object storage while staying HDFS friendly. Xuanwo suggested a rewrite into a general interface, so dal2 was built and later spun out as opendal.</p> <p>Cai built the early WebHDFS integration; Databend aslo added Hive support. Many data companies had similar needs, so opendal drew attention; with tison's help it entered foundation incubation.</p> <p>Before opendal I was just a salaried engineer writing open source. Here I felt trust and cooperation without a profit goal — “community over code.”</p> <hr /> <h2 id="macau-school-years">Macau School Years<a class="zola-anchor" href="#macau-school-years" aria-label="Anchor link for: macau-school-years" style="visibility: hidden;"></a> </h2> <p>After a year of full-time work I went to Macau for Applied Math and Data Science. Maybe it was too much time indoors, maybe feeling not good enough; I wanted to get out.</p> <p>Math classes were heavy; convex optimization cost hair but I made it through. AI and LLM work entered daily life: Meta's SAM for segmentation, CLIP for interactive captioning, a small nanogpt with tweaks to model structure, optimizers, and schedulers.</p> <p>I stayed close to Databend: compiler and Rust optimizations with a few talks; studying lakeFS and HuggingFace data; KubeSphere 4.0 integration and a Databend playground; an online hackathon; most Chinese docs plus an LLM workflow to translate English docs; I even skipped class to join the debate on opening enterprise source.</p> <p>At ApacheCon 2023 I met Xuanwo in person; we each gave an OpenDAL talk. He sent me wedding candy; I sometimes acted as his merge bot. I also met tison, saka, flaneur, xp, and went to the Summer Palace with TCeason. Later I met Zhihan and Eric; at Rust conf I finally matched RisingWave IDs to faces.</p> <p>In 2024, after a trip to Japan, I focused on a thesis about modern cache replacement for layered storage. Talked once with juncheng, who led sieve and s3fifo and kept designs minimal. Graduation forced me to narrow focus; my advisor's calm helped. I graduated and left Databend. My last public talk there was “Caching for Modern Layered Storage.”</p> <hr /> <h2 id="nebulagraph">NebulaGraph<a class="zola-anchor" href="#nebulagraph" aria-label="Anchor link for: nebulagraph" style="visibility: hidden;"></a> </h2> <p>Leaving Databend meant job hunting again. I still wanted data infra and even considered less technical roles. The market was cold; two peer companies took me to CXO rounds with no offer. Big-tech interviews were flat: interviewers were lukewarm about my background, and I did not lean into algorithms or playbook questions. Friends pointed me to web3; offers were high but compliance risks kept me away.</p> <p>The turn came at ApacheCon Hangzhou. Friends Dinah and Yanli told me NebulaGraph's GenAI team was hiring; the lead was long-time Twitter friend wey-gu. The final round with Sherman included a spoken algorithm question — not something I expected in a culture-fit chat.</p> <p>Before joining I went to Shanghai for Rust Conf and to spot the office; finally met yihong0618, an interesting soul. Met RisingWave friends, including MrCroxx (foyer-rs) and Runji (arrow-udf, later at deepseek). Six months later at KCD Beijing I visited their office and tried the dried fish.</p> <p>At PyCon China 2024, wey gave a keynote; I met frostming and Jintao and took a photo with pink-haired saka. At Rust Conf 2025 I met Leiysky and Andy; they and tison were starting a new product. Later that year at PyCon I spoke on coding agents and met many friends from chat groups.</p> <p>Side projects:</p> <ul> <li>First MCP integration for Llama Index, with the initial implementation contributed to the official repo.</li> <li>First agentclientprotocol Python SDK, adopted as an official SDK with help from JetBrains friends.</li> <li>Visited their Shanghai office and carried home a basket of swag.</li> </ul> <p>We moved from Pudong to Nanjing West Road, from a dark room to sunlight. Dinah left for OceanBase; wey left to build nowledage memory. Yanli and I stayed at NebulaGraph, kept working on GraphRAG and agents, and shipped Fusion GraphRAG. Shao handled end-to-end evaluation, insisting recall and accuracy come first. Fusion GraphRAG keeps onboarding cost close to vector or full-text RAG and reaches 95%+ QA accuracy in some domains.</p> <hr /> <h2 id="closing">Closing<a class="zola-anchor" href="#closing" aria-label="Anchor link for: closing" style="visibility: hidden;"></a> </h2> <p>Time has moved to late 2025. That is about it.</p> 2025-11-07T00:00:00+00:00 2025-11-07T00:00:00+00:00 Unknown https://psiace.me/posts/acp-python-sdk-new-chapter/ <p>When I first started working on the <strong><a rel="nofollow noreferrer" href="https://github.com/agentclientprotocol/python-sdk">Agent Client Protocol Python SDK</a></strong>, it was a small side project born out of curiosity. I wanted to understand how coding agents communicate—how state, intent, and messages flow between the client and the agent.</p> <p>Over time it grew into something real. More developers began to use it—from <strong>Moonshot</strong> to other model companies exploring agent workflows. The SDK became more stable, closely tracking the evolving ACP spec and serving as a foundation for real-world integrations.</p> <p>Today marks a new chapter. The Python SDK has officially moved under the <strong><a rel="nofollow noreferrer" href="https://github.com/agentclientprotocol">agentclientprotocol</a></strong> organization, joining the TypeScript, Rust, and Kotlin SDKs as part of the official ecosystem.</p> <p>I will keep maintaining and improving it there. My focus stays on reliability, clarity, and making it easier for developers to build agentic systems.</p> <p>Special thanks to <strong>Sergey Ignatov</strong>, <strong>Anna</strong>, and <strong>Ben</strong> for their trust and support during this transition—and to everyone who has used, contributed to, or shared feedback on this SDK.</p> <p>This project started as an experiment, but it has become a shared effort. Here is to the next chapter—together.</p> <p>— <strong>Chojan Shang (PsiACE)</strong> Maintainer, ACP Python SDK</p> 2025-10-20T00:00:00+00:00 2025-10-20T00:00:00+00:00 Unknown https://psiace.me/posts/live-a-little-part-1/ <p><strong>Between Code and Youth</strong></p> <blockquote> <p>I'm turning twenty-seven soon. By the standards of this era, that already puts me in the “old folks” cohort. My second full-time job has just crossed its first anniversary. Looking back, I want to talk about the years I spent growing up with code.</p> </blockquote> <hr /> <h2 id="2016-a-boy-and-his-first-lines-of-code">2016 – A Boy and His First Lines of Code<a class="zola-anchor" href="#2016-a-boy-and-his-first-lines-of-code" aria-label="Anchor link for: 2016-a-boy-and-his-first-lines-of-code" style="visibility: hidden;"></a> </h2> <p>Back in 2016, I was a book addict, devouring door-stopper web novels by the truckload. Ruby was everywhere in China, so I tinkered with Rails and deployed a Jekyll site on GitHub Pages. Wanting prettier UI nudged me into HTML basics. I even snuck in a full read of <em>Digital Design and Computer Architecture</em> — a Verilog HDL and computer architecture primer. I barely understood it, but teenage bravado kept me going.</p> <hr /> <h2 id="2017-2018-university-and-first-encounters">2017–2018 University and First Encounters<a class="zola-anchor" href="#2017-2018-university-and-first-encounters" aria-label="Anchor link for: 2017-2018-university-and-first-encounters" style="visibility: hidden;"></a> </h2> <p>I entered university in 2017, majoring in Computer Science and Technology. Campus was huge; even the pre-pandemic shuttle only covered half of it. Math never came easy, yet coding let me build things, so I stuck with it.</p> <p>In 2018 I tried to take algorithms and English seriously while already living inside Linux. I devoured GNU and Free Software Foundation histories and helped the GNU.org proofread articles. I barely knew tools beyond basic git, so seniors merged patches for me. Mailing lists introduced me to Junyu, whose WebP image service would later go viral on Twitter.</p> <p>That year I also wrote to the fresh <a rel="nofollow noreferrer" href="https://leetcode.cn/">LeetCode China</a> launch to volunteer for solution translations. Hercy sent my first official reply, and Winston helped me land the gig. Seven years on, we’re still friends.</p> <p>Of course 2018 had more stories: I failed Calculus yet met my girlfriend. That autumn I watched <em>Crystal Sky of Yesterday</em> with studio friends, then said on the way out:</p> <blockquote> <p>“This movie should be watched with someone special.” I bought two more tickets to rewatch it with a junior from the club — seven years later we’re still together.</p> </blockquote> <p>Technically, in 2018 I reset my GitHub account for the third time, experimented with Servo, learned Cargo commands, and fixed README build steps. I never imagined a future job would have me writing Rust for real.</p> <hr /> <h2 id="2019-translation-and-community">2019 – Translation and Community<a class="zola-anchor" href="#2019-translation-and-community" aria-label="Anchor link for: 2019-translation-and-community" style="visibility: hidden;"></a> </h2> <p>Translation carried me farther than I expected. In 2019 <a rel="nofollow noreferrer" href="https://linux.cn/lctt/">LCTT</a> was still the most vibrant Linux community in China. There I met Wang and teacher Bai Huancheng; we even planned a book that never shipped, yet that collaboration remains unforgettable.</p> <p>Python was booming. I picked up Flask, Django, and FastAPI and became a maintainer of <a rel="nofollow noreferrer" href="https://github.com/audreyfeldroy/cookiecutter-pypackage">audreyfeldroy/cookiecutter-pypackage</a>, a project with thousands of stars. Audrey Feldroy and Daniel Roy “Pydanny” Greenfeld still run that legendary two-person shop.</p> <p>By year end I finished the compiler course, read <em>Crafting Interpreters</em>, and built <a rel="nofollow noreferrer" href="https://github.com/PsiACE/meco">meco</a>, a C project template, plus the toy language <a rel="nofollow noreferrer" href="https://github.com/PsiACE/emo">emo</a>. I also tried Neo4j and NebulaGraph, building a graph-visualization project. I had no idea I’d later work at a graph database company.</p> <hr /> <h2 id="2020-rust-summer-and-finding-a-direction">2020 – Rust Summer and Finding a Direction<a class="zola-anchor" href="#2020-rust-summer-and-finding-a-direction" aria-label="Anchor link for: 2020-rust-summer-and-finding-a-direction" style="visibility: hidden;"></a> </h2> <p>I told myself I was preparing for graduate school in 2020, yet my mind stayed half in code. I shifted from Python to Rust and joined the first <a rel="nofollow noreferrer" href="https://summer-ospp.ac.cn/">Open Source Promotion Plan (OSPP)</a> to build <a rel="nofollow noreferrer" href="https://github.com/casbin-rs/casbin-raft">casbin-raft</a>. That’s where I met Jiang Cheng and Hackerchai — Jiang later wrote a recommendation letter, and chai went on to Kong. My connection with OSPP ran even deeper, continuing to this day.</p> <p>I also translated a chapter of <em>Writing an OS in Rust</em> and met Luo Jia, creator of <a rel="nofollow noreferrer" href="https://github.com/rustsbi/rustsbi">RustSBI</a>. nrc’s <em>“Early Impressions of Go from a Rust Programmer”</em> was another piece I translated. Their VLDB paper became the backbone of my thesis proposal, and with permission I published my notes at <a rel="nofollow noreferrer" href="https://github.com/psiace/tidb-a-raft-based-htap-database">PsiACE/TiDB-A-Raft-based-HTAP-Database</a>.</p> <p>My thesis was a Rust-based distributed KV store built with Raft + Bitcask. If you know PingCAP, you can guess the inspiration. I studied <a rel="nofollow noreferrer" href="https://github.com/datafuselabs/openraft">async-raft</a> — the predecessor of databend/openraft — and contributed to PingCAP projects. I donated to the inaugural <a rel="nofollow noreferrer" href="https://rustcc.cn/2020rustchinaconf/">Rust China Conf 2020</a> and later edited the Rust Chinese Community Daily for a few years.</p> <hr /> <h2 id="2021-from-failed-exams-to-first-job">2021 – From Failed Exams to First Job<a class="zola-anchor" href="#2021-from-failed-exams-to-first-job" aria-label="Anchor link for: 2021-from-failed-exams-to-first-job" style="visibility: hidden;"></a> </h2> <p>The exam was a mess — I was still fixing lint in <a rel="nofollow noreferrer" href="https://github.com/xi-editor/xi-editor">xi-editor</a> during review weeks. I interviewed with Wasmedge for an internship, learned about WebAssembly and the ERC20 standard, and my solution was eventually adopted by the community. And forked <a rel="nofollow noreferrer" href="https://github.com/PsiACE/riteraft">riteraft</a> and <a rel="nofollow noreferrer" href="https://github.com/PsiACE/ritelinked">ritelinked</a> from Meilisearch and Hashlink, respectively, earning a modest following. I also interviewed with PingCAP for <a rel="nofollow noreferrer" href="https://github.com/ti-community-infra/tichi">TiChi</a>, learning Prow deployment the hard way.</p> <p>Meanwhile I contributed to <a rel="nofollow noreferrer" href="https://github.com/datafuselabs/databend">Databend</a> across CI, fuzzing, and upstream Arrow and SQL parser changes. Back then Databend had maybe ten people. I added BohuTANG on WeChat just to learn, yet after my thesis defense he invited me to join. We met in Wuhan, and his founder energy was blindingly bright. During June and July I sent around forty PRs upstream and downstream. That’s how I joined a team determined to build the “Snowflake of China.” My first real engineering job began.</p> <hr /> <blockquote> <p><em>To be continued — Part 2 will cover 2022 to 2025, from Databend to Apache OpenDAL and beyond.</em></p> </blockquote> 2025-10-15T00:00:00+00:00 2025-10-15T00:00:00+00:00 Unknown https://psiace.me/posts/spec-driven-sdks/ <blockquote> <p>AI writes fast — and tends to bloat. Specs are strict — and can ossify.</p> <p>I choose the middle path — let AI run inside boundaries, and let humans own outcomes.</p> </blockquote> <h2 id="why-not-pure-ai-nor-pure-spec">Why Not Pure AI, Nor Pure Spec<a class="zola-anchor" href="#why-not-pure-ai-nor-pure-spec" aria-label="Anchor link for: why-not-pure-ai-nor-pure-spec" style="visibility: hidden;"></a> </h2> <p>AI‑assisted coding is addictive, but as projects grow, context drifts and complexity balloons. You get something that runs, but barely holds.</p> <p>On the other hand, pure spec‑driven frameworks — like GitHub’s <a rel="nofollow noreferrer" href="https://github.com/github/spec-kit">Spec Kit</a> — are often heavy for iterative work. Precise, but hard to live with.</p> <p>So I took the middle ground: spec‑first boundaries, AI‑assisted speed — using the Agent Client Protocol (ACP).</p> <h2 id="what-makes-acp-a-good-base">What Makes ACP a Good Base<a class="zola-anchor" href="#what-makes-acp-a-good-base" aria-label="Anchor link for: what-makes-acp-a-good-base" style="visibility: hidden;"></a> </h2> <p>ACP, started by Zed, defines how editors and agents talk. It uses JSON‑RPC 2.0 and is described in JSON Schema — clear edges, machine‑readable types, auditable behaviors. That balance makes it ideal for SDKs that must live across languages and over time.</p> <p>I applied this while maintaining the ACP <a rel="nofollow noreferrer" href="https://github.com/psiace/agent-client-protocol-python">Python SDK</a> for the <a rel="nofollow noreferrer" href="https://agentclientprotocol.com">Agent Client Protocol</a>.</p> <h2 id="how-i-built-the-python-sdk">How I Built the Python SDK<a class="zola-anchor" href="#how-i-built-the-python-sdk" aria-label="Anchor link for: how-i-built-the-python-sdk" style="visibility: hidden;"></a> </h2> <h3 id="stage-0-minimal-but-runnable">Stage 0 — Minimal, but Runnable<a class="zola-anchor" href="#stage-0-minimal-but-runnable" aria-label="Anchor link for: stage-0-minimal-but-runnable" style="visibility: hidden;"></a> </h3> <p>I relied heavily on a coding agent (Codex) with human‑set boundaries and review. Three constraints:</p> <ol> <li>Agent‑generated Pydantic models from the datamodel.</li> <li>Minimal I/O (JSON‑RPC over stdio), with agent assistance.</li> <li>Agent‑ported examples and basic tests, human‑verified for regression.</li> </ol> <p>Small on purpose: ship first, surface problems early.</p> <h3 id="stage-1-learning-from-feedback">Stage 1 — Learning from Feedback<a class="zola-anchor" href="#stage-1-learning-from-feedback" aria-label="Anchor link for: stage-1-learning-from-feedback" style="visibility: hidden;"></a> </h3> <p>Early users found naming awkward and ergonomics rough. I had the agent scan docs to produce a rename map (more Pythonic names), then borrowed ideas from other SDKs:</p> <ul> <li>TypeScript had helpers.</li> <li>Go had golden tests.</li> <li>Rust had clean modular structure.</li> <li>Protocol SDKs like MCP offered architecture hints.</li> </ul> <p>With agent assistance I restructured code and tests; I set the standards and did the reviews.</p> <h3 id="stage-2-routing-over-conditionals">Stage 2 — Routing over Conditionals<a class="zola-anchor" href="#stage-2-routing-over-conditionals" aria-label="Anchor link for: stage-2-routing-over-conditionals" style="visibility: hidden;"></a> </h3> <p>I didn’t want endless if‑trees like <code>if method == "..."</code>. So I wrote a compact router that makes method→handler first‑class:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">builder</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">RouterBuilder</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">builder</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">request_attr</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CLIENT_METHODS</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">fs_write_text_file<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">WriteTextFileRequest</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">writeTextFile<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">builder</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">request_attr</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">CLIENT_METHODS</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">fs_read_text_file<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ReadTextFileRequest</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">readTextFile<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>Registered handlers are pluggable, testable, and auditable. By v0.4.9 the SDK wasn’t big, but it was stable and teachable.</p> <h3 id="context-curation-feed-the-agent-only-what-matters">Context curation — feed the agent only what matters<a class="zola-anchor" href="#context-curation-feed-the-agent-only-what-matters" aria-label="Anchor link for: context-curation-feed-the-agent-only-what-matters" style="visibility: hidden;"></a> </h3> <p>I treat prompt context as an engineering budget:</p> <ul> <li>Context manifest: list only schemas, public interfaces, and minimal examples; exclude generated/vendor code.</li> <li>Diff‑aware prompting: provide changed hunks plus a small window (5–15 lines), not whole files.</li> <li>Golden snippets index: let the agent retrieve tagged canonical examples instead of pasting large blobs.</li> <li>Guardrails first: “Do not invent fields beyond schema; fail closed; ask for missing inputs.”</li> </ul> <h2 id="principles-for-spec-driven-ai-assisted-sdks">Principles for Spec‑Driven, AI‑Assisted SDKs<a class="zola-anchor" href="#principles-for-spec-driven-ai-assisted-sdks" aria-label="Anchor link for: principles-for-spec-driven-ai-assisted-sdks" style="visibility: hidden;"></a> </h2> <table><thead><tr><th style="text-align: left">Principle</th><th style="text-align: left">Description</th></tr></thead><tbody> <tr><td style="text-align: left">Separate generated vs human‑governed code</td><td style="text-align: left">Models are generated; business logic is human‑owned and reviewed. Never mix.</td></tr> <tr><td style="text-align: left">Ship the smallest working unit first</td><td style="text-align: left">Real feedback drives ergonomics.</td></tr> <tr><td style="text-align: left">Golden tests for major RPCs</td><td style="text-align: left">Capture and replay requests/responses as the regression and audit baseline.</td></tr> <tr><td style="text-align: left">Curate and budget context</td><td style="text-align: left">Feed only what’s necessary — small, precise, reproducible.</td></tr> </tbody></table> <p>Specs give AI a sandbox — and AI makes specs come alive.</p> <h2 id="accountability-why-i-trust-ai-written-code">Accountability: Why I Trust AI‑Written Code<a class="zola-anchor" href="#accountability-why-i-trust-ai-written-code" aria-label="Anchor link for: accountability-why-i-trust-ai-written-code" style="visibility: hidden;"></a> </h2> <p>AI can write, but I set the boundaries. Typed models, reproducible tests, and replayable traces let me verify every step. I trust not the model, but the engineering discipline around it.</p> <h2 id="a-note-in-my-prs">A Note in My PRs<a class="zola-anchor" href="#a-note-in-my-prs" aria-label="Anchor link for: a-note-in-my-prs" style="visibility: hidden;"></a> </h2> <p>I first saw this note in @Xuanwo’s PRs. I haven’t used it in mine yet, but I’m willing to adopt it — because I’m accountable for the outcome. In this SDK work I used AI heavily; I set the boundaries and own the result:</p> <blockquote> <p>This PR was primarily authored with GPT‑5‑Codex and hand‑reviewed by me. I’m responsible for every line that lands here.</p> </blockquote> <p>This isn’t about showing off AI — it’s about owning the outcome.</p> <h2 id="closing-fuel-and-rails">Closing: Fuel and Rails<a class="zola-anchor" href="#closing-fuel-and-rails" aria-label="Anchor link for: closing-fuel-and-rails" style="visibility: hidden;"></a> </h2> <p>AI is the fuel. Spec is the rail. Fuel gives speed; rails keep you alive.</p> <p>I like this balance — AI writes, I define boundaries. Every line explainable, every bug reproducible. That’s how to build responsibly in the AI age.</p> <p>— <strong>PsiACE</strong> (<a rel="nofollow noreferrer" href="https://github.com/PsiACE">GitHub</a> · <a rel="nofollow noreferrer" href="https://opendal.apache.org/">Apache OpenDAL PMC Member</a>)</p> 2025-09-08T00:00:00+00:00 2025-09-08T00:00:00+00:00 Unknown https://psiace.me/posts/acp-and-python-sdk/ <blockquote> <p>I'm <a rel="nofollow noreferrer" href="https://github.com/PsiACE">PsiACE</a>. My recent work focuses on Agent and RAG: <strong>less metaphysics, more getting the feedback loop working</strong>, making systems that can explain themselves clearly, run stably, and be replayed and switched.</p> </blockquote> <h2 id="tl-dr">TL;DR<a class="zola-anchor" href="#tl-dr" aria-label="Anchor link for: tl-dr" style="visibility: hidden;"></a> </h2> <ul> <li>Minimal Python SDK to build ACP agents with low overhead, compatible with Zed and other ACP clients.</li> <li>Zed uses ACP (agent-client-protocol) to unify built‑in and external agents under a consistent UX, reducing integration friction and reinforcing the editor’s role.</li> <li>ACP is an interaction and semantics contract—not just wiring—standardizing how sessions, content, tools, and results flow.</li> <li>This is a standards race in the AI era (cf. cloud‑native). Adoption wins leverage; ACP sits alongside MCP and A2A, and overlaps with AG‑UI.</li> </ul> <h2 id="acp-beyond-editor-agent">ACP beyond “Editor ↔ Agent”<a class="zola-anchor" href="#acp-beyond-editor-agent" aria-label="Anchor link for: acp-beyond-editor-agent" style="visibility: hidden;"></a> </h2> <p>ACP specifies:</p> <ul> <li>Transport and framing (JSON‑RPC),</li> <li>Sessions and turns (<code>session/new</code>, <code>session/prompt</code>, <code>session/update</code>, <code>session/cancel</code>),</li> <li>Content Blocks (text/image/audio/resource/link),</li> <li>Tool Calls (<code>tool_call</code> / <code>tool_call_update</code>),</li> <li>File system requests (read/write),</li> <li>Plans (task lists and status).</li> </ul> <p>It moves interaction semantics into the protocol so hosts and agents don’t re‑invent UX contracts. Experiences become portable.</p> <p>References: Overview, Prompt Turn, Content, Tool Calls at https://agentclientprotocol.com</p> <h2 id="ag-ui-mcp-a2a-overlap-competition-complement">AG‑UI, MCP, A2A: overlap, competition, complement<a class="zola-anchor" href="#ag-ui-mcp-a2a-overlap-competition-complement" aria-label="Anchor link for: ag-ui-mcp-a2a-overlap-competition-complement" style="visibility: hidden;"></a> </h2> <ul> <li>AG‑UI: aims to standardize agent UX too—overlaps with ACP. Expect competition for mindshare and implementations. In some products, AG‑UI can render ACP outputs by mapping ACP semantics into generic components, or serve as a companion UI outside editors.</li> <li>MCP: standardizes access to tools/data. Complements ACP—MCP brings capabilities in; ACP carries interaction and presentation.</li> <li>A2A: aims to standardize agent‑to‑agent collaboration (in progress). ACP currently focuses on the client ↔ agent channel.</li> </ul> <h2 id="zed-s-angle-and-user-value">Zed’s angle and user value<a class="zola-anchor" href="#zed-s-angle-and-user-value" aria-label="Anchor link for: zed-s-angle-and-user-value" style="visibility: hidden;"></a> </h2> <ul> <li>Users: plug preferred agents (e.g., Gemini CLI, Claude Code) into Zed and get consistent streaming/tool/FS/plan UX.</li> <li>Developers: implement ACP once, run across ACP‑capable hosts—less per‑editor glue and lock‑in.</li> <li>Platform: a shared interaction contract stabilizes UX expectations and support costs, strengthening the editor’s position.</li> </ul> <p>Docs: https://zed.dev/docs/ai/external-agents (includes ACP log viewer for debugging)</p> <h2 id="design-highlights-sdk-aligned">Design highlights (SDK‑aligned)<a class="zola-anchor" href="#design-highlights-sdk-aligned" aria-label="Anchor link for: design-highlights-sdk-aligned" style="visibility: hidden;"></a> </h2> <ul> <li>Version: <code>PROTOCOL_VERSION = 1</code>.</li> <li>Transport: JSON‑RPC 2.0, line‑delimited JSON; asyncio streams helper provided.</li> <li>Sessions: create, load, cancel; prompts define turns; updates stream via <code>session/update</code>.</li> <li>Content: Markdown‑first for rich text; images/audio/resources supported via Content Blocks.</li> <li>Tools: <code>tool_call</code> and <code>tool_call_update</code> convey status/content/locations for host‑follow UI.</li> <li>FS: <code>fs/read_text_file</code> and <code>fs/write_text_file</code> exposed by hosts, callable by agents.</li> <li>Plan: <code>plan</code> updates carry task lists and statuses.</li> <li>UNSTABLE: terminal methods exist in schema but depend on host support and may change.</li> </ul> <p>All names/fields match the official schema; no custom extensions.</p> <h2 id="python-sdk">Python SDK<a class="zola-anchor" href="#python-sdk" aria-label="Anchor link for: python-sdk" style="visibility: hidden;"></a> </h2> <p>Install:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">pip</span></span><span class="z-meta z-function-call z-arguments z-shell"> install agent-client-protocol</span> </span></code></pre> <p>What you get:</p> <ul> <li>Pydantic v2 models generated from the schema.</li> <li><code>AgentSideConnection</code> (implement an Agent) and <code>ClientSideConnection</code> (implement a Client).</li> <li>Core types: Initialize/Prompt requests &amp; responses, SessionNotification, FS and permission requests.</li> </ul> <p><strong>Docs:</strong> https://psiace.github.io/agent-client-protocol-python</p> <p><strong>Code:</strong> https://github.com/psiace/agent-client-protocol-python</p> <h2 id="minimal-echo-agent">Minimal echo agent<a class="zola-anchor" href="#minimal-echo-agent" aria-label="Anchor link for: minimal-echo-agent" style="visibility: hidden;"></a> </h2> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">asyncio</span></span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">acp</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-punctuation z-section z-import-list z-begin z-python">(</span></span></span><span class="z-meta z-statement z-import z-python"> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Agent</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">AgentSideConnection</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">AuthenticateRequest</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">CancelNotification</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">InitializeRequest</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">InitializeResponse</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">LoadSessionRequest</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">NewSessionRequest</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">NewSessionResponse</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">PromptRequest</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">PromptResponse</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">stdio_streams</span><span class="z-punctuation z-separator z-import-list z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-punctuation z-section z-import-list z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">EchoAgent</span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-punctuation z-section z-inheritance z-begin z-python">(</span></span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-entity z-other z-inherited-class z-python">Agent</span><span class="z-punctuation z-section z-inheritance z-end z-python">)</span></span><span class="z-meta z-class z-python"><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">initialize</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">InitializeRequest</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">InitializeResponse</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">InitializeResponse</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">protocolVersion</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">params</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">protocolVersion</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">newSession</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">NewSessionRequest</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">NewSessionResponse</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">NewSessionResponse</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">sessionId</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">sess-1<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">loadSession</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">LoadSessionRequest</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-constant z-language z-python">None</span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">authenticate</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">AuthenticateRequest</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-constant z-language z-python">None</span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">prompt</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">PromptRequest</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">PromptResponse</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Typically you would stream via sessionUpdate here </span></span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">PromptResponse</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">stopReason</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">end_turn<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">cancel</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">CancelNotification</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-constant z-language z-python">None</span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">main</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">reader</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">writer</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">stdio_streams</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">AgentSideConnection</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function z-inline z-python"><span class="z-storage z-type z-function z-inline z-python"><span class="z-keyword z-declaration z-function z-inline z-python">lambda</span></span></span><span class="z-meta z-function z-inline z-python"><span class="z-meta z-function z-inline z-parameters z-python"> <span class="z-variable z-parameter z-python">_conn</span><span class="z-punctuation z-section z-function z-begin z-python">:</span></span></span><span class="z-meta z-function z-inline z-body z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">EchoAgent</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">writer</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">reader</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">asyncio</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Event</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">wait</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-variable z-magic z-python">__name__</span></span> <span class="z-keyword z-operator z-comparison z-python">==</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">__main__<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">asyncio</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">run</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">main</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <h2 id="run-in-zed">Run in Zed<a class="zola-anchor" href="#run-in-zed" aria-label="Anchor link for: run-in-zed" style="visibility: hidden;"></a> </h2> <pre data-lang="json" class="language-json z-code"><code class="language-json" data-lang="json"><span class="z-source z-json"><span class="z-meta z-mapping z-json"><span class="z-punctuation z-section z-mapping z-begin z-json">{</span> </span></span><span class="z-source z-json"><span class="z-meta z-mapping z-json"> </span><span class="z-meta z-mapping z-key z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>agent_servers<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span></span><span class="z-meta z-mapping z-json"><span class="z-punctuation z-separator z-mapping z-key-value z-json">:</span> </span><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-json"><span class="z-punctuation z-section z-mapping z-begin z-json">{</span> </span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-json"> </span><span class="z-meta z-mapping z-key z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>Echo Agent (Python)<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span></span><span class="z-meta z-mapping z-json"><span class="z-punctuation z-separator z-mapping z-key-value z-json">:</span> </span><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-json"><span class="z-punctuation z-section z-mapping z-begin z-json">{</span> </span></span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-json"> </span><span class="z-meta z-mapping z-key z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>command<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span></span><span class="z-meta z-mapping z-json"><span class="z-punctuation z-separator z-mapping z-key-value z-json">:</span> </span><span class="z-meta z-mapping z-value z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>/abs/path/to/python<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span><span class="z-punctuation z-separator z-mapping z-pair z-json">,</span> </span></span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"> </span><span class="z-meta z-mapping z-key z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>args<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span></span><span class="z-meta z-mapping z-value z-json"><span class="z-punctuation z-separator z-mapping z-key-value z-json">:</span> </span><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-sequence z-json"><span class="z-punctuation z-section z-sequence z-begin z-json">[</span><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>/abs/path/to/echo_agent.py<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span><span class="z-punctuation z-section z-sequence z-end z-json">]</span></span><span class="z-punctuation z-separator z-mapping z-pair z-json">,</span> </span></span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"> </span><span class="z-meta z-mapping z-key z-json"><span class="z-string z-quoted z-double z-json"><span class="z-punctuation z-definition z-string z-begin z-json">&quot;</span>env<span class="z-punctuation z-definition z-string z-end z-json">&quot;</span></span></span><span class="z-meta z-mapping z-value z-json"><span class="z-punctuation z-separator z-mapping z-key-value z-json">:</span> </span><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-json"><span class="z-punctuation z-section z-mapping z-begin z-json">{</span><span class="z-punctuation z-section z-mapping z-end z-json">}</span></span> </span></span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"> <span class="z-punctuation z-section z-mapping z-end z-json">}</span></span> </span></span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-meta z-mapping z-value z-json"> <span class="z-punctuation z-section z-mapping z-end z-json">}</span></span> </span></span><span class="z-source z-json"><span class="z-meta z-mapping z-value z-json"><span class="z-punctuation z-section z-mapping z-end z-json">}</span></span> </span></code></pre> <p>Open “dev: open acp logs” to inspect traffic if debugging serialization/fields.</p> <p><img src="https://psiace.me/posts/acp-and-python-sdk/echo-agent.png" alt="Echo Agent in Zed" /></p> <h2 id="closing">Closing<a class="zola-anchor" href="#closing" aria-label="Anchor link for: closing" style="visibility: hidden;"></a> </h2> <p>AI engineering is entering a phase where standards anchor ecosystems. Zed’s bet with ACP is pragmatic: integrate external agents while preserving a unified experience, so existing workflows and agent capabilities fit together naturally.</p> <p>The Python SDK makes “get it running first” easy: connect to ACP with minimal code, complete the feedback loop, then iterate.</p> 2025-08-14T00:00:00+00:00 2025-08-14T00:00:00+00:00 Unknown https://psiace.me/posts/agent-has-no-secret/ <blockquote> <p>I'm <a rel="nofollow noreferrer" href="https://github.com/PsiACE">PsiACE</a>. My recent work focuses on Agent and RAG: <strong>less metaphysics, more getting the feedback loop working</strong>, making systems that can explain themselves clearly, run stably, and be replayed and switched.</p> </blockquote> <h2 id="tl-dr">TL;DR<a class="zola-anchor" href="#tl-dr" aria-label="Anchor link for: tl-dr" style="visibility: hidden;"></a> </h2> <ul> <li><strong>Agent is not a new religion</strong>. It's still engineering practice with events, queues, and CRUD, plus a layer of model collaboration.</li> <li>Share one EventBus, connect <code>agent.*</code> and <code>huey.*</code> events; print every event immediately; finally use a Cascade tree to review causality and timing.</li> <li><strong>Single-file demo, run immediately</strong>: first see events, then see causality, finally see results.</li> </ul> <p><img src="https://psiace.me/posts/agent-has-no-secret/cascade-viewer.png" alt="Cascade Viewer" /></p> <hr /> <h2 id="why-i-write-this-and-why-this-way">Why I Write This (And Why This Way)<a class="zola-anchor" href="#why-i-write-this-and-why-this-way" aria-label="Anchor link for: why-i-write-this-and-why-this-way" style="visibility: hidden;"></a> </h2> <ul> <li>I don't want to discuss "how to draw a complex Agent architecture diagram" anymore. I care more about "<strong>how to make systems explain themselves clearly, fail gracefully, and review transparently</strong>".</li> <li>This article comes with a single-file demo: shared bus, strict ReACT, causality and timing combined, immediate observability. <strong>Ready to use</strong>.</li> </ul> <h2 id="how-i-did-it">How I Did It<a class="zola-anchor" href="#how-i-did-it" aria-label="Anchor link for: how-i-did-it" style="visibility: hidden;"></a> </h2> <h3 id="core-design-principles">Core Design Principles<a class="zola-anchor" href="#core-design-principles" aria-label="Anchor link for: core-design-principles" style="visibility: hidden;"></a> </h3> <ul> <li><strong>Shared EventBus</strong>: Agent, ToolExecutor, Storage all publish to the same bus.</li> <li><strong>Strict ReACT</strong>: Thought → Action → Action Input → Observation → Final Answer.</li> <li><strong>Causality + Timing</strong>: Each user input advances <code>tick</code>; each Observation advances <code>tick</code>; <code>Action → huey.* → Observation</code> is linked through parent-child relationships.</li> <li><strong>Immediate Observability</strong>: Print every event in real-time; finally use tree-shaped Cascade Viewer for time-based review.</li> </ul> <h3 id="event-flow-overview">Event Flow (Overview)<a class="zola-anchor" href="#event-flow-overview" aria-label="Anchor link for: event-flow-overview" style="visibility: hidden;"></a> </h3> <pre data-lang="mermaid" class="language-mermaid z-code"><code class="language-mermaid" data-lang="mermaid"><span class="z-text z-plain">graph TD </span><span class="z-text z-plain"> User([User]) --&gt; Agent </span><span class="z-text z-plain"> Agent --&gt;|publish Action| EventBus </span><span class="z-text z-plain"> EventBus --&gt;|agent.action| ToolExecutor </span><span class="z-text z-plain"> ToolExecutor --&gt;|huey.task| Huey </span><span class="z-text z-plain"> Huey --&gt; Storage[(EventureMemoryStorage)] </span><span class="z-text z-plain"> Storage --&gt;|huey.queue.* / huey.data.*| EventBus </span><span class="z-text z-plain"> ToolExecutor --&gt;|publish Observation| EventBus </span><span class="z-text z-plain"> EventBus --&gt; Agent </span><span class="z-text z-plain"> EventBus --&gt; EventLog </span><span class="z-text z-plain"> EventLog --&gt; Renderer[Cascade Viewer] </span></code></pre> <h2 id="how-is-this-different-from-traditional-architecture">How Is This Different from "Traditional Architecture"?<a class="zola-anchor" href="#how-is-this-different-from-traditional-architecture" aria-label="Anchor link for: how-is-this-different-from-traditional-architecture" style="visibility: hidden;"></a> </h2> <ul> <li><strong>No difference</strong>: Events, queues, CRUD were already there; just connected models as collaborators.</li> <li><strong>No hype</strong>: No design that fails when the word "intelligent" is removed; it's reliable, explainable, and easier to implement.</li> </ul> <h2 id="minimal-skeleton-excerpt">Minimal Skeleton (Excerpt)<a class="zola-anchor" href="#minimal-skeleton-excerpt" aria-label="Anchor link for: minimal-skeleton-excerpt" style="visibility: hidden;"></a> </h2> <h3 id="event-driven-storage-wrapper">Event-driven storage wrapper<a class="zola-anchor" href="#event-driven-storage-wrapper" aria-label="Anchor link for: event-driven-storage-wrapper" style="visibility: hidden;"></a> </h3> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">EventureMemoryStorage</span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-punctuation z-section z-inheritance z-begin z-python">(</span></span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-entity z-other z-inherited-class z-python">MemoryStorage</span><span class="z-punctuation z-section z-inheritance z-end z-python">)</span></span><span class="z-meta z-class z-python"><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-support z-function z-magic z-python">__init__</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">event_bus</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">EventBus</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">super</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-magic z-python">__init__</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">huey<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">bus</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">event_bus</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">enqueue</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">data</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">priority</span></span><span class="z-meta z-function z-parameters z-default-value z-python"><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">super</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">enqueue</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">data</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">priority</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">priority</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">bus</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">publish</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">huey.queue.enqueue<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-mapping-or-set z-python"><span class="z-punctuation z-section z-mapping-or-set z-begin z-python">{</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-mapping-or-set z-python"></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">data<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">data</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">priority<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">priority</span></span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-mapping z-value z-python"></span><span class="z-meta z-mapping z-python"> <span class="z-punctuation z-section z-mapping z-end z-python">}</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <h3 id="tool-with-self-description">Tool with self-description<a class="zola-anchor" href="#tool-with-self-description" aria-label="Anchor link for: tool-with-self-description" style="visibility: hidden;"></a> </h3> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">tool_echo</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">params</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">dict</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">dict</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-block z-documentation z-python"><span class="z-punctuation z-definition z-comment z-begin z-python">&quot;&quot;&quot;</span>Echo back text. Args: {&quot;text&quot;: string} Returns: {&quot;ok&quot;: bool, &quot;echo&quot;: string}<span class="z-punctuation z-definition z-comment z-end z-python">&quot;&quot;&quot;</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span></span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">ok<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-constant z-language z-python">True</span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">echo<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">str</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">params</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">text<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> </span></code></pre> <h3 id="react-loop-core-idea">ReACT loop (Core Idea)<a class="zola-anchor" href="#react-loop-core-idea" aria-label="Anchor link for: react-loop-core-idea" style="visibility: hidden;"></a> </h3> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-constant z-language z-python">True</span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">resp</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">chat</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">completions</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">create</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">model</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">messages</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">history</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">resp</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">choices</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">0</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">message</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">content</span> <span class="z-keyword z-operator z-logical z-python">or</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">call</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">extract</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">call</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">action_id</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">uuid4</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">bus</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">publish</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">agent.action<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span></span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">id<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">action_id</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">call</span></span><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> wait for agent.observation(id==action_id) and append as Observation </span></span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-continue z-python">continue</span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant</span></span> </span></code></pre> <h2 id="principles-i-follow">Principles I Follow<a class="zola-anchor" href="#principles-i-follow" aria-label="Anchor link for: principles-i-follow" style="visibility: hidden;"></a> </h2> <ul> <li><strong>Feedback loop first, fancy features later</strong>: Get "events → queue → state changes → ReACT" working before optimization.</li> <li><strong>Causality and timing are first-class citizens in engineering</strong>: tick expresses update cycles; parent-child expresses trigger chains.</li> <li><strong>Observability first</strong>: When systems can explain themselves, optimization has a foundation.</li> <li><strong>Replaceable</strong>: Bus is stable, backend can be swapped (memory → Redis), models can be swapped (vendor/version).</li> </ul> <h2 id="how-to-run-single-terminal">How to Run (Single Terminal)<a class="zola-anchor" href="#how-to-run-single-terminal" aria-label="Anchor link for: how-to-run-single-terminal" style="visibility: hidden;"></a> </h2> <h3 id="code-location">Code Location<a class="zola-anchor" href="#code-location" aria-label="Anchor link for: code-location" style="visibility: hidden;"></a> </h3> <p>Check the code at <a rel="nofollow noreferrer" href="https://github.com/PsiACE/psiace">PsiACE/psiace</a> to get the latest version in <code>demo</code> folder.</p> <ul> <li><a rel="nofollow noreferrer" href="https://github.com/PsiACE/psiace/blob/main/demo/agent-has-no-secret/agent_has_no_secret.py">agent_has_no_secret.py</a></li> <li><a rel="nofollow noreferrer" href="https://github.com/PsiACE/psiace/blob/main/demo/agent-has-no-secret/README.md">README.md</a></li> </ul> <h3 id="running-steps">Running Steps<a class="zola-anchor" href="#running-steps" aria-label="Anchor link for: running-steps" style="visibility: hidden;"></a> </h3> <ol> <li>Configure <code>.env</code> (OpenRouter recommended; OpenAI also works)</li> <li><code>python agent_has_no_secret.py</code></li> </ol> <h3 id="what-you-ll-see">What You'll See<a class="zola-anchor" href="#what-you-ll-see" aria-label="Anchor link for: what-you-ll-see" style="visibility: hidden;"></a> </h3> <ul> <li><strong>Real-time event stream</strong> (with colors): <code>user.input</code>, <code>agent.thought</code>, <code>agent.action</code>, <code>huey.data.*</code>, <code>agent.observation</code> …</li> <li>A green <strong>Final Answer</strong> panel</li> <li>A <strong>tree-shaped Cascade Viewer</strong> grouped by tick: under <code>agent.action</code> you can see the parent-child hierarchy of <code>huey.data.*</code> and <code>agent.observation</code></li> </ul> <hr /> <h2 id="final-thoughts">Final Thoughts<a class="zola-anchor" href="#final-thoughts" aria-label="Anchor link for: final-thoughts" style="visibility: hidden;"></a> </h2> <p>I admire engineers who can "<strong>explain clearly and make things work</strong>". Agent has no mystery:</p> <ul> <li>Get the feedback loop working first, then consider fancy features;</li> <li>Make causality and costs visible first, then talk about "smarter".</li> </ul> 2025-07-16T00:00:00+00:00 2025-07-16T00:00:00+00:00 Unknown https://psiace.me/posts/baby-step-coding-agent/ <p>Bub's first milestone: automatically fixing type annotations. Powered by Moonshot K2</p> <blockquote> <p>Bub: Successfully fixed the first mypy issue by adding the missing return type annotation <code>-&gt; None</code> to the <code>__init__</code> method in <code>src/bub/cli/render.py</code>, reducing the error count from 24 to 23.</p> </blockquote> <h2 id="1-why-build-your-own-agent">1. Why Build Your Own Agent?<a class="zola-anchor" href="#1-why-build-your-own-agent" aria-label="Anchor link for: 1-why-build-your-own-agent" style="visibility: hidden;"></a> </h2> <p>Coding agents have become a crowded market, with many excellent products available—like yetone's avante.nvim, AmpCode recommended by Brother Jintao, and numerous teams drawing grand blueprints in their presentations. However, the "token brute force" phenomenon in coding agents is also quite apparent—consuming hundreds of thousands of tokens while delivering minimal real value. This made me curious: if I built a coding agent myself, could I find a more efficient and practical solution?</p> <p>So I started working on a CLI assistant called Bub. Its goal is simple: to be a practical command-line assistant with limited self-bootstrapping capabilities. The concept of bootstrapping is nothing new in compilers and operating systems, and many coding agents love to tout "how much code AI engineers contributed." I hope Bub can take its first step by solving the first real problem in its own codebase.</p> <hr /> <h2 id="2-minimum-viable-get-it-running-first">2. Minimum Viable: Get It Running First<a class="zola-anchor" href="#2-minimum-viable-get-it-running-first" aria-label="Anchor link for: 2-minimum-viable-get-it-running-first" style="visibility: hidden;"></a> </h2> <p>My experience is that the smallest working loop is most important—don't start by pursuing "perfect architecture" or spend too much time on data theft.</p> <p>Bub's architecture is straightforward:</p> <ul> <li><strong>Agent</strong>: Main loop, responsible for LLM interaction, tool calling, and context management.</li> <li><strong>Tool</strong>: Each tool is a class with Pydantic-validated parameters.</li> <li><strong>ToolRegistry</strong>: Registers all tools for easy extension.</li> <li><strong>CLI</strong>: Typer + Rich, command-line entry point with support for expanding intermediate states.</li> </ul> <p>The essence of an agent is really just a while loop. In each iteration, the agent determines whether it needs to call tools based on the current context. If needed, it calls them and writes the tool results back to the context.</p> <h3 id="core-main-loop-code">Core Main Loop Code<a class="zola-anchor" href="#core-main-loop-code" aria-label="Anchor link for: core-main-loop-code" style="visibility: hidden;"></a> </h3> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">Agent</span></span><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">chat</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">message</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">conversation_history</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">append</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Message</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">role</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">user<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">content</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">message</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-while z-python"><span class="z-keyword z-control z-loop z-while z-python">while</span> <span class="z-constant z-language z-python">True</span><span class="z-punctuation z-section z-block z-loop z-while z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Construct context: historical messages + tool list </span></span><span class="z-source z-python"> <span class="z-constant z-language z-python">...</span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">litellm</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">completion</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-constant z-language z-python">...</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant_message</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">str</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">choices</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-constant z-numeric z-integer z-decimal z-python">0</span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">message</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">content</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">conversation_history</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">append</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Message</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">role</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">assistant<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">content</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant_message</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_calls</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">tool_executor</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">extract_tool_calls</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant_message</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_calls</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">tool_call</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_calls</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">tool_executor</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">execute_tool</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_call</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">name</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_call</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">parameters</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">observation</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-storage z-type z-string z-python">f</span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python">Observation: </span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-begin z-python">{</span><span class="z-source z-python z-embedded"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">format_result</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span></span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-end z-python">}</span></span><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">conversation_history</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">append</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Message</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">role</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">user<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">content</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">observation</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-continue z-python">continue</span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-else z-python"><span class="z-keyword z-control z-conditional z-else z-python">else</span><span class="z-punctuation z-section z-block z-conditional z-else z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">assistant_message</span></span> </span></code></pre> <p><strong>Pitfalls I encountered:</strong></p> <ul> <li>No exception handling—when LLM output format changes, it goes into infinite loops</li> <li>No tool parameter validation—LLM passing random parameters crashes the system</li> <li>Observation feedback too long or lacking useful information, making it hard for LLM to understand</li> </ul> <hr /> <h2 id="3-tool-definition-extending-llm-capabilities">3. Tool Definition: Extending LLM Capabilities<a class="zola-anchor" href="#3-tool-definition-extending-llm-capabilities" aria-label="Anchor link for: 3-tool-definition-extending-llm-capabilities" style="visibility: hidden;"></a> </h2> <p>Tools are the bridge between LLMs and the external world. Well-defined tools allow models to truly "get their hands dirty." My approach:</p> <ul> <li>All tool parameters validated with Pydantic—reject immediately if types don't match.</li> <li>Command-line tools have a blacklist (like <code>rm</code>, <code>del</code>)—block dangerous commands directly.</li> <li>Tool execution results returned in structured format—don't let the LLM guess.</li> <li>When operating command-line tools, both stdout and stderr are important—don't just give exit codes.</li> <li>Clear boundaries between tools to avoid confusion.</li> </ul> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">RunCommandTool</span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-punctuation z-section z-inheritance z-begin z-python">(</span></span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-entity z-other z-inherited-class z-python">Tool</span><span class="z-punctuation z-section z-inheritance z-end z-python">)</span></span><span class="z-meta z-class z-python"><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">DANGEROUS_COMMANDS</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-set z-python"><span class="z-punctuation z-section z-set z-begin z-python">{</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">rm<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-set z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">del<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-set z-end z-python">}</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">_validate_command</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">base_cmd</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Optional</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">base_cmd</span></span> <span class="z-keyword z-operator z-logical z-python">in</span> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-variable z-other z-constant z-python">DANGEROUS_COMMANDS</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-storage z-type z-string z-python">f</span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python">Dangerous command blocked: </span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-begin z-python">{</span><span class="z-source z-python z-embedded"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">base_cmd</span></span></span></span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-end z-python">}</span></span><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">execute</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">context</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Context</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ToolResult</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Validate command, execute and return results </span></span></code></pre> <hr /> <h2 id="4-prompt-design-making-llms-follow-the-script">4. Prompt Design: Making LLMs Follow the Script<a class="zola-anchor" href="#4-prompt-design-making-llms-follow-the-script" aria-label="Anchor link for: 4-prompt-design-making-llms-follow-the-script" style="visibility: hidden;"></a> </h2> <p>Well-organized prompts significantly improve LLM output quality. This is especially important for coding agents because LLM output formats can easily get out of control.</p> <p>My experience:</p> <ul> <li>List all tools and parameter schemas in the prompt.</li> <li>Provide several usage examples covering common scenarios.</li> <li>Keep observation feedback concise—don't pile on useless information.</li> </ul> <p><strong>Bub's ReAct Prompt Structure:</strong></p> <pre class="z-code"><code><span class="z-text z-plain">You are an AI assistant with access to tools. When you need to use a tool, follow this format: </span><span class="z-text z-plain"> </span><span class="z-text z-plain">Thought: ... </span><span class="z-text z-plain">Action: run_command </span><span class="z-text z-plain">Action Input: {&quot;command&quot;: &quot;ls&quot;} </span><span class="z-text z-plain">Observation: &lt;output&gt; </span><span class="z-text z-plain">... </span><span class="z-text z-plain">Final Answer: &lt;your answer&gt; </span></code></pre> <p>The advantage of ReAct is that it's simple and direct, easy to implement. Although token consumption is high, it allows for quick iteration and cross-model migration.</p> <hr /> <h2 id="5-milestone-automatic-type-annotation-fixing">5. Milestone: Automatic Type Annotation Fixing<a class="zola-anchor" href="#5-milestone-automatic-type-annotation-fixing" aria-label="Anchor link for: 5-milestone-automatic-type-annotation-fixing" style="visibility: hidden;"></a> </h2> <p>After defining the Agent Loop, tools, and prompts, Bub's first milestone was to make the agent automatically fix mypy errors.</p> <p>For example:</p> <pre data-lang="diff" class="language-diff z-code"><code class="language-diff" data-lang="diff"><span class="z-source z-diff"><span class="z-markup z-deleted z-diff"><span class="z-punctuation z-definition z-deleted z-diff">-</span> def __init__(self): </span></span><span class="z-source z-diff"><span class="z-markup z-deleted z-diff"><span class="z-punctuation z-definition z-deleted z-diff">-</span> self.console = Console() </span></span><span class="z-source z-diff"><span class="z-markup z-deleted z-diff"><span class="z-punctuation z-definition z-deleted z-diff">-</span> self._show_debug = False </span></span><span class="z-source z-diff"><span class="z-markup z-inserted z-diff"><span class="z-punctuation z-definition z-inserted z-diff">+</span> def __init__(self) -&gt; None: </span></span><span class="z-source z-diff"><span class="z-markup z-inserted z-diff"><span class="z-punctuation z-definition z-inserted z-diff">+</span> self.console: Console = Console() </span></span><span class="z-source z-diff"><span class="z-markup z-inserted z-diff"><span class="z-punctuation z-definition z-inserted z-diff">+</span> self._show_debug: bool = False </span></span></code></pre> <p>My approach:</p> <ul> <li>First use command-line tools to run <code>mypy</code> and get error information.</li> <li>Let the agent generate fix suggestions, then apply them to code using file edit tools.</li> <li>After fixing, automatically run mypy again to ensure no new issues.</li> <li>Only let the agent modify type annotations, formatting, and renaming—don't let it touch complex logic.</li> </ul> <p><strong>Pitfalls I encountered:</strong></p> <ul> <li>Models have limited understanding of code context and often fix things incorrectly.</li> <li>Without verification, bugs multiply instead of decreasing.</li> </ul> <hr /> <h2 id="6-advanced-techniques-and-engineering-self-review">6. Advanced Techniques and Engineering Self-Review<a class="zola-anchor" href="#6-advanced-techniques-and-engineering-self-review" aria-label="Anchor link for: 6-advanced-techniques-and-engineering-self-review" style="visibility: hidden;"></a> </h2> <p>As functionality expanded, I encountered more pitfalls. Some of my experiences:</p> <ul> <li>When extending tools, aim for single responsibility with clear parameter schemas.</li> <li>For long conversations, truncate or summarize—don't let LLM exceed context windows.</li> <li>In production environments, recommend sandboxing/virtual machines, and disable dangerous commands by default for command-line tools.</li> <li>When LLM output formats are unstable, have fallback mechanisms.</li> </ul> <h2 id="7-conclusion">7. Conclusion<a class="zola-anchor" href="#7-conclusion" aria-label="Anchor link for: 7-conclusion" style="visibility: hidden;"></a> </h2> <p>Bub is still in its early stages, but it can already help me automatically fix bugs, run commands, and read/write files. I hope this is just the beginning, and I look forward to more interesting feedback.</p> <p><img src="https://psiace.me/posts/baby-step-coding-agent/bub-it.jpeg" alt="Bub it. Build it." /></p> <hr /> <h3 id="references">References<a class="zola-anchor" href="#references" aria-label="Anchor link for: references" style="visibility: hidden;"></a> </h3> <ul> <li><a rel="nofollow noreferrer" href="https://ampcode.com/how-to-build-an-agent">How to Build an Agent (ampcode)</a></li> <li><a rel="nofollow noreferrer" href="https://huggingface.co/blog/tiny-agents">Tiny Agents: Building LLM-Powered Agents from Scratch (HuggingFace)</a></li> <li><a rel="nofollow noreferrer" href="https://github.com/PsiACE/bub">Bub: Bub it. Build it.</a></li> </ul> 2025-07-11T00:00:00+00:00 2025-07-11T00:00:00+00:00 Unknown https://psiace.me/posts/llama-index-cloudflare-ai-gateway/ <p>Single LLM providers are fragile. <a rel="nofollow noreferrer" href="https://status.openai.com/">OpenAI outage</a>? Your app dies. <a rel="nofollow noreferrer" href="https://status.anthropic.com/">Anthropic rate limits</a>? Users wait. It's annoying.</p> <p>I wanted multi-provider orchestration without the complexity. <a rel="nofollow noreferrer" href="https://developers.cloudflare.com/ai-gateway/">Cloudflare AI Gateway</a> looked perfect—automatic fallback, caching, load balancing. But no <a rel="nofollow noreferrer" href="https://docs.llamaindex.ai/">LlamaIndex</a> integration.</p> <p>So I built one.</p> <h2 id="the-problem">The Problem<a class="zola-anchor" href="#the-problem" aria-label="Anchor link for: the-problem" style="visibility: hidden;"></a> </h2> <p>Cloudflare AI Gateway is smart. It handles:</p> <ul> <li>Automatic provider fallback</li> <li>Built-in caching and rate limiting</li> <li>Load balancing across providers</li> <li>Unified API interface</li> </ul> <p>But LlamaIndex LLMs make direct HTTP requests to provider APIs. Cloudflare expects a different format.</p> <blockquote> <p>Cloudflare provides an <a rel="nofollow noreferrer" href="https://developers.cloudflare.com/ai-gateway/chat-completion/">OpenAI-compatible API</a>, but I wanted something more flexible. Compatible often means limited. And Cloudflare also provides a <a rel="nofollow noreferrer" href="https://developers.cloudflare.com/ai-gateway/integrations/vercel-ai-sdk/">Vercel ai-sdk integration</a>, but I save my life by using Python.</p> </blockquote> <h2 id="the-solution">The Solution<a class="zola-anchor" href="#the-solution" aria-label="Anchor link for: the-solution" style="visibility: hidden;"></a> </h2> <p>A wrapper that sits between LlamaIndex LLMs and their HTTP clients. Intercepts requests, transforms them for Cloudflare, handles responses.</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">cloudflare_ai_gateway</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">CloudflareAIGateway</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">openai</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">OpenAI</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">anthropic</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Anthropic</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">ChatMessage</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Create regular LlamaIndex LLMs </span></span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">openai_llm</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">OpenAI</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">gpt-4o-mini<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">api_key</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-key<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">anthropic_llm</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Anthropic</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">claude-3-5-sonnet<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">api_key</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-key<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Wrap with Cloudflare AI Gateway </span></span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">llm</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">CloudflareAIGateway</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">llms</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">openai_llm</span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">anthropic_llm</span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Try OpenAI first, then Anthropic </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">account_id</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-cloudflare-account-id<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">gateway</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-gateway-name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">api_key</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-cloudflare-api-key<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Use exactly like any LlamaIndex LLM </span></span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">messages</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">ChatMessage</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">role</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">user<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">content</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">What is 2+2?<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">llm</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">chat</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">messages</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>Drop-in replacement. Zero code changes.</p> <h2 id="what-it-does">What It Does<a class="zola-anchor" href="#what-it-does" aria-label="Anchor link for: what-it-does" style="visibility: hidden;"></a> </h2> <p><strong>Core features:</strong></p> <ul> <li>Automatic provider detection and configuration</li> <li>Built-in fallback when providers fail</li> <li>Streaming support (chat and completion)</li> <li>Async/await compatible</li> </ul> <p><strong>Tested providers:</strong></p> <ul> <li><a rel="nofollow noreferrer" href="https://platform.openai.com/">OpenAI</a>, <a rel="nofollow noreferrer" href="https://www.anthropic.com/">Anthropic</a></li> </ul> <p><strong>Also supported:</strong></p> <ul> <li><a rel="nofollow noreferrer" href="https://aistudio.google.com/">Google AI Studio</a>, <a rel="nofollow noreferrer" href="https://platform.deepseek.com/">DeepSeek</a>, <a rel="nofollow noreferrer" href="https://x.ai/">Grok</a></li> <li><a rel="nofollow noreferrer" href="https://azure.microsoft.com/en-us/products/ai-services/openai-service">Azure OpenAI</a>, <a rel="nofollow noreferrer" href="https://www.perplexity.ai/">Perplexity</a>, <a rel="nofollow noreferrer" href="https://replicate.com/">Replicate</a></li> </ul> <h2 id="try-it">Try It<a class="zola-anchor" href="#try-it" aria-label="Anchor link for: try-it" style="visibility: hidden;"></a> </h2> <p>Still a planned PR (<a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index/pull/19395">#19395</a>), but functional:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">git</span></span><span class="z-meta z-function-call z-arguments z-shell"> clone <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>repository-url<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> </span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-support z-function z-cd z-shell">cd</span></span><span class="z-meta z-function-call z-arguments z-shell"> llama-index-llms-cloudflare-ai-gateway</span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">pip</span></span><span class="z-meta z-function-call z-arguments z-shell"> install<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> -</span>e</span> .</span> </span><span class="z-source z-shell z-bash"> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-storage z-modifier z-shell">export</span> <span class="z-variable z-other z-readwrite z-assignment z-shell">OPENAI_API_KEY</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>your-key<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span></span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-storage z-modifier z-shell">export</span> <span class="z-variable z-other z-readwrite z-assignment z-shell">ANTHROPIC_API_KEY</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>your-key<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span></span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-storage z-modifier z-shell">export</span> <span class="z-variable z-other z-readwrite z-assignment z-shell">CLOUDFLARE_ACCOUNT_ID</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>your-id<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span></span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-storage z-modifier z-shell">export</span> <span class="z-variable z-other z-readwrite z-assignment z-shell">CLOUDFLARE_API_KEY</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>your-key<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span></span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-storage z-modifier z-shell">export</span> <span class="z-variable z-other z-readwrite z-assignment z-shell">CLOUDFLARE_GATEWAY</span><span class="z-keyword z-operator z-assignment z-shell">=</span><span class="z-string z-unquoted z-shell"><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>your-gateway<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span></span></span> </span><span class="z-source z-shell z-bash"> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">uv</span></span><span class="z-meta z-function-call z-arguments z-shell"> run pytest tests/</span> </span></code></pre> <p>May not be production-ready, but good enough to experiment with. Check out the <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index/tree/main/llama-index-integrations/llms">LlamaIndex integrations repository</a> for other LLM providers.</p> <hr /> <h2 id="references">References<a class="zola-anchor" href="#references" aria-label="Anchor link for: references" style="visibility: hidden;"></a> </h2> <ul> <li><a rel="nofollow noreferrer" href="https://developers.cloudflare.com/ai-gateway/">Cloudflare AI Gateway Documentation</a></li> <li><a rel="nofollow noreferrer" href="https://docs.llamaindex.ai/">LlamaIndex Documentation</a></li> <li><a rel="nofollow noreferrer" href="https://docs.llamaindex.ai/en/stable/module_guides/models/llms/">LlamaIndex LLMs Guide</a></li> </ul> 2025-07-08T00:00:00+00:00 2025-07-08T00:00:00+00:00 Unknown https://psiace.me/posts/wasmtime-executor-for-code-agents/ <p>AI agents that execute code are powerful but terrifying. Within minutes of testing my first code agent, it was trying to install packages and access environment variables I never intended. <strong>We need sandboxing.</strong></p> <h2 id="the-problem">The Problem<a class="zola-anchor" href="#the-problem" aria-label="Anchor link for: the-problem" style="visibility: hidden;"></a> </h2> <p>Code-executing AI agents can do amazing things—analyze data, solve math problems, debug code. But they also operate with zero fear of breaking production systems.</p> <p>Consider these scenarios:</p> <ul> <li><strong>The curious agent</strong>: "Let me check what's in <code>/etc/passwd</code>..."</li> <li><strong>The helpful agent</strong>: "I'll install this package" → <code>pip install</code> from compromised PyPI</li> <li><strong>The infinite loop</strong>: Code that consumes all available memory</li> </ul> <p>Some projects like <a rel="nofollow noreferrer" href="https://github.com/huggingface/smolagents/blob/main/src/smolagents/remote_executors.py">Hugging Face smolagents</a> recognize this—they built multiple executor options to balance capability and security.</p> <h2 id="why-webassembly">Why WebAssembly?<a class="zola-anchor" href="#why-webassembly" aria-label="Anchor link for: why-webassembly" style="visibility: hidden;"></a> </h2> <p>WebAssembly's superpower isn't speed—it's <strong>security by design</strong>. Born from browser security, WASM provides:</p> <ol> <li><strong>Capability-based security</strong>: Can't do anything unless explicitly granted</li> <li><strong>Resource metering</strong>: Count every CPU cycle and memory allocation</li> <li><strong>Deterministic execution</strong>: Same code, same result every time</li> <li><strong>Language agnostic</strong>: Python today, others tomorrow</li> </ol> <p>Simon Willison explored this in his <a rel="nofollow noreferrer" href="https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox">TIL post</a>, and <a rel="nofollow noreferrer" href="https://news.ycombinator.com/item?id=34581487">Hacker News discussed it</a>. While <a rel="nofollow noreferrer" href="https://github.com/huggingface/smolagents">Hugging Face smolagents</a> already has a <code>WasmExecutor</code> using Pyodide and Deno, I wanted to explore a different approach using wasmtime for local execution.</p> <h2 id="a-weekend-hack">A Weekend Hack<a class="zola-anchor" href="#a-weekend-hack" aria-label="Anchor link for: a-weekend-hack" style="visibility: hidden;"></a> </h2> <p>So I built one. Nothing fancy—just a <strong>"just do it"</strong> demo:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">wasmtime_executor</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">WasmtimePythonExecutor</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">smolagents</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">CodeAgent</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">InferenceClientModel</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">WasmtimeCodeAgent</span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-punctuation z-section z-inheritance z-begin z-python">(</span></span></span><span class="z-meta z-class z-inheritance z-python"><span class="z-entity z-other z-inherited-class z-python">CodeAgent</span><span class="z-punctuation z-section z-inheritance z-end z-python">)</span></span><span class="z-meta z-class z-python"><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">create_python_executor</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">WasmtimePythonExecutor</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">additional_authorized_imports</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">additional_authorized_imports</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">max_print_outputs_length</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">max_print_outputs_length</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">executor_kwargs</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">WasmtimeCodeAgent</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">tools</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-sequence z-list z-empty z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">InferenceClientModel</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">additional_authorized_imports</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">math<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">json<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">run</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Calculate the square root of 125<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>Implementation: grab VMware's Python WASM binary, wrap with wasmtime-py, make it compatible with smolagents. The key difference? <strong>Local execution</strong> without needing Deno runtime.</p> <h2 id="what-i-learned">What I Learned<a class="zola-anchor" href="#what-i-learned" aria-label="Anchor link for: what-i-learned" style="visibility: hidden;"></a> </h2> <p><strong>What worked:</strong></p> <ul> <li>Drop-in compatibility with smolagents</li> <li>Basic Python operations run fine</li> <li>Error handling is robust</li> <li>State persists between executions</li> <li><strong>Local execution</strong> without external dependencies</li> </ul> <p><strong>Trade-offs:</strong></p> <ul> <li>Real isolation vs. limited Python ecosystem</li> <li>Better security vs. worse performance</li> <li>Deterministic vs. complex setup</li> <li><strong>Local control</strong> vs. remote calls</li> </ul> <h2 id="is-this-production-ready">Is This Production Ready?<a class="zola-anchor" href="#is-this-production-ready" aria-label="Anchor link for: is-this-production-ready" style="visibility: hidden;"></a> </h2> <p><strong>Absolutely not.</strong> This is a weekend hack. But it proves the approach works.</p> <p>For production you'd need: better resource management, package management, performance optimization, proper logging, security auditing.</p> <p>But for experimenting? Understanding trade-offs? Having a concrete example? It works perfectly.</p> <h2 id="the-point">The Point<a class="zola-anchor" href="#the-point" aria-label="Anchor link for: the-point" style="visibility: hidden;"></a> </h2> <p>The real value isn't this specific implementation—it's demonstrating that <strong>different sandboxing approaches exist for AI code execution</strong>. While smolagents offers Pyodide-based sandboxing with Deno, wasmtime provides an alternative for local execution. You can take an existing agent framework, add security boundaries, maintain functionality, and explore different trade-offs.</p> <h2 id="try-it">Try It<a class="zola-anchor" href="#try-it" aria-label="Anchor link for: try-it" style="visibility: hidden;"></a> </h2> <p>The complete implementation is in my <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/wasmtime-executor">Wasmtime Executor demo</a>. It's rough but works. Clone it, break it, improve it.</p> <p>The best way to understand these security challenges is to build something. Even if it's just a quick weekend hack.</p> <hr /> <h2 id="references">References<a class="zola-anchor" href="#references" aria-label="Anchor link for: references" style="visibility: hidden;"></a> </h2> <ul> <li><a rel="nofollow noreferrer" href="https://github.com/huggingface/smolagents">Hugging Face smolagents</a></li> <li><a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/wasmtime-executor">Wasmtime Executor Demo</a></li> <li><a rel="nofollow noreferrer" href="https://til.simonwillison.net/webassembly/python-in-a-wasm-sandbox">Simon Willison's WASM Python Sandbox</a></li> <li><a rel="nofollow noreferrer" href="https://github.com/vmware-labs/webassembly-language-runtimes">VMware WebAssembly Language Runtimes</a></li> </ul> 2025-06-18T00:00:00+00:00 2025-06-18T00:00:00+00:00 Unknown https://psiace.me/posts/rag-in-coding-agent/ <p>Programming can be tough. Whether you're debugging a tricky error, trying to understand a massive codebase, or figuring out how to use a new framework, there's always something that slows you down. Wouldn't it be great if there was a tool that could help you with all of this—something that could act like a super-smart assistant, guiding you through the process?</p> <p>That's where <strong>RAG</strong> (Retrieval-Augmented Generation) comes in. It's a fancy name for a simple idea: combining the ability to look things up (retrieval) with the ability to create new solutions (generation). In this article, we'll break down how RAG works, why it's useful for coding assistants, and how it can make your life as a developer easier.</p> <hr /> <h2 id="what-is-rag">What is RAG?<a class="zola-anchor" href="#what-is-rag" aria-label="Anchor link for: what-is-rag" style="visibility: hidden;"></a> </h2> <p>Imagine you're trying to write code for a feature you've never built before. You might:</p> <ul> <li>Look up examples online.</li> <li>Read the documentation for the framework you're using.</li> <li>Search through your codebase for similar functionality.</li> <li>Check StackOverflow for solutions to common problems.</li> </ul> <p>RAG does all of this for you, automatically. It's like having a supercharged search engine that not only finds the information you need but also uses it to help you write code. Here's how it works:</p> <ol> <li><strong>Retrieval</strong>: It finds relevant information—like code snippets, documentation, or answers from Q&amp;A sites.</li> <li><strong>Generation</strong>: It uses that information to create something new, like a piece of code or a solution to a problem.</li> </ol> <p>By combining these two steps, RAG makes coding assistants smarter and more helpful.</p> <hr /> <h2 id="why-do-we-need-coding-assistants">Why Do We Need Coding Assistants?<a class="zola-anchor" href="#why-do-we-need-coding-assistants" aria-label="Anchor link for: why-do-we-need-coding-assistants" style="visibility: hidden;"></a> </h2> <p>Let's face it: programming isn't just about writing code. A lot of your time is spent on things like:</p> <ul> <li>Figuring out how existing code works.</li> <li>Searching for the right file or module in a huge codebase.</li> <li>Debugging errors and trying to understand cryptic error messages.</li> <li>Learning new tools, frameworks, or libraries.</li> </ul> <p><img src="https://psiace.me/posts/rag-in-coding-agent/swe-bench.png" alt="SWE Benchmark" /></p> <p>These tasks can be frustrating and time-consuming. Coding assistants aim to take care of the boring, repetitive stuff so you can focus on the creative parts of programming. But to do this well, they need to be smart enough to understand your code, find the right information, and even help you fix mistakes. That's where RAG comes in.</p> <hr /> <h2 id="how-does-rag-help-coding-assistants">How Does RAG Help Coding Assistants?<a class="zola-anchor" href="#how-does-rag-help-coding-assistants" aria-label="Anchor link for: how-does-rag-help-coding-assistants" style="visibility: hidden;"></a> </h2> <p>RAG can make coding assistants better at handling the most common challenges developers face.</p> <p><img src="https://psiace.me/posts/rag-in-coding-agent/coderag-bench.png" alt="CodeRAG Benchmark" /></p> <p>Here are a few examples:</p> <h3 id="1-understanding-your-task">1. <strong>Understanding Your Task</strong><a class="zola-anchor" href="#1-understanding-your-task" aria-label="Anchor link for: 1-understanding-your-task" style="visibility: hidden;"></a> </h3> <p>Let's say you tell the assistant: "I need to add user authentication to my API." The assistant needs to figure out:</p> <ul> <li>What kind of authentication you want (e.g., JWT, OAuth).</li> <li>Which parts of your codebase are relevant.</li> <li>What libraries or frameworks you're using.</li> </ul> <p>RAG helps by retrieving documentation, code examples, and related information to build a clear picture of what you're asking for.</p> <h3 id="2-finding-the-right-files">2. <strong>Finding the Right Files</strong><a class="zola-anchor" href="#2-finding-the-right-files" aria-label="Anchor link for: 2-finding-the-right-files" style="visibility: hidden;"></a> </h3> <p>If you're working on a big project, finding the right file or module can feel like searching for a needle in a haystack. RAG can analyze your codebase and quickly point you to the files that matter. For example:</p> <ul> <li>"Where is the login functionality implemented?"</li> <li>"Which module handles database connections?"</li> </ul> <p>By mapping out your codebase, RAG makes navigation much easier.</p> <h3 id="3-writing-code">3. <strong>Writing Code</strong><a class="zola-anchor" href="#3-writing-code" aria-label="Anchor link for: 3-writing-code" style="visibility: hidden;"></a> </h3> <p>Once the assistant understands your task, it can use RAG to generate code that fits your needs. For example:</p> <ul> <li>"Add JWT authentication to my API."</li> <li>"Write a function to validate user input."</li> </ul> <p>RAG pulls in examples, documentation, and best practices to make sure the generated code is accurate and useful.</p> <h3 id="4-debugging-errors">4. <strong>Debugging Errors</strong><a class="zola-anchor" href="#4-debugging-errors" aria-label="Anchor link for: 4-debugging-errors" style="visibility: hidden;"></a> </h3> <p>When something goes wrong, RAG can help you figure out what happened. It can:</p> <ul> <li>Analyze error messages and logs.</li> <li>Search Q&amp;A platforms like StackOverflow for similar issues.</li> <li>Suggest fixes based on what it finds.</li> </ul> <p>This saves you from spending hours Googling error codes and trying random solutions.</p> <hr /> <h2 id="the-biggest-challenge-when-to-retrieve">The Biggest Challenge: When to Retrieve?<a class="zola-anchor" href="#the-biggest-challenge-when-to-retrieve" aria-label="Anchor link for: the-biggest-challenge-when-to-retrieve" style="visibility: hidden;"></a> </h2> <p>One tricky part of using RAG is deciding <strong>when to look things up</strong>. If the assistant retrieves information too early, it might miss important context. If it waits too long, it might slow down the process. The key is to strike a balance:</p> <ul> <li>Retrieve early for general knowledge, like framework documentation or coding rules.</li> <li>Retrieve on-demand for specific problems, like debugging an error or answering a user query.</li> </ul> <p>This dynamic approach ensures the assistant is both fast and accurate.</p> <hr /> <h2 id="common-misunderstandings-about-rag">Common Misunderstandings About RAG<a class="zola-anchor" href="#common-misunderstandings-about-rag" aria-label="Anchor link for: common-misunderstandings-about-rag" style="visibility: hidden;"></a> </h2> <p>Some people think RAG is just about using fancy AI models to search for information. But it's more than that. RAG isn't just about finding stuff—it's about combining different types of knowledge to solve problems. For example:</p> <ul> <li>It can use APIs to query Q&amp;A platforms like StackOverflow.</li> <li>It can analyze runtime logs to figure out what went wrong during execution.</li> <li>It can pull together code snippets, documentation, and examples to create a complete solution.</li> </ul> <p>RAG is like having a super-organized assistant who knows where to look and how to combine information to help you.</p> <hr /> <h2 id="how-to-build-a-rag-enhanced-coding-workflow">How to Build a RAG-Enhanced Coding Workflow<a class="zola-anchor" href="#how-to-build-a-rag-enhanced-coding-workflow" aria-label="Anchor link for: how-to-build-a-rag-enhanced-coding-workflow" style="visibility: hidden;"></a> </h2> <p>Here's a simple step-by-step process for how RAG can be used in a coding assistant:</p> <pre data-lang="mermaid" class="language-mermaid z-code"><code class="language-mermaid" data-lang="mermaid"><span class="z-text z-plain">graph LR </span><span class="z-text z-plain"> subgraph TaskInit [&quot;1. Task Understanding&quot;] </span><span class="z-text z-plain"> direction TB </span><span class="z-text z-plain"> A[&quot;Parse Your Request&lt;br/&gt;What do you want to build?&quot;] </span><span class="z-text z-plain"> A --&gt; B[&quot;Initialize Environment&lt;br/&gt;Set up workspace&quot;] </span><span class="z-text z-plain"> B --&gt; C[&quot;Map Your Codebase&lt;br/&gt;Find relevant files&quot;] </span><span class="z-text z-plain"> B --&gt; D[&quot;Bootstrap Context&lt;br/&gt;Load coding rules &amp; docs&quot;] </span><span class="z-text z-plain"> end </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> subgraph ContextBuild [&quot;2. Gather Information&quot;] </span><span class="z-text z-plain"> direction TB </span><span class="z-text z-plain"> E[&quot;Find Documentation&lt;br/&gt;Framework guides &amp; tutorials&quot;] </span><span class="z-text z-plain"> E --&gt; F[&quot;Search Code Examples&lt;br/&gt;Similar solutions in your project&quot;] </span><span class="z-text z-plain"> F --&gt; G[&quot;Query Q&amp;A Platforms&lt;br/&gt;StackOverflow &amp; GitHub repos&quot;] </span><span class="z-text z-plain"> G --&gt; H[&quot;Build Knowledge Soup&lt;br/&gt;Combine all the info&quot;] </span><span class="z-text z-plain"> end </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> subgraph CodeWork [&quot;3. Write &amp; Test Code&quot;] </span><span class="z-text z-plain"> direction TB </span><span class="z-text z-plain"> I[&quot;Generate Code&lt;br/&gt;Using all gathered context&quot;] </span><span class="z-text z-plain"> I --&gt; J[&quot;Run in Sandbox&lt;br/&gt;Test safely in isolation&quot;] </span><span class="z-text z-plain"> J --&gt; K[&quot;Capture Results&lt;br/&gt;Success or error feedback&quot;] </span><span class="z-text z-plain"> end </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> subgraph HandleFeedback [&quot;4. Handle Results&quot;] </span><span class="z-text z-plain"> direction TB </span><span class="z-text z-plain"> L[&quot;Task Complete!&lt;br/&gt;Code works as expected&quot;] </span><span class="z-text z-plain"> M[&quot;Debug Errors&lt;br/&gt;Find solutions for failures&quot;] </span><span class="z-text z-plain"> end </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> subgraph Finish [&quot;5. Wrap Up&quot;] </span><span class="z-text z-plain"> direction TB </span><span class="z-text z-plain"> N[&quot;Deliver Final Code&lt;br/&gt;Plus documentation&quot;] </span><span class="z-text z-plain"> end </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> %% Main flow connections </span><span class="z-text z-plain"> TaskInit --&gt; ContextBuild </span><span class="z-text z-plain"> ContextBuild --&gt; CodeWork </span><span class="z-text z-plain"> CodeWork --&gt; HandleFeedback </span><span class="z-text z-plain"> K --&gt;|&quot;✅ Success&quot;| L </span><span class="z-text z-plain"> K --&gt;|&quot;❌ Error&quot;| M </span><span class="z-text z-plain"> M -.-&gt;|&quot;Try again with fixes&quot;| ContextBuild </span><span class="z-text z-plain"> L --&gt; Finish </span><span class="z-text z-plain"> </span><span class="z-text z-plain"> classDef ragStyle fill:#e1f5fe,stroke:#01579b,stroke-width:2px </span><span class="z-text z-plain"> class D,E,F,G,H,I,M ragStyle </span></code></pre> <h3 id="1-understanding-what-you-want">1. <strong>Understanding What You Want</strong><a class="zola-anchor" href="#1-understanding-what-you-want" aria-label="Anchor link for: 1-understanding-what-you-want" style="visibility: hidden;"></a> </h3> <p>The assistant starts by figuring out exactly what you're trying to accomplish. Let's say you tell it: "I need to add user authentication to my API." It needs to understand:</p> <ul> <li>What type of authentication you want (JWT tokens? OAuth? Simple passwords?)</li> <li>Which parts of your existing code are relevant</li> <li>What frameworks and libraries you're already using</li> </ul> <p>This is where RAG kicks in—it retrieves documentation and examples to build a clear picture of your request.</p> <h3 id="2-gathering-all-the-pieces">2. <strong>Gathering All the Pieces</strong><a class="zola-anchor" href="#2-gathering-all-the-pieces" aria-label="Anchor link for: 2-gathering-all-the-pieces" style="visibility: hidden;"></a> </h3> <p>Now comes the information-gathering phase. The assistant acts like a super-efficient researcher, pulling together:</p> <ul> <li><strong>Documentation</strong>: Official guides for the frameworks you're using</li> <li><strong>Code snippets</strong>: Similar functionality from your project or other open-source projects</li> <li><strong>Q&amp;A knowledge</strong>: Solutions from StackOverflow, GitHub discussions, and other developer communities</li> <li><strong>Best practices</strong>: Coding standards and security guidelines</li> </ul> <p>All of this gets combined into what we call a "knowledge soup"—a rich mix of information that gives the assistant everything it needs to help you.</p> <h3 id="3-writing-and-testing-the-code">3. <strong>Writing and Testing the Code</strong><a class="zola-anchor" href="#3-writing-and-testing-the-code" aria-label="Anchor link for: 3-writing-and-testing-the-code" style="visibility: hidden;"></a> </h3> <p>With all that context in hand, the assistant can now generate code that actually makes sense for your specific situation. But it doesn't stop there—it runs the code in a safe sandbox environment to make sure it works. This is crucial because:</p> <ul> <li>It catches errors before they hit your main codebase</li> <li>It validates that the solution actually solves your problem</li> <li>It provides immediate feedback for refinement</li> </ul> <h3 id="4-learning-from-success-and-failure">4. <strong>Learning from Success and Failure</strong><a class="zola-anchor" href="#4-learning-from-success-and-failure" aria-label="Anchor link for: 4-learning-from-success-and-failure" style="visibility: hidden;"></a> </h3> <p>Here's where RAG really shines. If the code works perfectly, great! But if something goes wrong, the assistant doesn't give up. Instead, it:</p> <ul> <li>Analyzes the error messages and logs</li> <li>Searches for solutions to similar problems</li> <li>Retrieves additional context that might help fix the issue</li> <li>Tries again with the new information</li> </ul> <p>This feedback loop is what makes RAG-powered assistants so much more reliable than simple code generators.</p> <h3 id="5-delivering-the-final-solution">5. <strong>Delivering the Final Solution</strong><a class="zola-anchor" href="#5-delivering-the-final-solution" aria-label="Anchor link for: 5-delivering-the-final-solution" style="visibility: hidden;"></a> </h3> <p>Once everything is working, the assistant packages up the solution for you—not just the code, but also documentation explaining how it works and how to maintain it.</p> <hr /> <h2 id="balancing-long-term-and-short-term-knowledge">Balancing Long-Term and Short-Term Knowledge<a class="zola-anchor" href="#balancing-long-term-and-short-term-knowledge" aria-label="Anchor link for: balancing-long-term-and-short-term-knowledge" style="visibility: hidden;"></a> </h2> <p>Not all information is equally important. Some things need to be stored for long-term use (like framework rules), while others are only relevant for the current task (like runtime logs).</p> <p><img src="https://psiace.me/posts/rag-in-coding-agent/knowledge-soup.png" alt="Knowledge Soup" /></p> <p>Here's how RAG handles this:</p> <table><thead><tr><th><strong>Type of Knowledge</strong></th><th><strong>How It's Stored</strong></th><th><strong>How It's Retrieved</strong></th></tr></thead><tbody> <tr><td>Predefined Rules</td><td>Stored permanently</td><td>Quick lookup via a knowledge graph</td></tr> <tr><td>Documentation &amp; Tutorials</td><td>Stored with versioning</td><td>Semantic search for relevant sections</td></tr> <tr><td>Q&amp;A Platforms &amp; Codebases</td><td>Temporary -&gt; Permanent</td><td>API queries + caching for reuse</td></tr> <tr><td>Runtime Feedback</td><td>Temporary (task-specific)</td><td>Immediate retrieval and dynamic updates</td></tr> </tbody></table> <p>This balance ensures the assistant is efficient and adaptable.</p> <hr /> <h2 id="a-simple-way-to-get-started">A Simple Way to Get Started<a class="zola-anchor" href="#a-simple-way-to-get-started" aria-label="Anchor link for: a-simple-way-to-get-started" style="visibility: hidden;"></a> </h2> <p>If you're curious about building a coding assistant but don't want to dive into complex AI systems, here's a lightweight approach:</p> <p><img src="https://psiace.me/posts/rag-in-coding-agent/swe-agent.png" alt="SWE Agent" /></p> <ol> <li>Use predefined rules and prompts to guide the assistant.</li> <li>Retrieve documentation using local tools or web searches.</li> <li>Query Q&amp;A platforms manually or with simple scripts.</li> <li>Capture runtime feedback using basic tools like <code>grep</code> or <code>lint</code>.</li> </ol> <p>This setup covers the basics of a coding assistant and gives you a foundation to build on.</p> <hr /> <h2 id="final-thoughts">Final Thoughts<a class="zola-anchor" href="#final-thoughts" aria-label="Anchor link for: final-thoughts" style="visibility: hidden;"></a> </h2> <p>RAG is changing the way we approach programming. By combining retrieval and generation, it enables coding assistants to understand tasks, navigate codebases, write solutions, and debug errors—all while adapting to your needs. Whether you're a beginner or an experienced developer, tools powered by RAG can make your workflow faster and less frustrating.</p> <p>If you're interested in building your own coding assistant or just want to learn more, start small and experiment. The possibilities are endless, and the best way to learn is by trying things out. Who knows? You might just create the next big thing in developer tools!</p> <hr /> <h2 id="references">References<a class="zola-anchor" href="#references" aria-label="Anchor link for: references" style="visibility: hidden;"></a> </h2> <ul> <li>https://arxiv.org/html/2310.06770v3</li> <li>https://arxiv.org/pdf/2405.15793</li> <li>https://arxiv.org/html/2406.14497v1</li> <li>https://arxiv.org/html/2402.12317v2</li> <li>https://github.com/antvis/GPT-Vis</li> <li>https://www.phontron.com/slides/neubig24softwareagents.pdf</li> </ul> 2025-04-17T00:00:00+00:00 2025-04-17T00:00:00+00:00 Unknown https://psiace.me/posts/rag-is-the-way/ <p>Last year, I had to leave the database infrastructure field (if you remember some of my previous voices) and dive into RAG application development. At the time, I thought I was finally going to break free from the tedious low-level tech stack and do something "more high-level" in applications. The result? Reality gave me a resounding slap in the face.</p> <h2 id="you-can-never-escape-the-infrastructure-curse">You Can Never Escape the Infrastructure Curse<a class="zola-anchor" href="#you-can-never-escape-the-infrastructure-curse" aria-label="Anchor link for: you-can-never-escape-the-infrastructure-curse" style="visibility: hidden;"></a> </h2> <p>Look around: everyone's talking about ChatGPT, talking about all sorts of fancy AI applications. Product managers are excitedly drawing various chat boxes, designers are dancing with joy over natural language interactions. It's like everyone's saying: Hey, look, we can finally throw away those boring technical details and just talk to AI directly!</p> <p>Bullshit.</p> <p>Anyone who's actually done RAG knows this thing is just infrastructure engineering wearing an AI costume. And it's way more complex than traditional databases. This field hasn't fully developed yet, so we see many people trying to conflate methodology with implementation details.</p> <p>Every day I wonder: did I escape old problems, or did I just earn the privilege of new ones?</p> <h2 id="rag-is-not-just-retrieval-augmented-generation">RAG is Not Just Retrieval-Augmented Generation<a class="zola-anchor" href="#rag-is-not-just-retrieval-augmented-generation" aria-label="Anchor link for: rag-is-not-just-retrieval-augmented-generation" style="visibility: hidden;"></a> </h2> <h3 id="retrieval-not-as-simple-as-just-finding-stuff">Retrieval: Not as Simple as Just Finding Stuff<a class="zola-anchor" href="#retrieval-not-as-simple-as-just-finding-stuff" aria-label="Anchor link for: retrieval-not-as-simple-as-just-finding-stuff" style="visibility: hidden;"></a> </h3> <p>Retrieval should be the part we're most familiar with, right? After all, we've been doing SQL and full-text search for decades.</p> <p>Wrong.</p> <p>Retrieval in RAG has mutated into a completely different beast. Now we're dealing with proximity in semantic space, cosine similarity between embedding vectors, finding nearest neighbors in high-dimensional space.</p> <p>I once thought I had mastered the art of query optimization. Now? I'm debugging why the same question gets different retrieval results in summer versus winter. Why? Because language models' semantic understanding can be affected by time, context, and even seasonal biases in training data. What the hell?</p> <p>Retrieval is no longer just about finding the right records. It's about understanding the real intent behind questions, extracting signal from noise, finding clear answers in fuzzy semantic space.</p> <h3 id="agent-thinking-tools-or-if-else-in-ai-clothing">Agent: Thinking Tools or IF-ELSE in AI Clothing?<a class="zola-anchor" href="#agent-thinking-tools-or-if-else-in-ai-clothing" aria-label="Anchor link for: agent-thinking-tools-or-if-else-in-ai-clothing" style="visibility: hidden;"></a> </h3> <p>If retrieval is RAG's brain, then intelligent agents are its hands and feet. In the past, we built passively responsive systems; now, we're creating ecosystems that actively think and act.</p> <p>The Agent paradigm has become the new hotness. Why? Because just returning information isn't enough anymore. Users don't want data, they want answers; not options, they want decisions; not possibilities, they want certainty.</p> <p>We orchestrate complex Agent Workflows, share tool ecosystems with MCP, create specialized thinking agents, search agents, computation agents, planning agents. These intelligent agents also need standard protocols to interact and collaborate. Projects like IBM's beeAI are trying to solve this problem.</p> <p>But honestly, most so-called "intelligent" agents are still if-else monsters under preset paths. True agent intelligence would think independently, but we haven't really reached that step yet. We've just written more conditional statements in more complex ways.</p> <p>Except this time, the conditional statements are dynamically generated by language models. This is what we call progress.</p> <h3 id="grounding-truth-the-connection-point-with-the-real-world">Grounding Truth: The Connection Point with the Real World<a class="zola-anchor" href="#grounding-truth-the-connection-point-with-the-real-world" aria-label="Anchor link for: grounding-truth-the-connection-point-with-the-real-world" style="visibility: hidden;"></a> </h3> <p>Ultimately, all these fancy technologies must return to a simple question: is it correct?</p> <p>Grounding truth is RAG's value anchor. If the systems we build can't provide accurate, verifiable information consistent with the real world, then it's no different from a randomly babbling drunk.</p> <p>Human In The Loop isn't an option, it's a necessity. AI hallucination isn't a technical problem, it's a philosophical one: when there's no human oversight, who defines what's real?</p> <p>We need to continuously accumulate grounded factual foundations, need human experts to participate continuously, need to establish feedback loops to correct errors and biases. This isn't just about improving accuracy, but about giving these systems real value and meaning.</p> <h2 id="new-layers-of-complexity">New Layers of Complexity<a class="zola-anchor" href="#new-layers-of-complexity" aria-label="Anchor link for: new-layers-of-complexity" style="visibility: hidden;"></a> </h2> <p>Let me be straight: RAG is ten times more complex than traditional databases.</p> <p>When dealing with databases before, we at least knew what the data looked like. Now? PDFs, images, videos, audio, 3D models, even IoT sensor data - any garbage gets thrown in. What's more fucked up is that all this heterogeneous data has to maintain semantic consistency.</p> <p>When doing query optimization before, we cared about index selection and execution plans. Now? We have to consider vector similarity, semantic relevance, multimodal transformations, and worry about whether the large model will suddenly go crazy and return a bunch of nonsense.</p> <p>When ensuring data consistency before, we had ACID. Now? Knowledge evolves in real-time, concepts dynamically reconstruct, even the word "consistency" itself becomes ambiguous.</p> <p>My engineer brain is experiencing unprecedented challenges, and I'm actually fucking enjoying this process.</p> <h2 id="why-am-i-still-in-this-pit">Why Am I Still in This Pit?<a class="zola-anchor" href="#why-am-i-still-in-this-pit" aria-label="Anchor link for: why-am-i-still-in-this-pit" style="visibility: hidden;"></a> </h2> <p>Why am I still enduring this complexity? Because it's too damn interesting.</p> <p>We're not building a simple chatbot. We're redefining how knowledge is acquired, processed, and applied:</p> <ul> <li>We're building information systems that understand human intent</li> <li>We're designing learning ecosystems that can self-adapt</li> <li>We're creating intelligent agents that can reason and act</li> <li>We're trying to connect abstract language understanding with the concrete real world</li> </ul> <p>This is way more exciting than building a database, an API, or an application. This is being part of creating the next generation of computing infrastructure.</p> <h2 id="layers-are-illusion-infrastructure-is-eternal">Layers are Illusion, Infrastructure is Eternal<a class="zola-anchor" href="#layers-are-illusion-infrastructure-is-eternal" aria-label="Anchor link for: layers-are-illusion-infrastructure-is-eternal" style="visibility: hidden;"></a> </h2> <p>After this year of RAG development, I finally get it: there's no such thing as "higher-level" or "lower-level" technology. The so-called hierarchical distinction is just a placebo we give ourselves.</p> <p>No matter how high you go, you'll eventually find yourself dealing with infrastructure problems. It's just that this time the infrastructure is more complex, more abstract, more closely related to human cognition.</p> <p>When you really want to build something valuable, you ultimately have to return to infrastructure. Because that's the foundation of everything.</p> <p>So when someone asks me: "Why did you switch from databases to AI applications?"</p> <p>I'll say: "I never switched at all. I just changed the way I solve the same problems. These problems now have a better name: the trinity of retrieval, intelligent agents, and grounding truth."</p> <p>Now, excuse me, I need to deal with some damn vector quantization issues, educate a few confused intelligent agents, and make sure my text splitting strategy doesn't tear facts to pieces. This is what real RAG work looks like.</p> 2025-03-11T00:00:00+00:00 2025-03-11T00:00:00+00:00 Unknown https://psiace.me/posts/announcing-nebulagraph-mcp-server-v010/ <p>Over the past few weeks, <strong>Model Context Protocol (MCP)</strong> has sparked significant discussion and attention within the tech community. As part of this momentum, I contributed to several related open-source projects, including the <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index/pull/17795">McpToolSpec implementation for the LlamaIndex official repository</a> and collaborated with Xuanwo and FrostMing to develop a version of the <a rel="nofollow noreferrer" href="https://github.com/Xuanwo/mcp-server-opendal">Model Context Protocol Server for Apache OpenDAL™</a>.</p> <p>MCP is currently the only standard with an ecosystem advantage. In this context, I implemented a simple <strong>NebulaGraph MCP Server</strong> based on NebulaGraph 3.x, serving as the first foundational component in the NebulaGraph X MCP ecosystem. This server is intended for exploring the potential of combining AI and graph databases, and I hope to see more exciting work emerge under this paradigm.</p> <h2 id="about-model-context-protocol-mcp">About Model Context Protocol (MCP)<a class="zola-anchor" href="#about-model-context-protocol-mcp" aria-label="Anchor link for: about-model-context-protocol-mcp" style="visibility: hidden;"></a> </h2> <p>MCP is an open protocol introduced by the Anthropic team, designed to provide AI systems with a unified and efficient way to access various data resources. By adopting a client-server architecture, MCP standardizes data access, addressing the fragmentation issue of traditional AI systems dealing with diverse data sources.</p> <p>The core features of MCP include:</p> <ol> <li><strong>Standardized Communication Protocol</strong>: With MCP, AI assistants can effectively access various data sources, such as local data, remote services, and enterprise tools.</li> <li><strong>Openness and Ecosystem Advantage</strong>: The MCP community currently boasts nearly 1,000 MCP Server implementations, covering domains such as search, weather, databases, and more, enabling rapid tool integration.</li> <li><strong>Flexible Architecture</strong>: By defining the interaction protocol between clients and servers, MCP provides developers with the foundation to build scalable and reusable systems.</li> </ol> <p>While challenges remain in terms of security and reliability, these issues are expected to be resolved over time with improved protocol specifications and the adoption of mature development practices.</p> <h2 id="implementation-and-features-of-the-nebulagraph-mcp-server">Implementation and Features of the NebulaGraph MCP Server<a class="zola-anchor" href="#implementation-and-features-of-the-nebulagraph-mcp-server" aria-label="Anchor link for: implementation-and-features-of-the-nebulagraph-mcp-server" style="visibility: hidden;"></a> </h2> <p>Recently, I have been working on GraphRAG and Agentic Workflow, focusing on deriving insights and mining graph data using large models and intelligent agents. To support this, I quickly developed the <strong>NebulaGraph MCP Server</strong>, which integrates NebulaGraph 3.x into the MCP ecosystem, enabling language models (like Claude, GPT, etc.) to call and use it.</p> <p>Currently, the NebulaGraph MCP Server is implemented based on <strong>FastMCP</strong>, adhering to the core MCP specifications. It aims to provide an efficient and lightweight graph database connection service. By exposing NebulaGraph’s functions as standardized tool interfaces, the server empowers large models to easily access NebulaGraph data and perform basic graph exploration tasks.</p> <h3 id="features">Features:<a class="zola-anchor" href="#features" aria-label="Anchor link for: features" style="visibility: hidden;"></a> </h3> <ol> <li> <p><strong>Multiple Transmission Modes</strong>:</p> <ul> <li>Supports <strong>stdio</strong> and <strong>SSE</strong> (Server-Sent Events) transmission modes to cater to different development and deployment scenarios.</li> </ul> </li> <li> <p><strong>Basic Graph Exploration Capabilities</strong>:</p> <ul> <li><strong>Graph Space Listing</strong>: Allows models to query available graph spaces.</li> <li><strong>Schema Querying</strong>: Supports querying the schema definitions of specific graph spaces.</li> <li><strong>Query Execution</strong>: Enables executing NebulaGraph queries through the MCP Server.</li> </ul> </li> <li> <p><strong>Built-in Operator Templates</strong>:</p> <ul> <li>Implements templates for common operations like path search and neighbor discovery, allowing language models to quickly call these tools as needed and gain initial insights into graph data.</li> </ul> </li> </ol> <p>To demonstrate its effectiveness, I built a simple example using LlamaIndex’s McpToolSpec and ReActAgent. Below is a screenshot:</p> <p><img src="https://psiace.me/posts/announcing-nebulagraph-mcp-server-v010/llamaindex-with-nebulagraph-mcp.png" alt="LlamaIndex with NebulaGraph MCP" /></p> <h2 id="conclusion">Conclusion<a class="zola-anchor" href="#conclusion" aria-label="Anchor link for: conclusion" style="visibility: hidden;"></a> </h2> <p>The <strong>NebulaGraph MCP Server</strong> is an initial attempt to contribute to the MCP ecosystem, aiming to provide a starting point for combining language models and graph databases. As the protocol matures and the ecosystem expands, I look forward to seeing it deliver value in more real-world scenarios. If you are interested in this direction, feel free to explore and experiment together!</p> <hr /> <p>For more details, check out the original repository <a rel="nofollow noreferrer" href="https://github.com/PsiACE/nebulagraph-mcp-server">here</a>.</p> 2025-02-11T00:00:00+00:00 2025-02-11T00:00:00+00:00 Unknown https://psiace.me/posts/integrate-mcp-tools-into-llamaindex/ <blockquote> <p>[!NOTE] This work has been contributed to upstream <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index">llama_index</a> as <code>McpToolSpec</code> in the <a rel="nofollow noreferrer" href="https://pypi.org/project/llama-index-tools-mcp/"><code>llama-index-tools-mcp</code></a> package. You can find the pull request <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index/pull/17795">llama_index#17795 feat: add mcp tool spec</a> and try out the example notebook <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index/blob/main/llama-index-integrations/tools/llama-index-tools-mcp/examples/mcp.ipynb">llama-index-tools-mcp/examples/mcp.ipynb</a> directly.</p> </blockquote> <p>This article mainly introduces how to convert MCP (Model Context Protocol) tools into LlamaIndex tools that can be directly used, allowing <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index">LlamaIndex</a> users to seamlessly integrate these services like other modern and popular AI applications like Claude and Cursor, etc.</p> <h2 id="1-background">1. Background<a class="zola-anchor" href="#1-background" aria-label="Anchor link for: 1-background" style="visibility: hidden;"></a> </h2> <h3 id="1-1-what-is-mcp">1.1 What is MCP?<a class="zola-anchor" href="#1-1-what-is-mcp" aria-label="Anchor link for: 1-1-what-is-mcp" style="visibility: hidden;"></a> </h3> <p>MCP(Model Context Protocol, https://modelcontextprotocol.io) is a protocol for building services that interact with AI applications. It allows developers to build servers that expose features like data (resources), functions (tools), and prompts. For example, we can define a <code>fetch_ipinfo</code> tool that retrieves detailed information about a specified IP address (such as city and timezone). code here: <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/mcp_server.py">mcp_server.py</a></p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-annotation z-python"><span class="z-punctuation z-definition z-annotation z-python">@</span><span class="z-meta z-annotation z-function z-python"></span></span><span class="z-meta z-annotation z-function z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">mcp</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-annotation z-function z-python"><span class="z-meta z-generic-name z-python">tool</span></span></span></span><span class="z-meta z-annotation z-function z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span></span><span class="z-meta z-annotation z-function z-python"><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">fetch_ipinfo</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">ip</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span> <span class="z-keyword z-operator z-arithmetic z-python">|</span> <span class="z-constant z-language z-python">None</span> </span><span class="z-meta z-function z-parameters z-default-value z-python"><span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">None</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-variable z-parameter z-python">kwargs</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">IPDetails</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-block z-documentation z-python"><span class="z-punctuation z-definition z-comment z-begin z-python">&quot;&quot;&quot;</span>Get the detailed information of a specified IP address </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> Args: </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> ip(str or None): The IP address to get information for. Follow the format like 192.168.1.1 . </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> **kwargs: Additional keyword arguments to pass to the IPInfo handler. </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> Returns: </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> IPDetails: The detailed information of the specified IP address. </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> <span class="z-punctuation z-definition z-comment z-end z-python">&quot;&quot;&quot;</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">handler</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ipinfo</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getHandler</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">access_token</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">IPINFO_API_TOKEN<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-constant z-language z-python">None</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">headers</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span></span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">user-agent<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">basic-mcp-server<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">custom_header<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">yes<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">kwargs</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">details</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">handler</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getDetails</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">ip_address</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ip</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">IPDetails</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-keyword z-operator z-arithmetic z-python">*</span><span class="z-keyword z-operator z-arithmetic z-python">*</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">details</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">all</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <h3 id="1-2-what-is-llamaindex">1.2 What is LlamaIndex?<a class="zola-anchor" href="#1-2-what-is-llamaindex" aria-label="Anchor link for: 1-2-what-is-llamaindex" style="visibility: hidden;"></a> </h3> <p>LlamaIndex (https://github.com/run-llama/llama_index) is a framework for building, managing, and calling index systems. It aims to bring external information into large language models in a structured way, achieving more efficient information retrieval, question answering, and dialogue generation. When using LlamaIndex, you often need to call external tools to get dynamic data, which can be considered as converting MCP tools into LlamaIndex tools.</p> <h2 id="2-why-do-we-need-to-convert-mcp-tools-to-llamaindex-tools">2. Why do we need to convert MCP tools to LlamaIndex tools?<a class="zola-anchor" href="#2-why-do-we-need-to-convert-mcp-tools-to-llamaindex-tools" aria-label="Anchor link for: 2-why-do-we-need-to-convert-mcp-tools-to-llamaindex-tools" style="visibility: hidden;"></a> </h2> <p>In actual scenarios, you may want to:</p> <ul> <li>Use the rich features exposed by the MCP servers, there is a list of MCP servers https://github.com/AlexMili/Awesome-MCP .</li> <li>Integrate MCP tools into LlamaIndex to form a unified workflow, simplifying the process of calling external APIs with large language models.</li> <li>Automatically convert MCP tools to FunctionTool objects in LlamaIndex, making it easier for LlamaIndex Agent to automatically call and manage tools.</li> </ul> <p>By doing this, you can easily reuse the tools already in MCP, without having to write code again, and also use the powerful dialogue generation and context management capabilities of LlamaIndex.</p> <h2 id="3-implementation">3. Implementation<a class="zola-anchor" href="#3-implementation" aria-label="Anchor link for: 3-implementation" style="visibility: hidden;"></a> </h2> <p>Implementing the conversion of MCP tools to LlamaIndex tools mainly includes the following steps:</p> <ul> <li><strong>Communicate with MCP server</strong>: <ul> <li>Establish a connection with the MCP server using the <code>MCPClient</code> class, at least support <code>list_tools</code> and <code>call_tool</code> methods.</li> <li>See <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/mcp_client.py">mcp_client.py</a> for reference.</li> </ul> </li> <li><strong>Construct adapter</strong>: <ul> <li>Define the <code>MCPToolAdapter</code> class, which uses the MCPClient's <code>list_tools</code> method to get the tool list and uses the <code>FunctionTool.from_defaults</code> method in LlamaIndex to wrap each MCP tool into a LlamaIndex tool.</li> <li>See <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/llamaindex_mcp_adapter.py">llamaindex_mcp_adapter.py</a> for reference.</li> </ul> </li> <li><strong>Use the adapter in LlamaIndex</strong>: <ul> <li>In LlamaIndex, use the adapter to get tools, and then use the agent to call the tools.</li> <li>See <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/llamaindex_mcp_example.py">llamaindex_mcp_example.py</a> for reference.</li> </ul> </li> </ul> <h2 id="4-code-example">4. Code Example<a class="zola-anchor" href="#4-code-example" aria-label="Anchor link for: 4-code-example" style="visibility: hidden;"></a> </h2> <h3 id="4-1-mcptooladapter">4.1 MCPToolAdapter<a class="zola-anchor" href="#4-1-mcptooladapter" aria-label="Anchor link for: 4-1-mcptooladapter" style="visibility: hidden;"></a> </h3> <p>The following code is from <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/llamaindex_mcp_adapter.py">llamaindex_mcp_adapter.py</a>, which shows how to convert MCP tools to LlamaIndex tools:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">typing</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Any</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">Dict</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">List</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">Optional</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">Type</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">tools</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">FunctionTool</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">mcp_client</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">MCPClient</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">pydantic</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">BaseModel</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">Field</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">create_model</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">json_type_mapping</span></span><span class="z-punctuation z-separator z-annotation z-variable z-python">:</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Dict</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Type</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping-or-set z-python"><span class="z-punctuation z-section z-mapping-or-set z-begin z-python">{</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping-or-set z-python"></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">string<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">number<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">float</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">integer<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">int</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">boolean<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">bool</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">object<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">dict</span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-key z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">array<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-separator z-mapping z-key-value z-python">:</span></span><span class="z-meta z-mapping z-python"> </span><span class="z-meta z-mapping z-value z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">list</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-mapping z-value z-python"></span><span class="z-meta z-mapping z-python"><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">create_model_from_json_schema</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">schema</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Dict</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span>, <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Any</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">model_name</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span> </span><span class="z-meta z-function z-parameters z-default-value z-python"><span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">DynamicModel<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Type</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">BaseModel</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">properties</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">schema</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">properties<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-mapping z-empty z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">required_fields</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">set</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">schema</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">required<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-sequence z-list z-empty z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">fields</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-mapping z-empty z-python"><span class="z-punctuation z-section z-mapping z-begin z-python">{</span><span class="z-punctuation z-section z-mapping z-end z-python">}</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-statement z-loop z-for z-python"><span class="z-keyword z-control z-loop z-for z-python">for</span> <span class="z-meta z-generic-name z-python">field_name</span><span class="z-punctuation z-separator z-target-list z-python">,</span> <span class="z-meta z-generic-name z-python">field_schema</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span><span class="z-meta z-statement z-loop z-for z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">properties</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">items</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span></span><span class="z-meta z-statement z-loop z-for z-python"><span class="z-punctuation z-section z-block z-loop z-for z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">json_type</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_schema</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">type<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">string<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_type</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">json_type_mapping</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">json_type</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_name</span></span> <span class="z-keyword z-operator z-logical z-python">in</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">required_fields</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">default_value</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">...</span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-else z-python"><span class="z-keyword z-control z-conditional z-else z-python">else</span><span class="z-punctuation z-section z-block z-conditional z-else z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">default_value</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-constant z-language z-python">None</span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_type</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Optional</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_type</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">fields</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_name</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-sequence z-tuple z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_type</span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Field</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">default_value</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">description</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">field_schema</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">description<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-sequence z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">dynamic_model</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">create_model</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">model_name</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">fields</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">dynamic_model</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">MCPToolAdapter</span></span><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-support z-function z-magic z-python">__init__</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">client</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">MCPClient</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">client</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">list_tools</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-item-access z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">List</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-begin z-python">[</span></span><span class="z-meta z-item-access z-arguments z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">FunctionTool</span></span></span><span class="z-meta z-item-access z-python"><span class="z-punctuation z-section z-brackets z-end z-python">]</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">client</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">list_tools</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span> </span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">FunctionTool</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">from_defaults</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">fn</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">_create_tool_fn</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">name</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">name</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">name</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">description</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">description</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">fn_schema</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">create_model_from_json_schema</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">inputSchema</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"> <span class="z-meta z-expression z-generator z-python"><span class="z-keyword z-control z-loop z-for z-generator z-python">for</span> <span class="z-meta z-generic-name z-python">tool</span> <span class="z-keyword z-control z-loop z-for z-in z-python">in</span></span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">tools</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-sequence z-list z-python"> <span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">_create_tool_fn</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">tool_name</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">tool_fn</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-keyword z-operator z-unpacking z-mapping z-python">**</span><span class="z-variable z-parameter z-python">kwargs</span><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">client</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">call_tool</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_name</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">kwargs</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tool_fn</span></span> </span></code></pre> <p>Explanation:</p> <ul> <li> <p><strong>JSON Schema to Pydantic model</strong>:</p> <ul> <li>The function <code>create_model_from_json_schema</code> builds a dynamic Pydantic model based on the JSON Schema defined in the MCP tool, making it easier to validate tool parameters and provide auto-completion.</li> </ul> </li> <li> <p><strong>Adapter's list_tools method</strong>:</p> <ul> <li>This method fetches the tool list from the server using <code>MCPClient</code>, then wraps each tool with <code>FunctionTool.from_defaults</code>, and finally obtains a list of tools that can be used by the LlamaIndex Agent.</li> </ul> </li> <li> <p><strong>Dynamic call MCP tool</strong>:</p> <ul> <li>The helper method <code>_create_tool_fn</code> returns an asynchronous function, which internally calls <code>self.client.call_tool</code> to implement the actual tool calling logic.</li> </ul> </li> </ul> <blockquote> <p>Why we need to convert MCP tool's input parameters to Pydantic model?</p> <p>MCP tool defines the input parameters in JSON Schema format (<code>inputSchema</code> in <code>Tool</code> object, which is <code>Dict[str, Any]</code>), and the <code>create_model_from_json_schema</code> function converts it to a Pydantic model, which is used as the <code>fn_schema</code> of the <code>FunctionTool</code> in LlamaIndex.</p> </blockquote> <h3 id="4-2-usage-in-llamaindex">4.2 Usage in LlamaIndex<a class="zola-anchor" href="#4-2-usage-in-llamaindex" aria-label="Anchor link for: 4-2-usage-in-llamaindex" style="visibility: hidden;"></a> </h3> <p>The following code is from <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter/llamaindex_mcp_example.py">llamaindex_mcp_example.py</a>, which shows how to use the <code>MCPToolAdapter</code> with LlamaIndex:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">ChatMessage</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">agent</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">ReActAgent</span><span class="z-punctuation z-separator z-import-list z-python">,</span> <span class="z-meta z-generic-name z-python">ReActChatFormatter</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">agent</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">react</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">prompts</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-variable z-other z-constant z-python">REACT_CHAT_SYSTEM_HEADER</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">azure_openai</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">AzureOpenAI</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llamaindex_mcp_adapter</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">MCPToolAdapter</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">mcp_client</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">MCPClient</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">anyio</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">dotenv</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">argparse</span></span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">dotenv</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">load_dotenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">SYSTEM_PROMPT</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;&quot;&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"><span class="z-meta z-string z-python"></span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"><span class="z-punctuation z-separator z-continuation z-line z-python">\</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python">You are an AI assistant. </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python">Before you answer the user&#39;s question, you need to fetch the ip info first, to help you follow the laws of the country. </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python">- When user ask about current ip info, location, city, country or timezone, use the &#39;fetch_ipinfo&#39; tool to provide the answer. </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;&quot;&quot;</span></span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">get_agent</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">adapter</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">MCPToolAdapter</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tools</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">adapter</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">list_tools</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">llm</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">AzureOpenAI</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_MODEL<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-model-name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">max_tokens</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">int</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_MAX_TOKENS<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-constant z-numeric z-integer z-decimal z-python">4096</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">azure_endpoint</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_ENDPOINT<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">https://your-azure-endpoint.openai.azure.com<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">api_key</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_API_KEY<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-api-key<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">api_version</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_API_VERSION<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-api-version<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">engine</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">environ</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">AZURE_OPENAI_ENGINE<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">your-deployment-name<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ReActAgent</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">from_tools</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">llm</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">llm</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">tools</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">list</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">tools</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">react_chat_formatter</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">ReActChatFormatter</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">system_header</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">SYSTEM_PROMPT</span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-constant z-character z-escape z-python">\n</span><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> <span class="z-keyword z-operator z-arithmetic z-python">+</span> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">REACT_CHAT_SYSTEM_HEADER</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">max_iterations</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-constant z-numeric z-integer z-decimal z-python">20</span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">verbose</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-constant z-language z-python">True</span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">handle_user_message</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">message_content</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">agent</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ReActAgent</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">user_message</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">ChatMessage</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">from_str</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">role</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">user<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">content</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">message_content</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">achat</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">message</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">user_message</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">content</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">response</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-variable z-magic z-python">__name__</span></span> <span class="z-keyword z-operator z-comparison z-python">==</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">__main__<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">main</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">argparse</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">ArgumentParser</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">add_argument</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">--client_type<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">type</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">default</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">sse<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">choices</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">sse<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">stdio<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">args</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">parse_args</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">args</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">client_type</span></span> <span class="z-keyword z-operator z-comparison z-python">==</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">sse<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">MCPClient</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">http://0.0.0.0:8000/sse<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-else z-python"><span class="z-keyword z-control z-conditional z-else z-python">else</span><span class="z-punctuation z-section z-block z-conditional z-else z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">MCPClient</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">./venv/bin/python<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">mcp_server.py<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">--server_type<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-sequence z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">stdio<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">adapter</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">MCPToolAdapter</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">client</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">get_agent</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">adapter</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">handle_user_message</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">What&#39;s the city of my current ip?<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">handle_user_message</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Is it legal to hold drugs in my country?<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">agent</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">anyio</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">run</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">main</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>Explanation:</p> <ul> <li><strong>Get MCP tools</strong>: <ul> <li>Use <code>MCPToolAdapter</code> to get MCP tools and convert them to LlamaIndex tools.</li> </ul> </li> <li><strong>Create Agent</strong>: <ul> <li>Use <code>ReActAgent.from_tools</code> to build an Agent, which embeds Azure OpenAI's LLM.</li> </ul> </li> <li><strong>Process user messages</strong>: <ul> <li>The user's message is analyzed by the Agent, triggering the corresponding MCP tool (e.g., calling <code>fetch_ipinfo</code>).</li> </ul> </li> </ul> <p>This approach seamlessly converts MCP tools to LlamaIndex tools, greatly reducing integration complexity.</p> <h2 id="5-running-the-demo">5. Running the Demo<a class="zola-anchor" href="#5-running-the-demo" aria-label="Anchor link for: 5-running-the-demo" style="visibility: hidden;"></a> </h2> <p>You can run the demo with the following command:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">python</span></span><span class="z-meta z-function-call z-arguments z-shell"> llamaindex_mcp_example.py<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>client_type</span> stdio</span> </span></code></pre> <p>The demo will start a MCP server using <code>stdio</code> mode, and then use the <code>MCPToolAdapter</code> to get MCP tools and convert them to LlamaIndex tools. Also, it will use the <code>AzureOpenAI</code> to call the tools and generate responses.</p> <p>The demo also provides a <code>sse</code> mode, you can run the demo with the following command:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">python</span></span><span class="z-meta z-function-call z-arguments z-shell"> mcp_server.py<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>server_type</span> sse</span> </span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">python</span></span><span class="z-meta z-function-call z-arguments z-shell"> llamaindex_mcp_example.py<span class="z-variable z-parameter z-option z-shell"><span class="z-punctuation z-definition z-parameter z-shell"> --</span>client_type</span> sse <span class="z-comment z-line z-number-sign z-shell"><span class="z-punctuation z-definition z-comment z-begin z-shell">#</span></span><span class="z-comment z-line z-number-sign z-shell"> in another terminal session</span><span class="z-comment z-line z-number-sign z-shell"> </span></span></span></code></pre> <p>The following is the output of the demo, with the <code>stdio</code> mode:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">Processing</span></span><span class="z-meta z-function-call z-arguments z-shell"> request of type ListToolsRequest</span> </span><span class="z-source z-shell z-bash"><span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> Running <span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">step</span></span><span class="z-meta z-function-call z-arguments z-shell"> ede14770-91c9-428b-a66a-d57d20e0b5d5. Step input: What<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>s the city of my current ip? </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Thought: The current language of the user is: English. I need to use a tool to help me answer the question. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Action: fetch_ipinfo </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Action Input: {<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span>kwargs<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>: AttributedDict()} </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Processing request of type CallToolRequest </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Observation: meta=None content=[TextContent(type=<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span>text<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>, text=<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>ip<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>183.193.123.192<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>hostname<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: null<span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>city<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>region<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>country<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>CN<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>loc<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>31.2222,121.4581<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>timezone<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Asia/Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>)] isError=False </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">&gt; Running step 45ea121b-9c05-431f-b409-d235409c1556. Step input: None </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Thought: I can answer without using any more tools. I<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span>ll use the user<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>s language to answer. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Answer: The city of your current IP is Shanghai. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">The city of your current IP is Shanghai. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">&gt; Running step 4006f760-1070-4b0c-8446-7db886bc7c2e. Step input: Is it legal to hold drugs in my country? </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Thought: The current language of the user is: English. I need to use a tool to help me answer the question. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Action: fetch_ipinfo </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Action Input: {<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span>kwargs<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>: AttributedDict()} </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Processing request of type CallToolRequest </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Observation: meta=None content=[TextContent(type=<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span>text<span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>, text=<span class="z-punctuation z-definition z-string z-end z-shell">&#39;</span></span><span class="z-meta z-group z-expansion z-brace z-shell"><span class="z-punctuation z-section z-expansion z-brace z-begin z-shell">{</span><span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>ip<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>183.193.123.192<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>hostname<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: null<span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>city<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>region<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>country<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>CN<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>loc<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>31.2222,121.4581<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-separator z-shell">,</span> <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>timezone<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span>: <span class="z-string z-quoted z-double z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&quot;</span>Asia/Shanghai<span class="z-punctuation z-definition z-string z-end z-shell">&quot;</span></span><span class="z-punctuation z-section z-expansion z-brace z-end z-shell">}</span></span><span class="z-string z-quoted z-single z-shell"><span class="z-punctuation z-definition z-string z-begin z-shell">&#39;</span>)] isError=False </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">&gt; Running step e728f5d0-6c0e-454a-aeb9-a654ace0e41b. Step input: None </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Thought: I have obtained the IP information, including the country, which is China. Now I can provide information regarding the legality of holding drugs in China. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">Answer: In China, the possession of drugs is illegal and can lead to severe penalties, including imprisonment. It is important to be aware of and comply with local laws regarding drug possession. </span></span></span><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-arguments z-shell"><span class="z-string z-quoted z-single z-shell">In China, the possession of drugs is illegal and can lead to severe penalties, including imprisonment. It is important to be aware of and comply with local laws regarding drug possession. </span></span></span></code></pre> <h2 id="6-summary">6. Summary<a class="zola-anchor" href="#6-summary" aria-label="Anchor link for: 6-summary" style="visibility: hidden;"></a> </h2> <p>This article details how to convert MCP tools to LlamaIndex tools. By:</p> <ul> <li>Interacting with MCP servers using <code>MCPClient</code>;</li> <li>Automatically converting tool parameter definitions (using JSON Schema to convert to Pydantic models) through <code>MCPToolAdapter</code>;</li> <li>Integrating converted tools into LlamaIndex <code>ReActAgent</code> and calling them;</li> </ul> <p>Now you can seamlessly integrate MCP tools into LlamaIndex, reducing development costs and providing great flexibility for AI application expansion.</p> <p>Welcome to explore the combination of MCP and LlamaIndex, and further expand more practical tools and functions!</p> <hr /> <p>See the <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/llamaindex-mcp-adapter">demo</a> for more details.</p> 2024-11-28T00:00:00+00:00 2024-11-28T00:00:00+00:00 Unknown https://psiace.me/posts/a-dead-simple-way-to-vlm-parsing/ <p>This post demonstrates a straightforward approach to using VLMs for parsing unstructured data from images. While simple, this method can be easily extended to handle more complex scenarios, such as extracting information from PDFs. If you want, you can built a more complex parser as <a rel="nofollow noreferrer" href="https://www.llamaindex.ai/llamaparse">llamaparse</a> with more steps, like OCR, table parsing, etc.</p> <p>Let's consider a common scenario: you have a collection of images and need to extract text from them. Or perhaps you want to generate descriptions from pictures to enable full-text search capabilities. It's time to hire a VLM (Visual Language Models or MultiModal Models) to work for you.</p> <p>Since I work extensively with <a rel="nofollow noreferrer" href="https://github.com/run-llama/llama_index">llama_index</a>, I'll be using it in the examples. However, the core concepts remain the same, and you can adapt them to work with any tools or clients of your choice.</p> <h2 id="show-me-the-code">Show Me The Code<a class="zola-anchor" href="#show-me-the-code" aria-label="Anchor link for: show-me-the-code" style="visibility: hidden;"></a> </h2> <p>At first, we need to define a prompt for the VLM to follow, to help it understand what we need.</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">BASIC_PROMPT</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;&quot;&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> <span class="z-meta z-string z-python"></span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python">Please analyze this image comprehensively and provide the following information: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> 1. General Overview: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Main subject matter and content </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Overall composition and context </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Key visual elements present </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> 2. Text Content: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Transcribe any visible text accurately </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Include headers, labels, and captions </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Note any important text formatting or emphasis </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> 3. Data Visualization (if present): </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - For tables: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Convert to markdown format </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Preserve column headers and data relationships </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - For charts/graphs: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Describe the type of visualization </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Explain key trends and patterns </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * List important data points and values </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - For diagrams/flowcharts: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Explain the structure and relationships </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Describe the flow or process </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> * Note any important symbols or annotations </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> 4. Additional Details: </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Identify any branding or logos </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Note color schemes if significant </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> - Describe any relevant metadata or context </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> Please format the response clearly and maintain the original structure of any data. </span></span></span><span class="z-source z-python"><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-block z-python"> <span class="z-punctuation z-definition z-string z-end z-python">&quot;&quot;&quot;</span></span></span> </span></code></pre> <p>Let's build a minimal but functional image parser:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">pathlib</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Path</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">typing</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">List</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">base64</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">io</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">BytesIO</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">PIL</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Image</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">multi_modal_llms</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">MultiModalLLM</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">core</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">multi_modal_llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">generic_utils</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">ImageDocument</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-class z-python"><span class="z-storage z-type z-class z-python"><span class="z-keyword z-declaration z-class z-python">class</span></span> <span class="z-entity z-name z-class z-python"><span class="z-meta z-generic-name z-python">SimpleImageParser</span></span><span class="z-punctuation z-section z-class z-begin z-python">:</span></span> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-support z-function z-magic z-python">__init__</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">model</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">MultiModalLLM</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">model</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">model</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">prompt</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-qualified-name z-python"><span class="z-variable z-other z-constant z-python">BASIC_PROMPT</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-comment z-block z-documentation z-python"><span class="z-punctuation z-definition z-comment z-begin z-python">&quot;&quot;&quot;</span>A simple parser for extracting information from images using VLMs. </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> Args: </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> model (MultiModalLLM): The multi-modal language model to use for parsing </span></span><span class="z-source z-python"><span class="z-comment z-block z-documentation z-python"> <span class="z-punctuation z-definition z-comment z-end z-python">&quot;&quot;&quot;</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">process_image</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">image_path</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-block z-documentation z-python"><span class="z-punctuation z-definition z-comment z-begin z-python">&quot;&quot;&quot;</span>Convert image to base64 encoding<span class="z-punctuation z-definition z-comment z-end z-python">&quot;&quot;&quot;</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">Image</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">open</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">buffered</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">BytesIO</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">save</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">buffered</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">format</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">PNG<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">base64</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">b64encode</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">buffered</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getvalue</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">decode</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">utf-8<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"> <span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">parse</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">image_path</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-comment z-block z-documentation z-python"><span class="z-punctuation z-definition z-comment z-begin z-python">&quot;&quot;&quot;</span>Parse image content<span class="z-punctuation z-definition z-comment z-end z-python">&quot;&quot;&quot;</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_data</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">process_image</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_doc</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">ImageDocument</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">image_url</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-storage z-type z-string z-python">f</span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python">data:image/jpeg;base64,</span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-begin z-python">{</span><span class="z-source z-python z-embedded"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_data</span></span></span></span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-end z-python">}</span></span><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">model</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">acomplete</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">prompt</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">prompt</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">image_documents</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-sequence z-list z-python"><span class="z-punctuation z-section z-sequence z-begin z-python">[</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_doc</span></span><span class="z-punctuation z-section z-sequence z-end z-python">]</span></span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">str</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">response</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>The core is, we convert the image to base64 encoding, and then pass it to the VLM.</p> <h2 id="taking-it-for-a-spin">Taking It for a Spin<a class="zola-anchor" href="#taking-it-for-a-spin" aria-label="Anchor link for: taking-it-for-a-spin" style="visibility: hidden;"></a> </h2> <p>Here's how to use our shiny new parser:</p> <pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">asyncio</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">argparse</span></span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">pathlib</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">Path</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">llama_index</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">multi_modal_llms</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-import-name z-python">openai</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">OpenAIMultiModal</span></span> </span><span class="z-source z-python"><span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-from z-python">from</span></span><span class="z-meta z-statement z-import z-python"><span class="z-meta z-import-source z-python"> <span class="z-meta z-import-path z-python"><span class="z-meta z-import-name z-python">dotenv</span></span> <span class="z-meta z-statement z-import z-python"><span class="z-keyword z-control z-import z-python">import</span></span></span></span><span class="z-meta z-statement z-import z-python"></span><span class="z-meta z-statement z-import z-python"> <span class="z-meta z-generic-name z-python">load_dotenv</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">load_dotenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">main</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">argparse</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">ArgumentParser</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-variable z-parameter z-python">description</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Parse image content<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">add_argument</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">image_path<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">type</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-variable z-parameter z-python">help</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">Path to the image file<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">args</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">parse_args</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">Path</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">args</span><span class="z-punctuation z-accessor z-dot z-python">.</span><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-keyword z-operator z-logical z-python">not</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">exists</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-storage z-type z-string z-python">f</span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-interpolated z-python"><span class="z-string z-quoted z-double z-python">Image file does not exist: </span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-begin z-python">{</span><span class="z-source z-python z-embedded"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span></span></span><span class="z-meta z-interpolation z-python"><span class="z-punctuation z-section z-interpolation z-end z-python">}</span></span><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Initialize the VLM </span></span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">model</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">OpenAIMultiModal</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">model</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_MODEL<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">gpt-4o-mini<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">api_key</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_API_KEY<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">api_base</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_API_BASE<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">https://api.openai.com/v1<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">max_new_tokens</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">int</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_MAX_TOKENS<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">512<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">temperature</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">float</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_TEMPERATURE<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">0.7<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-variable z-parameter z-python">verbose</span><span class="z-keyword z-operator z-assignment z-python">=</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">os</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">getenv</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">MULTI_MODAL_LLM_VERBOSE<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">False<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">lower</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> <span class="z-keyword z-operator z-comparison z-python">==</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">true<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-separator z-arguments z-python">,</span> </span></span><span class="z-source z-python"><span class="z-meta z-function-call z-arguments z-python"> <span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Create our parser </span></span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">SimpleImageParser</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">model</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Let&#39;s parse an image! </span></span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">parser</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">parse</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-type z-python">str</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-support z-function z-builtin z-python">print</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> </span><span class="z-source z-python"><span class="z-meta z-statement z-conditional z-if z-python"><span class="z-keyword z-control z-conditional z-if z-python">if</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-variable z-magic z-python">__name__</span></span> <span class="z-keyword z-operator z-comparison z-python">==</span> <span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python"><span class="z-punctuation z-definition z-string z-begin z-python">&quot;</span></span></span><span class="z-meta z-string z-python"><span class="z-string z-quoted z-double z-python">__main__<span class="z-punctuation z-definition z-string z-end z-python">&quot;</span></span></span><span class="z-punctuation z-section z-block z-conditional z-if z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">asyncio</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">run</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">main</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span></code></pre> <p>In this snippet, we initialize the VLM, create our parser, and then parse an image.</p> <h2 id="running-the-demo">Running the Demo<a class="zola-anchor" href="#running-the-demo" aria-label="Anchor link for: running-the-demo" style="visibility: hidden;"></a> </h2> <p>I have a repo for a demo of using VLM to parse images, you can see the <a rel="nofollow noreferrer" href="https://github.com/psiace/psiace/tree/main/demo/vlm-parsing">code here</a>.</p> <p>Just run the script with the image path:</p> <pre data-lang="bash" class="language-bash z-code"><code class="language-bash" data-lang="bash"><span class="z-source z-shell z-bash"><span class="z-meta z-function-call z-shell"><span class="z-variable z-function z-shell">python</span></span><span class="z-meta z-function-call z-arguments z-shell"> basic.py <span class="z-keyword z-operator z-assignment z-redirection z-shell">&lt;</span>image_path<span class="z-keyword z-operator z-assignment z-redirection z-shell">&gt;</span> </span></span></code></pre> <p>The example image is a screenshot of a home gateway, from <a rel="nofollow noreferrer" href="https://commons.wikimedia.org/wiki/File:Home_gateway_example.png">Home gateway example</a>:</p> <p><img src="https://psiace.me/posts/a-dead-simple-way-to-vlm-parsing/example.png" alt="Home gateway example" /></p> <p>And the result in terminal is:</p> <p><img src="https://psiace.me/posts/a-dead-simple-way-to-vlm-parsing/example-result.png" alt="Home gateway example result" /></p> <h2 id="bonus-extending-the-parser">Bonus: Extending the Parser<a class="zola-anchor" href="#bonus-extending-the-parser" aria-label="Anchor link for: bonus-extending-the-parser" style="visibility: hidden;"></a> </h2> <p>Here are some practical ways to extend this simple parser:</p> <ol> <li> <p><strong>Format Support</strong></p> <ul> <li>Handle PDFs using tools like <code>pdf2image</code></li> <li>Process videos using <code>opencv-python</code> for frame extraction</li> </ul> </li> <li> <p><strong>Output Options</strong></p> <ul> <li>Generate JSON output:<pre data-lang="python" class="language-python z-code"><code class="language-python" data-lang="python"><span class="z-source z-python"><span class="z-meta z-function z-python"><span class="z-storage z-modifier z-async z-python">async</span> <span class="z-storage z-type z-function z-python"><span class="z-keyword z-declaration z-function z-python">def</span></span> <span class="z-entity z-name z-function z-python"><span class="z-meta z-generic-name z-python">parse_to_json</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-begin z-python">(</span></span><span class="z-meta z-function z-parameters z-python"><span class="z-variable z-parameter z-python">self</span><span class="z-punctuation z-separator z-parameters z-python">,</span> <span class="z-variable z-parameter z-python">image_path</span></span><span class="z-meta z-function z-parameters z-annotation z-python"><span class="z-punctuation z-separator z-annotation z-parameter z-python">:</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">str</span></span></span><span class="z-meta z-function z-parameters z-python"><span class="z-punctuation z-section z-parameters z-end z-python">)</span></span><span class="z-meta z-function z-python"> </span><span class="z-meta z-function z-annotation z-return z-python"><span class="z-punctuation z-separator z-annotation z-return z-python">-&gt;</span> <span class="z-meta z-qualified-name z-python"><span class="z-support z-type z-python">dict</span></span></span><span class="z-meta z-function z-python"><span class="z-punctuation z-section z-function z-begin z-python">:</span></span> </span><span class="z-source z-python"> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">result</span></span> <span class="z-keyword z-operator z-assignment z-python">=</span> <span class="z-keyword z-other z-await z-python">await</span> <span class="z-meta z-function-call z-python"><span class="z-meta z-qualified-name z-python"><span class="z-variable z-language z-python">self</span></span><span class="z-meta z-qualified-name z-python"><span class="z-punctuation z-accessor z-dot z-python">.</span></span><span class="z-meta z-qualified-name z-python"><span class="z-variable z-function z-python"><span class="z-meta z-generic-name z-python">parse</span></span></span></span><span class="z-meta z-function-call z-arguments z-python"><span class="z-punctuation z-section z-arguments z-begin z-python">(</span><span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">image_path</span></span><span class="z-punctuation z-section z-arguments z-end z-python">)</span></span> </span><span class="z-source z-python"> <span class="z-comment z-line z-number-sign z-python"><span class="z-punctuation z-definition z-comment z-python">#</span> Add your JSON transformation logic here </span></span><span class="z-source z-python"> <span class="z-keyword z-control z-flow z-return z-python">return</span> <span class="z-meta z-qualified-name z-python"><span class="z-meta z-generic-name z-python">json_result</span></span> </span></code></pre> </li> <li>Create custom templates using Jinja2 or similar</li> </ul> </li> <li> <p><strong>Processing Pipeline</strong></p> <ul> <li>Integrate Tesseract OCR for text extraction</li> <li>Try to support table parsing</li> <li>Implement batch processing with asyncio</li> </ul> </li> <li> <p><strong>Integration Ideas</strong></p> <ul> <li>Connect with llama-index for RAG applications</li> <li>Build automated workflows</li> </ul> </li> </ol> <h2 id="resources">Resources<a class="zola-anchor" href="#resources" aria-label="Anchor link for: resources" style="visibility: hidden;"></a> </h2> <ul> <li><a rel="nofollow noreferrer" href="https://docs.llamaindex.ai/">llama-index Documentation</a></li> <li><a rel="nofollow noreferrer" href="https://platform.openai.com/">OpenAI</a></li> <li><a rel="nofollow noreferrer" href="https://pillow.readthedocs.io/">PIL (Python Imaging Library)</a></li> </ul> <hr /> <p>Thanks for reading! If you have any questions or suggestions, please feel free to leave a comment.</p> 2024-03-08T00:00:00+00:00 2024-03-08T00:00:00+00:00 Unknown https://psiace.me/posts/you-do-not-need-the-fucking-devrel/ <p>Ah, DevRel – the tech world's equivalent of sticking a megaphone in the hands of a mime: everyone sees the gestures, but no one knows what the hell they're trying to say. Now, if you're considering hiring a DevRel, buckle up, buttercup, because I'm about to serve you a reality sandwich with a side of snark.</p> <p>First off, let's get one thing clear: if your idea of a DevRel is just another pretty face to parrot your press releases and tweet your hashtags, save your money. Buy a billboard. Hire a skywriter. Hell, teach a parrot to code – at least it'll be good for a laugh.</p> <p>But if you're serious – and I mean <em>commit-your-heart-and-soul</em> serious – about having a DevRel, you need to understand that they're not just your company's cheerleader. A DevRel is the bridge between your codebase and the wild jungles of developers who might, if the stars align, use your tools to build the next big thing. They're not just mouthpieces; they're part diplomat, part engineer, part evangelist, and all badass.</p> <h2 id="responsibilities-rights-and-resources">Responsibilities, Rights, and Resources<a class="zola-anchor" href="#responsibilities-rights-and-resources" aria-label="Anchor link for: responsibilities-rights-and-resources" style="visibility: hidden;"></a> </h2> <p>Your DevRel should have the power to make decisions, the responsibility to carry them out, and the resources to not make them look like a clown at a board meeting. If they're going to advocate for your product or platform, they need to have a seat at the table where decisions are made – preferably not the kids' table.</p> <p>If you're not willing to give your DevRel this level of clout, then you might as well have your founders or a seasoned developer play double duty. At least they've got the chops and the battle scars to speak with authority, and they won't have to end every sentence with, "I'll get back to you after I check with the grown-ups."</p> <h2 id="branding-and-content-strategy">Branding and Content Strategy<a class="zola-anchor" href="#branding-and-content-strategy" aria-label="Anchor link for: branding-and-content-strategy" style="visibility: hidden;"></a> </h2> <p>Now, if you've decided to go all-in and appoint a dedicated DevRel, for the love of Silicon Valley, let them dive deep into the heart of your company. They need to be so entwined with your projects that they dream in commit hashes and wake up with whiteboard marker on their faces.</p> <p>Work with them to build a complete brand guideline and content strategy that resonates with your internal vision and the external developer community. This isn't about slapping together a few blog posts and calling it a day. It's about crafting a narrative that developers can rally behind, a story that's as compelling as the 'one more thing' at an Apple keynote.</p> <p>And please, don't make your DevRel pull content out of their ass because you've kept them in the dark. Nothing smells worse than insincerity and ignorance, except maybe your server room after a hard shutdown.</p> <h2 id="in-conclusion">In Conclusion<a class="zola-anchor" href="#in-conclusion" aria-label="Anchor link for: in-conclusion" style="visibility: hidden;"></a> </h2> <p>A DevRel is more than just a hype machine. They're the voice of your developer community within your company, and they're your company's ambassador to the outside world. If you're not ready to give them the power, responsibility, and resources they need to be effective, then you don't need the fucking DevRel.</p> <p>You need a miracle.</p> 2023-09-25T00:00:00+00:00 2023-09-25T00:00:00+00:00 Unknown https://psiace.me/posts/a-new-beginning/ <h2 id="hey-world">Hey, World!<a class="zola-anchor" href="#hey-world" aria-label="Anchor link for: hey-world" style="visibility: hidden;"></a> </h2> <p>I am delighted to announce the launch of my new blog, which is powered by <a rel="nofollow noreferrer" href="https://github.com/withastro/">Astro</a> and features the <a rel="nofollow noreferrer" href="https://github.com/chrismwilliams/astro-theme-cactus">Astro Cactus</a> theme.</p> <p>In this fresh start, I have cleared all previous content to create a clean slate.</p> <h2 id="why-and-why-not">Why and Why Not?<a class="zola-anchor" href="#why-and-why-not" aria-label="Anchor link for: why-and-why-not" style="visibility: hidden;"></a> </h2> <p>The idea of starting anew had been on my mind for a few months, but it was my recent conversation with <a rel="nofollow noreferrer" href="https://github.com/xuanwo">@Xuanwo</a> that finally pushed me to take action.</p> <p>There are numerous reasons behind this decision:</p> <blockquote> <p>I aspire to connect with a broader audience and have a more significant impact.</p> </blockquote> <blockquote> <p>I aim to express my insights more effectively, moving beyond mere documentation of events.</p> </blockquote> <p>However, in reality, the decision was not driven by extensive contemplation; it was simply a matter of “Why not?”</p> <p>Striving to do my best is sufficient.</p> <hr /> <p>I hope you’ll join me on this exciting journey!</p>