Skip to main content

No Framework

Why do you need a framework in the first place?

Crafting well-designed abstractions is very difficult, better to stick close to the metal.

Agents That Work​

Core Definition of an AI Agent:

  • Direction Following: Agents must be able to take directions (human or AI) related to a specific objective.
  • Tool Access & Response: Must have access to at least one tool and receive a response from it.
  • Autonomous Reasoning: Must autonomously reason how and when to use tools to achieve the objective, not just a predefined sequence.

Focus on Reasoning over Knowledge:

  • Prioritize enabling the agent to "think" rather than relying solely on its pre-existing knowledge.
  • Use discrete tool calls for retrieval to feed relevant context into the agent's context window.
  • Avoid overwhelming the agent with too much information in the initial prompt (e.g., large database schemas).

Agent Computer Interface (ACI) is Critical: The syntax and structure of tool calls (input and output) have a massive impact on agent accuracy and performance.Small, seemingly trivial tweaks to the ACI can make a significant difference.

  • Format Matters: Experiment with different response formats (JSON, XML, Markdown) depending on the underlying model. GPT-4o liked JSON, Claude preferred XML in their testing.
  • Model Choice Matters: Use more intelligent models (like Claude 3.5) for the critical task of determining which tool to call next. Balance speed, cost, and decision-making ability.
  • Pay attention to how a model hallucinates: This can indicate the model's expected input format for tool calls. Adjust your tool call definitions to match the model's expectations.

Fine-tuning is (Generally) a Waste of Time:

  • Fine-tuning can decrease reasoning ability by over-fitting the model to specific task sequences.
  • Focus on ACI iteration instead.

Don't Get Overly Reliant on Abstraction Frameworks (like LangGraph) Prematurely:

  • Consider your production goals first.
  • Frameworks can be useful for prototyping, but may introduce blockers when scaling, especially regarding security (e.g., user credential cascading).
  • Building a basic agent or multi-agent system from scratch doesn't require extensive code.

The "System Prompt" is Not the Secret Sauce: The ecosystem around the agent (user experience, connections, security protocols) is more important than the system prompt itself. This is the true differentiator.

Multi-Agent Systems: Key Principles:

  • Manager Agent: Implement a manager agent to own the final outcome and delegate subtasks to worker agents.
  • Team Size: Limit the number of agents working together (5-8 agents, like the "two-pizza rule"). Too many agents decrease the likelihood of success.
  • Incentivization, Not Forcing: Incentivize the manager agent to achieve the overall objective, relying on it to manage worker agents. Avoid forcing worker agents through rigid steps.

Prompt Chains​

Stay close to the metal

  1. Focus on raw prompts rather than relying on complex libraries or abstractions.
  2. The prompt is the most crucial element in generative AI - don't abstract it away.
  3. Break down complex tasks into smaller, manageable chunks using prompt chains.
  4. Use prompt chains to build on previous results and create more sophisticated workflows.
  5. Avoid unnecessary abstractions and premature optimizations in your AI workflows.

Examples

Development Process​

Step-by-Step Process:

  1. Start with a minimalist chainable API:
    • Create a simple class with a single method for chaining prompts.
    • Allow for context passing and back-referencing to previous prompt results.
  2. Determine when to use prompt chains by asking these questions:
    • Is the task too complex for a single prompt?
    • Do you need to increase performance and reduce errors?
    • Do you need to use the output of previous prompts as inputs?
    • Do you need an adaptive workflow that changes based on the flow of the prompt?
  3. Design your prompt chain:
    • Break down your complex task into smaller, focused prompts.
    • Plan how each prompt will build upon the results of previous ones.
  4. Implement the prompt chain:
    • Use a minimalist approach with no external libraries (except for the LLM API).
    • Create a list of prompts, each solving a specific sub-task.
    • Implement a method to run the prompts sequentially, passing context and results between them.
  5. Test and refine:
    • Run your prompt chain and analyze the outputs.
    • Adjust individual prompts or the chain structure as needed to improve results.
  6. Scale up to full agentic workflows:
    • Use your minimalist prompt chain as a building block for more complex workflows.
    • Implement additional logic for handling agent state and responses as needed.
    • Keep your abstractions simple and focused on your specific use case.
  7. Stay adaptable:
    • Keep your implementation flexible to accommodate new LLM models and capabilities.
    • Regularly review and update your prompts and chains as the AI landscape evolves. By following this process, you can create valuable agentic workflows that remain close to the metal, giving you full control over your AI agents while avoiding the pitfalls of over-relying on complex libraries or frameworks.

LLM Agents​

Source Code​