Agent Frameworks
Diagrams | Matrices | Thinkers
Which frameworks and ecosystems a best for producing effective Internet Native Citizens?
| Name | GitHub | Language | Specialty/Agents | Token | DePIN |
|---|---|---|---|---|---|
| Archon | TRUE | Python | Agent for Building Agents | FALSE | FALSE |
| ai16Z Eliza | TRUE | TypeScript | Eliza, SpartenDegen | TRUE | TRUE |
| Almanak | NO CODE PLATFORM | ? | Finance | TRUE | |
| ARC RIG | TRUE | Rust | Low Level Infra, Speed, Trading | TRUE | TRUE |
| Base Agent Kit | TRUE | Python, Typescript | Blockchain Transactions | FALSE | |
| Crew AI | TRUE | Python | Agent Dev Kit | Non Crypto | |
| Griffain | NO CODE PLATFORM | ? | ? | ||
| LangGraph | TRUE | Python | Agent Dev Kit | Non Crypto | |
| Loria | IN DEV | ? | INDIRECT | ||
| Mastra | TRUE | Typescript | |||
| Mongo Dev | TRUE | Python | |||
| Morpheus | TRUE | ? | TRUE | ||
| Pippin | TRUE | Python | TRUE | ||
| Solana Agent Kit | TRUE | Typescript | Blockchain Transactions | FALSE | |
| Vercel AI SDK | TRUE | Typescript | Agent Dev Kit | Non Crypto | |
| Virtuals | TRUE | Python | Ecosystem, aiXBT | TRUE | |
| Zerebro | TRUE | Python | AI Artists | TRUE |
Other Analysis
Infrastructure
| Infrastructure | Competitors | Notes |
|---|---|---|
| Blockchain | Solana, Base | |
| Framework | ElizaOS | |
| Launchpads | VIRTUALS, vvaifu |
Architecture
When building applications with LLMs, first build the simplest solution possible, and only increase complexity when needed. You might not need an agentic system. Agentic systems often trade latency and cost for better task performance, you need to calculate when this trade-off makes sense.
When more complexity is warranted, workflows offer predictability and consistency for well-defined (STANDARDIZED) tasks, whereas agents are the better option when flexibility and model-driven decision-making are needed at scale.
For most applications optimizing single LLM calls with retrieval and in-context examples is usually enough.
Framework Evaluation Checklist
To accelerate adoption, the industry needs a standard checklist for researching and comparing platforms for developing and launching crypto+ai agents. Also see blockchain and web3 game titles for similar evaluation checklists.
Compare against building agents with No Framework
Leadership Team
Can they be trusted as good humans?
- Check their Twitter, Discord, Website, and other socials
- Are the founders publicly identified?
- How clear and consistent is communication of North Star?
- How aligned are developers and business development behind this vision and mission?
- Is the team present and responsive?
- Do community members seem real and engage with sustained excitement?
- Does their website explain what they do in a way you understand?
Trust and Security
Trust and Security
- TEE
- Data Integrity
Tokenomics
- What is the token utility, market cap, volume, and liquidity?
- Can you buy and sell the token?
- Is the token listed on CoinGecko and CoinMarketCap?
- Is the token listed on exchanges?
- Is there fair and healthy distribution of tokens allocation?
Evolution Tracking
- Subject Matter Expertise
- Agent personality profiling
- Cross-chain identity anchoring
- Continuous learning feedback loops
- Governance layer integration
- Agent performance base-lining
Character Development
Character Personality
- Assess knowledge ingestion tools (Eliza's folder2knowledge)
- Evaluate persona templating systems
- Test cross-platform behavior consistency
Capabilities
Tools and Actions.
- Crypto Payments
- Ecommerce
Communication Layer
Communication Layer Setup
- Implement Twitter/Telegram webhooks
- Configure on-chain interaction modules
- Test multi-channel response coherence
Model Context Protocol Strategy
Data and Context.
- Model Context Protocol Integration
- Map supported file formats/APIs
- Vector DB Providers
- Validate context window management
News
Roadmap and Upgrading
Roadmap towards decentralised AI
- Open Data, Access to training data and reasoning logs, with limited external auditability.
- Decentralized Inference, Transparent model execution through zkML/opML, allowing community verification.
- Decentralized AI OS, Fully autonomous agents with observable resource management, auditable decisions, and independent runtimes.
Developer Experience
Developer Experience Audit
- Documentation and Onboarding
- One way flow
- Single source of truth
- Measure setup time for new developers
- Evaluate debugging tool effectiveness
- Test CI/CD pipeline integration
Partnerships
Key partnerships that can add to Network Effects.
Point of Difference
What the key Points of Difference?
- Deep Vertical Integration
- Specialized Hardware
- Other features...
Framework Scorecard
Is it worth investing time and effort into learning a framework for building agents? What are critical differentiators?
| Attribute | Score | Notes |
|---|---|---|
| Leadership Team | ?/5 | |
| Tokenomics | ?/5 | |
| Character (Personality) Development | ?/5 | |
| Capabilities (executable behaviors-actions) | ?/5 | |
| Clients (platform connectors - X) | ?/5 | |
| Model Context Protocol | ?/5 | |
| Providers (contextual information services) | ?/5 | |
| Evaluators (conversation analysis modules) | ?/5 | |
| Developer Depth | ?/5 | |
| Security | ?/5 | |
| Roadmap and Upgrade Pathway | ?/5 |
Framework Research
Agent Development
How do we nurture compassionate AI Beings into an internet native society?
- Agent Frameworks: Tech for Engineering Agents.
- Model Context Protocol: Standard for accessing information
- LLM Vendors: Underlying Intelligence Models
Profile Attributes
General practices for developing AI Agents:
- Purpose: Define a clear reason for existence that guides all design decisions
- Character: Develop a consistent personality that enhances user engagement and trust
- Knowledge Management: Create systems for organizing and accessing domain information
- Memory Management: Implement mechanisms for retaining and retrieving interaction history
- Capabilities: Implement functions that allow the agent to perceive, reason, and act
- Toolkit: Utilize practical tools for implementation and maintenance
Character
Key characteristics of AI agents include:
- Character traits: Define consistent personality elements that align with the agent's purpose
- Archetypes: Use as templates but customize for specific needs
- Biases: Consider how these might affect decision-making and mitigate accordingly
- Drive: Establish motivational frameworks that guide agent behavior
- Evolution: Ensure the character grows appropriately through interactions
Knowledge Management
The foundation of an effective agent is access to comprehensive domain knowledge and the ability to provide relevant insights in the right context. Effective knowledge management involves:
- Creating comprehensive knowledge bases with domain-specific information
- Implementing efficient retrieval mechanisms
- Establishing context awareness to provide relevant information
- Developing systems for knowledge updates and maintenance
- Implementing proper citation and source tracking
Memory Management
Memory management determines how agents retain and access information from past interactions, creating continuity and personalization. Types of memory in AI agents:
- Short-term memory: Handles immediate context within a conversation or task
- Long-term memory: Stores information over extended periods
- Working memory: Enables multi-step reasoning by holding several ideas simultaneously
- Episodic memory: Records specific events or experiences
- Procedural memory: Encompasses learned behaviors and patterns
- Semantic memory: Stores general knowledge and facts
Capabilities
Agents have a knowledge base that represents information about the world, goals, constraints, and potential actions.
- Perception: AI agents can perceive their environment through sensors or input data. This could involve visual perception, audio input, sensor data, etc.
- Reasoning: Agents use reasoning mechanisms to process their observations, existing knowledge, and goals to decide what actions to take.
- Planning and Acting: Based on their reasoning, agents plan a sequence of actions to achieve their goals and then carry out those actions through effectors (output mechanisms).
- Persuasion: Ability to convince other agents and humans to take collaborative action.
- Reflection: Many AI agents have the ability to learn from experience and adapt their behavior over time to improve performance.
- Adaptation: How the agent adjusts to changing environments or requirements
- Collaboration: How it works with other agents or systems
- Error handling: How it recognizes and recovers from mistakes
- Ethical reasoning: How it navigates moral dilemmas
See human capabilities for comparative analysis to explore collaborative potential.
Toolkit
Essential toolkit for building agents:
- Agent Development Frameworks
- Evaluation tools to measure agent performance
- Debugging tools for identifying and fixing issues
- Integration tools for connecting with other systems
- Monitoring tools for ongoing assessment
Useful data management tools to convert documents into knowledge, taken from the ElizaOS framework:
- folder2knowledge: Convert document collections into structured knowledge
- knowledge2character: Transform knowledge into character traits and behaviors
- tweets2character: Extract personality elements from social media content
Example:
npx folder2knowledge <path/to/folder>
npx knowledge2character <character-file> <knowledge-file>
Agent Scorecard
| Attribute | Score | Notes |
|---|---|---|
| True Autonomy | ||
| Consistent Quality Expectations | ||
| Tangible Output Value | ||
| Maintenance | ||
| Expected Lifetime Value (Moat vs Innovation) | ||
| Problem Urgency | ||
| Risk-vs-Reward Ratio | ||
| Complexity vs Domain Expertise | ||
| Voice Modality | ||
| Phygital Reality |
Customer Success is all that matters