Skip to main content

Software Architecture

What are the key principles for architecting systems that enable innovation and are performant?

ComponentTechnology/PatternPurpose
FrontendNext.js (TypeScript)User interface, real-time chat, analytics
BackendNext.js, PostgreSQL, RedisAPI, data persistence, caching, auth
BlockchainSolana, SUI, EthereumPermissonless Capital, Proof of Personhood
AI/LLMAI SDK, LLM of choiceAgent logic, knowledge supplementation
ObservabilityLangFuse, EvaliteTracing, logging, automated evaluation
Agent ArchitectureTask decomposition, workflowModular, scalable, maintainable agent logic
Feedback LoopUser feedback integrationContinuous improvement of agent performance

Refer to System Design Primer

Intents and Purposes​

Build a Platform to reinvent capitalism where the best ideas win to elevate global standards for living a meaningful life.

Key Requirements:

  • Production-Grade Infrastructure: Databases, caching, authentication, and persistent chat.
  • Observability & Evaluation: Comprehensive logging, tracing, and automated evaluation of LLM outputs.
  • Modular Agent Architecture: Support for complex, multi-step AI processes and workflow optimization.

Architectural Components​

Frontend​

  • Framework: Next.js (TypeScript)
  • UI Features:
    • Real-time chat interface
    • Responsive and seamless user experience
    • Display analytics and metrics

Backend​

  • Application Server: Next.js API routes
  • Database: PostgreSQL (via Drizzle ORM)
  • Caching: Redis
  • Authentication: Secure user authentication (implementation details to be defined)
  • Chat Persistence: All conversations saved to the database

AI/LLM Layer​

  • LLM Integration: Connect to an LLM of choice using the AI SDK
  • Agent Logic: Naive agent initially, evolving to modular, task-decomposed architecture
  • Tools: Search tool for knowledge supplementation

Observability & Evaluation​

  • Tracing: Integrated with LangFuse for LLM call tracing
  • Evaluation: Use Evalite (open-source, vitest-based runner) for automated testing of agent outputs
  • Feedback Loop: Capture user feedback and feed into traces for continuous improvement

Agent/Workflow Orchestration​

  • Task Decomposition: LLM determines next steps, delegates actions to specialized or cheaper models as needed
  • Workflow Patterns: Evaluator-optimizer loop for reliable, context-aware responses

Key Patterns​

  • Modular Agent Design: Avoid monolithic prompts; decompose tasks and delegate responsibilities for maintainability and scalability.
  • Evaluator-Optimizer Loop: Agent evaluates whether it has enough information to answer; if not, it searches for more data before responding.
  • Observability-Driven Development: Continuous monitoring, logging, and evaluation to ensure quality and reliability.
  • Feedback Integration: User feedback is captured and used to improve agent performance over time.

Data Flow​

Proprietary data is lifeblood of intelligence, but it takes culture to insights into valuable actions.

  1. User Interaction: User submits query via UI.
  2. Chat Persistence: Query and conversation history saved to PostgreSQL.
  3. Agent Processing: LLM agent processes query, using search tools if needed.
  4. Tracing & Logging: All LLM calls and agent decisions are traced via LangFuse.
  5. Response Generation: Agent generates response, possibly using task decomposition or workflow orchestration.
  6. Evaluation: Response is evaluated automatically (Evalite) and by user feedback.
  7. Feedback Integration: Negative feedback is captured and fed back into the system for improvement.
  8. UI Update: Response displayed to user; analytics/metrics updated.

Non-Functional Requirements​

  • Scalability: Modular design supports growth in complexity and user load.
  • Reliability: Observability and evaluation ensure consistent, high-quality outputs.
  • Maintainability: Avoid monolithic prompts; use task decomposition and workflow patterns.
  • Security: Secure authentication and data persistence.
  • Extensibility: Designed for easy integration of new tools, models, or feedback mechanisms.

Future-Proofing​

Use First Principles Thinking to architect systems that are resilient, scalable, and adaptable to change.

  • Leverage Multiplication: Every system should amplify output
  • Intent-Driven Design: Intent must be explicit and traceable
  • Separation of Concerns: Clean boundaries enable evolution
  • Feedback Loops: Every implementation generates learning

System Design Components​

Data Flow Optimisation​

Anyone can start building anything.

Not everyone will understand how to avoid technical debt.

During code reviews ask...

  1. How can I optimize data flow.
  2. How can minimize the cognitive load for developers to understand this code

Checklist

  1. Focus on data flow and locality:
    • Modern programming paradigms often hide data flow and locality, which are crucial for performance.
    • Systems should be designed with data flow in mind from the start.
    • Locality is critical for high-performance computing.
  2. Resource planning and allocation:
    • Pre-allocate resources (memory, cores) at job startup rather than relying on dynamic allocation.
    • Be explicit about resource requirements and limitations.
    • Avoid using features like malloc that pretend resources are infinite.
  3. Use a tile architecture:
    • Assign specific tasks to dedicated cores or "tiles".
    • This allows for specialization, efficient caching, and deterministic performance.
    • Provides natural security boundaries between components.
  4. Rethink abstraction layers:
    • Many modern APIs and operating systems hide important low-level details.
    • Create abstractions that allow developers to express locality and data flow without needing to understand all low-level details.
  5. Optimize for real-world constraints:
    • Consider physical limitations like the speed of light.
    • Design systems with concurrency budgets in mind (e.g., number of instruction pointers available).
  6. Encourage proper resource management:
    • Provide developers with tools and abstractions that naturally lead to good resource allocation practices.
    • Avoid relying on the operating system or language runtime to magically handle resource management.
  7. Focus on reliability and determinism:
    • Design systems to be hyper-deterministic and reliable.
    • This is crucial for financial applications that require quick, consistent responses.
  8. Plan for scale from the beginning:
    • For systems like Solana, think in terms of millions of transactions per second and huge numbers of accounts.
    • Design for quick recovery from outages.
  9. Rethink networking protocols:
    • Existing protocols like QUIC may not be optimal for validator needs.
    • Consider designing purpose-built protocols for high-performance blockchain systems.
  10. Address implicit behaviors:
    • Spend time identifying and specifying all implicit behaviors in the system.
    • This is crucial for creating a reliable and consistent implementation across different clients.

Crypto Architecture​

Context​