🤖 Explicar con IA

The Human Team¶

At the core of 4Geeks AI Agents is a fundamental principle: AI works best when guided by real humans. Every AI Agent deployed on the platform is backed by a dedicated team of real human geeks at 4Geeks who handle setup, monitoring, quality assurance, and continuous improvement. This document explains who these people are, how they work, and why the human-in-the-loop approach is our key differentiator.

Why Human-in-the-Loop?¶

Fully autonomous AI agents are unreliable for real-world business operations. They hallucinate, misinterpret context, fail on edge cases, and cannot adapt to novel situations without guidance. The human-in-the-loop (HITL) model solves these problems by keeping real humans in the driver’s seat at every stage of the agent lifecycle.

The Problem with Fully Autonomous AI¶

The 4Geeks Solution¶

Our human team addresses every one of these challenges through proactive monitoring, real-time intervention, and continuous learning:

graph TB
    subgraph Deploy["Deployment Phase"]
        DM[Discovery Meeting]
        SC[Solution Configuration]
        INT[Manual Integration]
        TEST[Testing & Calibration]
        GO[Go-Live]
    end

    subgraph Monitor["Monitoring Phase"]
        REAL[Real-Time Monitoring]
        QA[Quality Assurance Reviews]
        PERF[Performance Analytics]
        ALERT[Alert Management]
    end

    subgraph Improve["Improvement Phase"]
        ESC[Escalation Handling]
        LEARN[Learning from Resolutions]
        OPT[Prompt Optimization]
        UPDATE[Knowledge Base Updates]
    end

    DM --> SC --> INT --> TEST --> GO
    GO --> REAL
    REAL --> QA
    REAL --> PERF
    REAL --> ALERT
    QA --> ESC
    PERF --> OPT
    ALERT --> ESC
    ESC --> LEARN
    LEARN --> OPT
    OPT --> UPDATE
    UPDATE --> REAL

Team Structure¶

The 4Geeks AI Agents team is organized into specialized roles, each responsible for a specific aspect of the agent lifecycle.

Account Managers¶

Account Managers are your primary point of contact throughout the entire engagement.

Responsibilities:

• Conduct discovery meetings to understand your business needs, goals, and success metrics
• Translate business requirements into technical specifications for agent configuration
• Coordinate between your team and the technical implementation team
• Provide regular performance reports and strategic recommendations
• Manage billing, plan adjustments, and contract renewals
• Serve as escalation point for any issues or concerns

When you interact with them:

• Initial discovery and onboarding
• Regular check-ins (weekly or bi-weekly depending on plan)
• Performance reviews and strategy sessions
• Any time you need to make changes to your agents or plan

Typical background:

• Experience in SaaS account management or customer success
• Understanding of AI and automation concepts
• Industry knowledge across multiple verticals (healthcare, real estate, e-commerce, finance, etc.)
• Fluent in English and Spanish; additional languages supported

Integration Engineers¶

Integration Engineers are the technical specialists who connect AI Agents to your existing software stack.

Responsibilities:

• Design and implement custom integrations with your CRM, calendar, communication tools, and proprietary systems
• Write custom code to ensure secure and reliable data flow between systems
• Configure API connections, webhooks, and data synchronization pipelines
• Test integrations end-to-end to verify data accuracy and reliability
• Troubleshoot integration issues and implement fixes
• Document all integration configurations for future reference

Technologies they work with:

Typical background:

• Senior software engineering experience (5+ years)
• Expertise in API integration and middleware development
• Proficiency in Python, JavaScript/TypeScript, and SQL
• Experience with Supabase, PostgreSQL, and cloud platforms
• Background in building production-grade integrations

Prompt Engineers¶

Prompt Engineers craft and optimize the system prompts that define how each AI Agent behaves.

Responsibilities:

• Design initial system prompts based on business requirements and brand guidelines
• Test prompts extensively across hundreds of conversation scenarios
• Optimize prompts for accuracy, tone, consistency, and cost-efficiency
• Implement prompt versioning and A/B testing to measure improvements
• Analyze conversation logs to identify prompt weaknesses
• Update prompts based on escalation patterns and customer feedback
• Create prompt templates for new agent types and use cases

Prompt engineering process:

graph LR
    A[Gather Requirements] --> B[Draft Initial Prompt]
    B --> C[Internal Testing<br/>100+ scenarios]
    C --> D{Quality Check}
    D -->|Pass| E[Client Review]
    D -->|Fail| B
    E --> F{Client Approval}
    F -->|Approved| G[Deploy to Production]
    F -->|Changes| B
    G --> H[Monitor Performance]
    H --> I[Identify Improvements]
    I --> B

What goes into a system prompt:

Typical background:

• Experience in NLP, computational linguistics, or technical writing
• Deep understanding of LLM behavior and prompt engineering techniques
• Ability to analyze conversation data and identify patterns
• Experience with multiple LLM providers (OpenAI, Anthropic, Google)
• Strong attention to detail and commitment to quality

Quality Assurance Specialists¶

QA Specialists continuously monitor agent performance and ensure quality standards are met.

Responsibilities:

• Review a random sample of agent conversations daily for quality, accuracy, and brand consistency
• Score conversations against a quality rubric (accuracy, helpfulness, tone, completeness)
• Identify patterns in customer complaints or negative feedback
• Flag conversations that require human follow-up
• Create quality reports with trends and recommendations
• Validate that agents comply with regulatory requirements (GDPR, EU AI Act)
• Test agent responses after prompt or knowledge base updates

Quality scoring rubric:

Quality targets:

Typical background:

• Experience in QA, customer service management, or operations
• Strong analytical skills and attention to detail
• Understanding of AI capabilities and limitations
• Experience with customer service metrics and KPIs
• Ability to provide constructive feedback for improvement

Operations & Monitoring Team¶

The Operations team ensures the platform infrastructure runs smoothly 24/7.

Responsibilities:

• Monitor all platform services for availability, latency, and error rates
• Manage LLM provider relationships and handle provider outages
• Configure and maintain the Private AI Gateway routing and fallback logic
• Manage credit consumption monitoring and alerting
• Handle incident response and post-incident reviews
• Maintain backup and disaster recovery procedures
• Optimize platform performance and cost-efficiency

Monitoring coverage:

Typical background:

• DevOps / SRE experience
• Expertise in cloud infrastructure (AWS, GCP, Supabase)
• Experience with monitoring tools and incident management
• Understanding of LLM provider APIs and rate limiting
• On-call rotation experience

Escalation Workflow¶

When an AI Agent encounters a situation it cannot handle confidently, a structured escalation process ensures the customer receives proper assistance while the agent learns from the experience.

Escalation Triggers¶

An escalation is triggered when any of the following conditions are met:

Escalation Process¶

sequenceDiagram
    participant AI as AI Agent
    participant HITL as HITL Orchestrator
    participant Q as Escalation Queue
    participant H as Human Specialist
    participant C as Customer
    participant KB as Knowledge Base

    AI->>AI: Detect escalation trigger
    AI->>C: "Let me connect you with a specialist who can help..."
    AI->>HITL: Submit escalation with full context
    HITL->>HITL: Assign priority (based on trigger type, customer tier, wait time)
    HITL->>Q: Add to escalation queue
    HITL->>C: Confirm transfer and set expectations
    Q->>H: Route to available specialist
    H->>H: Review conversation history + AI context + suggested action
    H->>C: Continue conversation with customer
    H->>H: Resolve the issue
    H->>KB: Log resolution as training example
    H->>HITL: Mark escalation as resolved
    HITL->>AI: Feed resolution back to agent for learning

Escalation Priority Levels¶

Learning from Escalations¶

Every resolved escalation becomes a learning opportunity:

1. Resolution Capture — The human specialist’s response is recorded along with the full conversation context
2. Pattern Analysis — The QA team reviews escalations weekly to identify recurring patterns
3. Prompt Updates — If a pattern reveals a gap in the agent’s instructions, the prompt is updated
4. Knowledge Base Updates — If the escalation was caused by missing information, the knowledge base is updated with new documents
5. Guardrail Refinement — If the escalation was caused by the agent overstepping its boundaries, guardrails are tightened
6. Benchmark Addition — The escalation scenario is added to the testing benchmark to ensure the agent handles it correctly after improvement

Onboarding Process¶

When you become a 4Geeks AI Agents customer, here is the detailed onboarding journey:

Week 1: Discovery & Planning¶

Week 2: Build & Integrate¶

Week 3: Testing & Calibration¶

Week 4: Go-Live & Monitoring¶

Support & Communication¶

Support Channels¶

Regular Reporting¶

Depending on your plan, you receive regular performance reports:

What’s Included in a Performance Report¶

Continuous Improvement Cycle¶

The human-in-the-loop model creates a virtuous cycle of continuous improvement:

graph LR
    A[Deploy Agent] --> B[Monitor Interactions]
    B --> C[Identify Issues]
    C --> D{Issue Type?}
    D -->|Prompt Gap| E[Update Prompt]
    D -->|Knowledge Gap| F[Add to Knowledge Base]
    D -->|Integration Bug| G[Fix Integration]
    D -->|Workflow Gap| H[Add Workflow Step]
    E --> I[Test Changes]
    F --> I
    G --> I
    H --> I
    I --> J{Quality Improved?}
    J -->|Yes| B
    J -->|No| C

Improvement Metrics Over Time¶

As the continuous improvement cycle runs, you should see measurable improvements:

Team Availability¶

Standard Hours¶

Extended Coverage¶

• Growth Plan: Extended hours until 9:00 PM in your timezone
• Scale Plan: Extended hours until 11:00 PM in your timezone
• Enterprise Plan: 24/7 coverage with dedicated on-call team

Holidays & Time Off¶

• The platform (AI Agents) runs 24/7/365 regardless of team availability
• Human escalation response times may be longer during holidays
• Enterprise customers have guaranteed holiday coverage as part of their SLA
• Scheduled maintenance windows are communicated 48 hours in advance

What’s Next¶

• Cloud & LLM Architecture — Technical infrastructure powering the platform
• LLM Models — Detailed breakdown of the language models used
• 4Geeks AI Agents Overview — Return to the main documentation
• FAQs — Common questions about the platform

Aún con dudas? Pregunta en Discord o explore tutoriales