Skip to content

🤖 Explain with AI

Cross-Channel Orchestration

Enable seamless conversation handoffs between WhatsApp, Voice, and SMS channels with intelligent intent detection and full context preservation.

Overview

Cross-Channel Orchestration provides:

  • Intent Detection — Automatically detect when users want to switch channels
  • Context Passing — Preserve conversation history and key topics during handoffs
  • Rate Limiting — Prevent abuse with per-phone rate limits
  • Fallback Chains — Automatic channel fallback when primary channel fails
  • Per-Agent Configuration — Customize fallback behavior per agent
  • Idempotent Execution — Prevent duplicate handoffs

Supported Channels

Channel As Source As Target
WhatsApp Yes Yes
Voice Yes Yes
SMS Yes Yes

How Handoffs Work

End-to-End Flow

  1. Detection — Either the voice engine’s regex classifier or the orchestrator’s LLM classifier identifies a channel-switch intent
  2. Rate Limit Check — System verifies the phone number hasn’t exceeded the handoff limit
  3. Context Building — Load conversation history from the source channel, generate summary, extract key topics
  4. Handoff Record — Create a tracking record with status “pending”
  5. Channel Dispatch — Send message via target channel (WhatsApp message, voice call, or SMS)
  6. Status Progression — pending → triggered → delivered
  7. Rate Limit Recording — Increment the counter for rate limiting

Intent Detection

LLM-Based Detection

The orchestrator uses an AI model to classify whether the user wants to switch channels:

Aspect Detail
Model Advanced language model
Context Last 10 messages plus the latest message
Temperature 0.1 for consistent classification
Confidence Threshold 0.85 minimum
Output JSON: { shouldTrigger, confidence, reason }

Regex-Based Detection (Voice)

During voice calls, the system uses regex patterns to detect channel-switch intent:

Trigger Pattern Example
“Call me” “Can you call me back?”
“Send text” “Send me a text message”
“Switch channel” “Let’s continue via chat”
Spanish equivalents “Llámame”, “Enviar texto”

If confidence meets the threshold, a handoff record is created immediately.

Context Building

When a handoff is triggered, the system builds rich context:

From WhatsApp

Data Description
Conversation History Last 20 messages
Summary AI-generated conversation summary
Key Topics Extracted topics (appointment, payment, support, etc.)
Action Items Follow-up tasks identified

From Voice

Data Description
Call Transcript Full conversation transcript
Summary AI-generated summary
Key Topics Extracted topics
Action Items Follow-up tasks
Sentiment Conversation sentiment analysis

From SMS

SMS as a source channel is not supported for context building.

Channel Dispatch

WhatsApp Target

Sends a personalized text message via WhatsApp:

Hi! This is {agentName} from our support team. I see you were just
chatting with our assistant about: {summary}. I'm reaching out to
help you with this directly. Is now a good time to talk?
(If you'd prefer to continue via chat, just reply 'chat' and I'll
switch back.)

Voice Target

Initiates an outbound voice call via the 4Geeks Voice Engine with injected context:

  • Conversation summary
  • Key topics
  • Action items
  • Source channel information
  • Customer and agent names

SMS Target

Sends an SMS message:

Hi! This is a message from {agentName}. {summary}.
Reply HELP for support or STOP to unsubscribe.

Rate Limiting

How It Works

Parameter Value
Max Handoffs 3 per 10 minutes
Window Sliding 10-minute window
Scope Per phone number

Implementation

  • Each handoff attempt increments the counter
  • Old windows are cleaned automatically
  • If limit is reached, new handoffs are rejected
  • Rate limit status is visible in the dashboard

Fallback Chains

WhatsApp Failure → SMS Fallback

When WhatsApp delivery fails due to: - Meta restrictions - Rate limits - Account issues - Template rejections

The system can automatically fall back to SMS if configured.

Fallback Configuration

Setting Description
Fallback Enabled Enable/disable fallback for this agent
Fallback Channel Target channel for fallback (voice/sms)
Intent Threshold Confidence threshold for handoff detection

Fallback Records

When a fallback occurs: 1. A new handoff record is created with is_fallback = true 2. The original handoff is linked via original_handoff_id 3. A unique idempotency key prevents duplicates

Per-Agent Configuration

Each agent can independently configure:

Setting Options
Fallback Enabled On/Off
Fallback Channel WhatsApp, Voice, SMS
Intent Threshold 0.0–1.0 (default: 0.85)

Configuration Steps

  1. Navigate to AI Agents → My Agents
  2. Select an agent
  3. Go to Settings → Cross-Channel
  4. Configure fallback behavior
  5. Save changes

Handoff Status

Status Description
Pending Handoff created, not yet dispatched
Triggered Handoff dispatched to target channel
Delivered Target channel confirmed delivery
Failed Handoff failed to deliver
Fallback Initiated Primary channel failed, fallback triggered
Cancelled Handoff manually cancelled

Idempotency

Each handoff gets a unique idempotency key:

Source Key Format
Regular handoff {sourceChannel}:{sourceSessionId}:{timestamp}
Fallback handoff sms_fallback:{originalId}:{timestamp}

This prevents duplicate handoff execution even if the same request is sent multiple times.

Data Models

cross_channel_handoffs

Column Description
id Unique identifier
source_channel WhatsApp, Voice, or SMS
target_channel WhatsApp, Voice, or SMS
source_session_id Session ID in source channel
target_session_id Session ID in target channel
source_agent_id Agent in source channel
target_agent_id Agent in target channel
customer_phone Customer phone number
intent_confidence Detection confidence (0-1)
intent_reason Why the handoff was triggered
context_summary Conversation summary
status Current handoff status
idempotency_key Unique key preventing duplicates
is_fallback Whether this is a fallback handoff
fallback_from_channel Original channel (if fallback)
fallback_reason Why fallback was triggered
original_handoff_id Link to original handoff (if fallback)
triggered_at When handoff was dispatched
delivered_at When delivery was confirmed
error_message Failure reason (if any)
retry_count Number of retry attempts

sms_config

Column Description
workspace_id Workspace scope
twilio_account_sid SMS provider account
twilio_auth_token SMS provider auth
twilio_phone_number Sending phone number
fallback_enabled Enable SMS fallback
fallback_to_sms_on_whatsapp_fail Auto-fallback on WhatsApp failure
fallback_to_whatsapp_on_sms_fail Auto-fallback on SMS failure
daily_sms_limit Maximum SMS per day (default: 100)

handoff_rate_limits

Column Description
phone_number Phone number
window_start Start of 10-minute window
handoff_count Number of handoffs in window

Best Practices

  1. Set appropriate thresholds — Higher thresholds (0.9) reduce false positives; lower (0.7) catch more intents
  2. Configure fallbacks — Always have a fallback channel for critical agents
  3. Monitor rate limits — Track handoff volume to ensure limits aren’t too restrictive
  4. Test handoffs — Use the playground to verify handoff behavior
  5. Review context — Ensure context summaries are accurate and complete

What’s Next

Still questions? Ask on Discord or explore tutorials