> ## Documentation Index
> Fetch the complete documentation index at: https://brokeros.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect Pipecat: Modular Voice Pipelines & WebRTC Framework

> Open-source modular voice and multimodal AI pipelines orchestrated by Daily WebRTC and custom STT, LLM, and TTS transport microservices.

## Overview

Pipecat is an open-source framework created by Daily for building real-time voice, multimodal, and conversational AI agents. In BrokerOS, the Pipecat integration is implemented in [`integrations/voice/agents/pipecat/src/index.ts`](https://github.com/trybrokeros/BrokerOS/blob/main/integrations/voice/agents/pipecat/src/index.ts) under the Turborepo workspace package `@brokeros/int-agent-pipecat`.

BrokerOS administrators configure the Pipecat runner connection directly within the Web UI at **Dashboard -> Marketing -> Voice -> Settings**. All credentials and endpoint URLs are encrypted using AES-256-GCM and stored in PostgreSQL (`VoiceAgentIntegration`).

***

## Architecture and Dispatch Sequence

The Pipecat integration implements the `IVoiceAgentProvider` contract from [`@brokeros/types`](https://github.com/trybrokeros/BrokerOS/blob/main/packages/types). Outbound sessions initialize on a Pipecat runner microservice and connect to the lead via PSTN SIP trunks.

```mermaid theme={null}
sequenceDiagram
    autonumber
    participant Worker as BullMQ Worker
    participant Client as PipecatAgentClient
    participant Runner as Pipecat Runner Microservice
    participant Daily as Daily WebRTC / SIP Gateway
    participant Carrier as PSTN Carrier (Telnyx / Twilio)
    participant Lead as Lead Handset
    participant Deepgram as STT (Deepgram Nova-3)
    participant LLM as LLM (Claude / GPT-4o / Groq)
    participant Cartesia as TTS (Cartesia / 11Labs)

    Worker->>Client: dispatchOutboundCall(options)
    Client->>Runner: POST /dispatch (agent config, variables, script)
    Runner->>Daily: Create WebRTC Room / SIP Endpoint
    Daily->>Carrier: Dial Lead Telephone Handset
    Carrier->>Lead: Outbound Call Rings
    Lead-->>Carrier: Answers Phone
    Carrier-->>Daily: Streams Inbound Audio
    Daily->>Runner: Audio Stream Arrives at Pipeline
    loop Modular Real-Time Processing
        Runner->>Deepgram: Real-time Audio Stream
        Deepgram-->>Runner: Transcribed User Speech
        Runner->>LLM: Prompt + Conversation Context
        LLM-->>Runner: Token Stream
        Runner->>Cartesia: Synthesize Response Audio
        Cartesia-->>Runner: PCM Audio Output
        Runner->>Daily: Audio Return Stream
        Daily->>Carrier: Transmit over PSTN
        Carrier-->>Lead: Plays Synthesized Response
    end
    Lead-->>Carrier: Hangup
    Runner->>Worker: Post-call Analytics Webhook
```

***

## Step 1: Deploy Pipecat Runner & Obtain Credentials

1. Deploy your Pipecat real estate agent container on your cloud infrastructure (AWS ECS, Fly.io, or GCP Cloud Run).
2. Note your runner service endpoint URL (e.g. `https://pipecat.internal.yourfirm.com`).
3. Log into [dashboard.daily.co](https://dashboard.daily.co) to retrieve your Daily.co API key for WebRTC and SIP routing.

***

## Step 2: Connect in the BrokerOS Web Dashboard

In your BrokerOS dashboard:

1. Navigate to **Dashboard -> Marketing -> Voice -> Settings**.
2. Select the **Gateways & Voice Platforms** tab (`/dashboard/marketing/voice/settings?tab=providers`).
3. Click the **AI Voice Platforms** sub-tab.
4. Click **Add Platform** and select **Pipecat AI**.
5. Fill in the modal connection fields:

| UI Modal Field | Required | Description & Example |
| :- | :- | :- |
| **Engine Label / Name** | Yes | Friendly label in CRM (e.g. `Pipecat Modular Pipeline Engine`). |
| **Pipecat API Key** | Yes | Daily.co or runner authentication token. |
| **Server URL** | Yes | Deployed runner endpoint (`https://pipecat.internal.yourfirm.com`). |

6. Click **Save & Verify Engine**. BrokerOS verifies connectivity to the runner, encrypts credentials via AES-256-GCM, and saves them in `VoiceAgentIntegration`.

***

## Integration Implementation Details

The Pipecat client is defined in [`integrations/voice/agents/pipecat/src/index.ts`](https://github.com/trybrokeros/BrokerOS/blob/main/integrations/voice/agents/pipecat/src/index.ts).

### Live Verification Engine

Validation checks the runner configuration and credentials:

```typescript theme={null}
export class PipecatAgentClient implements IVoiceAgentProvider {
  readonly platformType: VoiceAgentPlatform = 'PIPECAT';
  private apiKey: string;
  private serverUrl: string;

  constructor(credentials?: VoiceAgentCredentials) {
    this.apiKey = credentials?.apiKey || process.env.PIPECAT_API_KEY || '';
    this.serverUrl = credentials?.serverUrl || process.env.PIPECAT_RUNNER_URL || '';
  }

  async validateCredentials(credentials?: VoiceAgentCredentials): Promise<boolean> {
    const key = credentials?.apiKey || this.apiKey;
    return !!key && key.length >= 8;
  }
}
```

### Modular Pipeline Dispatch

The dispatcher posts the outbound call configuration to your Pipecat runner service:

```typescript theme={null}
export async function dispatchPipecatCall(
  apiKey: string,
  serverUrl: string,
  options: SendVoiceOptions,
  credentials?: VoiceAgentCredentials,
): Promise<SendVoiceResult> {
  // Posts to Pipecat Runner /dispatch
  // Attaches project variables, prompt instructions, and Daily SIP transport parameters
}
```


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.