AI Call System
A Node.js application that enables AI-powered phone calls using Twilio and OpenAI's Realtime API.
Features
- 🤖 AI-powered phone calls with natural conversation
- 📞 Real-time voice interaction using OpenAI's Realtime API
- 🔄 Interruption handling for natural conversation flow
- 📊 Call status tracking and logging
- 🚀 Easy-to-use functions and CLI interface
Setup
-
Install dependencies:
npm install -
Create a
.envfile with your credentials:# Twilio credentials TWILIO_ACCOUNT_SID=your_twilio_account_sid TWILIO_AUTH_TOKEN=your_twilio_auth_token TWILIO_PHONE_NUMBER=your_twilio_phone_number TO_PHONE_NUMBER=number_to_call # OpenAI credentials OPENAI_API_KEY=your_openai_api_key # Server configuration (optional) PORT=5050 -
Make sure your Twilio phone number is configured for voice calls and webhooks.
Usage
Method 1: Using the Functions Directly
import { makeAICall, startAIServer } from "./src/make_call";
// Start the server
await startAIServer();
// Make an AI call
const callResult = await makeAICall();
console.log("Call SID:", callResult.sid);
Method 2: Using the CLI
Start the server:
npx tsx src/cli.ts server
Make a call:
npx tsx src/cli.ts call
Make a call to a specific number:
npx tsx src/cli.ts call +1234567890
Method 3: Using the Example
npx tsx src/example.ts
API Functions
makeAICall(toNumber?, fromNumber?, serverUrl?)
Initiates an AI call using Twilio.
Parameters:
toNumber(optional): Phone number to call (defaults toTO_PHONE_NUMBERfrom env)fromNumber(optional): Twilio phone number to call from (defaults toTWILIO_PHONE_NUMBERfrom env)serverUrl(optional): URL of your server (defaults tohttp://localhost:5050)
Returns: Promise with call details including SID, status, etc.
startAIServer(port?)
Starts the server that handles AI calls and webhooks.
Parameters:
port(optional): Port to run the server on (defaults to 5050)
How It Works
- Call Initiation: The
makeAICallfunction creates a Twilio call to the specified number - Webhook Handling: When the call connects, Twilio sends a webhook to
/incoming-call - Media Stream: The call connects to a WebSocket stream for real-time audio
- AI Processing: Audio is sent to OpenAI's Realtime API for processing
- Response: AI responses are converted to speech and sent back through the call
Environment Variables
| Variable | Description | Required |
|---|---|---|
TWILIO_ACCOUNT_SID | Your Twilio Account SID | Yes |
TWILIO_AUTH_TOKEN | Your Twilio Auth Token | Yes |
TWILIO_PHONE_NUMBER | Your Twilio phone number | Yes |
TO_PHONE_NUMBER | Default number to call | Yes |
OPENAI_API_KEY | Your OpenAI API key | Yes |
PORT | Server port (default: 5050) | No |
Development
Type checking:
npm run type-check
Formatting:
npm run format
Linting:
npm run lint:fix
Troubleshooting
- "Missing Twilio credentials": Make sure all required environment variables are set in your
.envfile - "Call failed": Check that your Twilio phone number is configured for voice calls
- "Server not accessible": Ensure your server is publicly accessible for Twilio webhooks (use ngrok for local development)
Local Development with ngrok
For local development, you'll need to expose your local server to the internet:
# Install ngrok
npm install -g ngrok
# Start your server
npx ts-node src/cli.ts server
# In another terminal, expose your server
ngrok http 5050
# Use the ngrok URL in your .env file or when calling makeAICall