Unity Real-time Conversational AI Kit

A comprehensive Unity package collection for implementing real-time conversational AI with speech-to-speech capabilities. Skip the traditional speech-to-text → text-to-speech pipeline and achieve true real-time voice interactions with minimal latency.

Overview

This kit provides Unity packages for integrating OpenAI and ElevenLabs conversational AI services into your Unity projects.

Demos

ElevenLabs

Eleven.Labs.mp4

OpenAI Realtime API + Tools

Line.Art.Tool.Demo.mp4

Documentation

OpenAI

Setup & Configuration

• Installation
• API Key Setup
• Core Prefabs
• Custom Tools
• Events

Sample Scenes

• RealtimeConvSample
• SimpleVoiceActionSample
• PCAConversationalAISample
• AgenticToolsSample
• Quick Start Guide

ElevenLabs

Setup & Configuration

• Installation
• Account & Agent Setup
• Core Prefabs
• Events

Sample Scenes

• ElevenLabsConvAISample
• Quick Start Guide

Requirements

• Unity: 6000.0.53f1 (LTS)
• Meta SDK: 77.0.0
• Platform: Meta Quest, Unity PC

Troubleshooting

Meta SDK 78 Issue: Meta SDK 78 has an issue with microphone permissions. I am working on it. Please use Meta SDK 77.0.0 for now.

License

This project is licensed under the MIT License - see the LICENSE file for details.