diff --git a/README.md b/README.md
index 0a76ff5eae..e6302c0034 100644
--- a/README.md
+++ b/README.md
@@ -2,7 +2,7 @@
-[](https://pypi.org/project/pipecat-ai)  [](https://codecov.io/gh/pipecat-ai/pipecat) [](https://docs.pipecat.ai) [](https://discord.gg/pipecat) [](https://deepwiki.com/pipecat-ai/pipecat)
+[](https://pypi.org/project/pipecat-ai)  [](https://codecov.io/gh/pipecat-ai/pipecat) [](https://docs.pipecat.ai) [](https://discord.gg/pipecat)
# 🎙️ Pipecat: Real-Time Voice & Multimodal AI Agents
@@ -21,8 +21,6 @@
🧭 Looking to build structured conversations? Check out [Pipecat Flows](https://github.com/pipecat-ai/pipecat-flows) for managing complex conversational states and transitions.
-🔍 Looking for help debugging your pipeline and processors? Check out [Whisker](https://github.com/pipecat-ai/whisker), a real-time Pipecat debugger.
-
## 🧠 Why Pipecat?
- **Voice-first**: Integrates speech recognition, text-to-speech, and conversation handling
@@ -30,41 +28,6 @@
- **Composable Pipelines**: Build complex behavior from modular components
- **Real-Time**: Ultra-low latency interaction with different transports (e.g. WebSockets or WebRTC)
-## 📱 Client SDKs
-
-You can connect to Pipecat from any platform using our official SDKs:
-
-
-
## 🎬 See it in action
@@ -75,12 +38,23 @@ You can connect to Pipecat from any platform using our official SDKs:
+## 📱 Client SDKs
+
+You can connect to Pipecat from any platform using our official SDKs:
+
+| Platform | SDK Repo | Description |
+| -------- | ------------------------------------------------------------------------------ | -------------------------------- |
+| Web | [pipecat-client-web](https://github.com/pipecat-ai/pipecat-client-web) | JavaScript and React client SDKs |
+| iOS | [pipecat-client-ios](https://github.com/pipecat-ai/pipecat-client-ios) | Swift SDK for iOS |
+| Android | [pipecat-client-android](https://github.com/pipecat-ai/pipecat-client-android) | Kotlin SDK for Android |
+| C++ | [pipecat-client-cxx](https://github.com/pipecat-ai/pipecat-client-cxx) | C++ client SDK |
+
## 🧩 Available services
| Category | Services |
| ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/stt/elevenlabs), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [NVIDIA Riva](https://docs.pipecat.ai/server/services/stt/riva), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [SambaNova (Whisper)](https://docs.pipecat.ai/server/services/stt/sambanova), [Soniox](https://docs.pipecat.ai/server/services/stt/soniox), [Speechmatics](https://docs.pipecat.ai/server/services/stt/speechmatics), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
-| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [Mistral](https://docs.pipecat.ai/server/services/llm/mistral), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [SambaNova](https://docs.pipecat.ai/server/services/llm/sambanova) [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
+| Speech-to-Text | [AssemblyAI](https://docs.pipecat.ai/server/services/stt/assemblyai), [AWS](https://docs.pipecat.ai/server/services/stt/aws), [Azure](https://docs.pipecat.ai/server/services/stt/azure), [Cartesia](https://docs.pipecat.ai/server/services/stt/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/stt/deepgram), [Fal Wizper](https://docs.pipecat.ai/server/services/stt/fal), [Gladia](https://docs.pipecat.ai/server/services/stt/gladia), [Google](https://docs.pipecat.ai/server/services/stt/google), [Groq (Whisper)](https://docs.pipecat.ai/server/services/stt/groq), [NVIDIA Riva](https://docs.pipecat.ai/server/services/stt/riva), [OpenAI (Whisper)](https://docs.pipecat.ai/server/services/stt/openai), [SambaNova (Whisper)](https://docs.pipecat.ai/server/services/stt/sambanova), [Soniox](https://docs.pipecat.ai/server/services/stt/soniox), [Speechmatics](https://docs.pipecat.ai/server/services/stt/speechmatics), [Ultravox](https://docs.pipecat.ai/server/services/stt/ultravox), [Whisper](https://docs.pipecat.ai/server/services/stt/whisper) |
+| LLMs | [Anthropic](https://docs.pipecat.ai/server/services/llm/anthropic), [AWS](https://docs.pipecat.ai/server/services/llm/aws), [Azure](https://docs.pipecat.ai/server/services/llm/azure), [Cerebras](https://docs.pipecat.ai/server/services/llm/cerebras), [DeepSeek](https://docs.pipecat.ai/server/services/llm/deepseek), [Fireworks AI](https://docs.pipecat.ai/server/services/llm/fireworks), [Gemini](https://docs.pipecat.ai/server/services/llm/gemini), [Grok](https://docs.pipecat.ai/server/services/llm/grok), [Groq](https://docs.pipecat.ai/server/services/llm/groq), [Mistral](https://docs.pipecat.ai/server/services/llm/mistral), [Nebius](https://docs.pipecat.ai/server/services/llm/nebius), [NVIDIA NIM](https://docs.pipecat.ai/server/services/llm/nim), [Ollama](https://docs.pipecat.ai/server/services/llm/ollama), [OpenAI](https://docs.pipecat.ai/server/services/llm/openai), [OpenRouter](https://docs.pipecat.ai/server/services/llm/openrouter), [Perplexity](https://docs.pipecat.ai/server/services/llm/perplexity), [Qwen](https://docs.pipecat.ai/server/services/llm/qwen), [SambaNova](https://docs.pipecat.ai/server/services/llm/sambanova) [Together AI](https://docs.pipecat.ai/server/services/llm/together) |
| Text-to-Speech | [Async](https://docs.pipecat.ai/server/services/tts/asyncai), [AWS](https://docs.pipecat.ai/server/services/tts/aws), [Azure](https://docs.pipecat.ai/server/services/tts/azure), [Cartesia](https://docs.pipecat.ai/server/services/tts/cartesia), [Deepgram](https://docs.pipecat.ai/server/services/tts/deepgram), [ElevenLabs](https://docs.pipecat.ai/server/services/tts/elevenlabs), [Fish](https://docs.pipecat.ai/server/services/tts/fish), [Google](https://docs.pipecat.ai/server/services/tts/google), [Groq](https://docs.pipecat.ai/server/services/tts/groq), [Inworld](https://docs.pipecat.ai/server/services/tts/inworld), [LMNT](https://docs.pipecat.ai/server/services/tts/lmnt), [MiniMax](https://docs.pipecat.ai/server/services/tts/minimax), [Neuphonic](https://docs.pipecat.ai/server/services/tts/neuphonic), [NVIDIA Riva](https://docs.pipecat.ai/server/services/tts/riva), [OpenAI](https://docs.pipecat.ai/server/services/tts/openai), [Piper](https://docs.pipecat.ai/server/services/tts/piper), [PlayHT](https://docs.pipecat.ai/server/services/tts/playht), [Rime](https://docs.pipecat.ai/server/services/tts/rime), [Sarvam](https://docs.pipecat.ai/server/services/tts/sarvam), [XTTS](https://docs.pipecat.ai/server/services/tts/xtts) |
| Speech-to-Speech | [AWS Nova Sonic](https://docs.pipecat.ai/server/services/s2s/aws), [Gemini Multimodal Live](https://docs.pipecat.ai/server/services/s2s/gemini), [OpenAI Realtime](https://docs.pipecat.ai/server/services/s2s/openai) |
| Transport | [Daily (WebRTC)](https://docs.pipecat.ai/server/services/transport/daily), [FastAPI Websocket](https://docs.pipecat.ai/server/services/transport/fastapi-websocket), [SmallWebRTCTransport](https://docs.pipecat.ai/server/services/transport/small-webrtc), [WebSocket Server](https://docs.pipecat.ai/server/services/transport/websocket-server), Local |
@@ -88,7 +62,7 @@ You can connect to Pipecat from any platform using our official SDKs:
| Video | [HeyGen](https://docs.pipecat.ai/server/services/video/heygen), [Tavus](https://docs.pipecat.ai/server/services/video/tavus), [Simli](https://docs.pipecat.ai/server/services/video/simli) |
| Memory | [mem0](https://docs.pipecat.ai/server/services/memory/mem0) |
| Vision & Image | [fal](https://docs.pipecat.ai/server/services/image-generation/fal), [Google Imagen](https://docs.pipecat.ai/server/services/image-generation/fal), [Moondream](https://docs.pipecat.ai/server/services/vision/moondream) |
-| Audio Processing | [Silero VAD](https://docs.pipecat.ai/server/utilities/audio/silero-vad-analyzer), [Krisp](https://docs.pipecat.ai/server/utilities/audio/krisp-filter), [Koala](https://docs.pipecat.ai/server/utilities/audio/koala-filter), [ai-coustics](https://docs.pipecat.ai/server/utilities/audio/aic-filter) |
+| Audio Processing | [Silero VAD](https://docs.pipecat.ai/server/utilities/audio/silero-vad-analyzer), [Krisp](https://docs.pipecat.ai/server/utilities/audio/krisp-filter), [Koala](https://docs.pipecat.ai/server/utilities/audio/koala-filter), [ai-coustics](https://docs.pipecat.ai/server/utilities/audio/aic-filter), [Noisereduce](https://docs.pipecat.ai/server/utilities/audio/noisereduce-filter) |
| Analytics & Metrics | [OpenTelemetry](https://docs.pipecat.ai/server/utilities/opentelemetry), [Sentry](https://docs.pipecat.ai/server/services/analytics/sentry) |
📚 [View full services documentation →](https://docs.pipecat.ai/server/services/supported-services)
@@ -155,11 +129,7 @@ You can get started with Pipecat running on your local machine, then move your a
2. Install development and testing dependencies:
```bash
- uv sync --group dev --all-extras \
- --no-extra gstreamer \
- --no-extra krisp \
- --no-extra local \
- --no-extra ultravox # (ultravox not fully supported on macOS)
+ uv sync --group dev --all-extras --no-extra gstreamer --no-extra krisp --no-extra local
```
3. Install the git pre-commit hooks:
@@ -168,6 +138,23 @@ You can get started with Pipecat running on your local machine, then move your a
uv run pre-commit install
```
+### Python 3.13+ Compatibility
+
+Some features require PyTorch, which doesn't yet support Python 3.13+. Install using:
+
+```bash
+uv sync --group dev --all-extras \
+ --no-extra gstreamer \
+ --no-extra krisp \
+ --no-extra local \
+ --no-extra local-smart-turn \
+ --no-extra mlx-whisper \
+ --no-extra moondream \
+ --no-extra ultravox
+```
+
+> **Tip:** For full compatibility, use Python 3.12: `uv python pin 3.12`
+
> **Note**: Some extras (local, gstreamer) require system dependencies. See documentation if you encounter build errors.
### Running tests
@@ -251,4 +238,4 @@ We aim to review all contributions promptly and provide constructive feedback to
➡️ [Read the docs](https://docs.pipecat.ai)
-➡️ [Reach us on X](https://x.com/pipecat_ai)
+➡️ [Reach us on X](https://x.com/pipecat_ai)
\ No newline at end of file
diff --git a/env.example b/env.example
index eb6dcc5c5e..70535a269f 100644
--- a/env.example
+++ b/env.example
@@ -129,6 +129,9 @@ TWILIO_AUTH_TOKEN=...
MINIMAX_API_KEY=...
MINIMAX_GROUP_ID=...
+# Nebius AI Studio
+NEBIUS_API_KEY=...
+
# Sarvam AI
SARVAM_API_KEY=...
diff --git a/examples/foundational/14x-function-calling-nebius.py b/examples/foundational/14x-function-calling-nebius.py
new file mode 100644
index 0000000000..4bb9e6fa74
--- /dev/null
+++ b/examples/foundational/14x-function-calling-nebius.py
@@ -0,0 +1,170 @@
+#
+# Copyright (c) 2024–2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+
+import os
+
+from dotenv import load_dotenv
+from loguru import logger
+
+from pipecat.adapters.schemas.function_schema import FunctionSchema
+from pipecat.adapters.schemas.tools_schema import ToolsSchema
+from pipecat.audio.vad.silero import SileroVADAnalyzer
+from pipecat.frames.frames import TTSSpeakFrame
+from pipecat.pipeline.pipeline import Pipeline
+from pipecat.pipeline.runner import PipelineRunner
+from pipecat.pipeline.task import PipelineParams, PipelineTask
+from pipecat.processors.aggregators.llm_response import LLMUserAggregatorParams
+from pipecat.processors.aggregators.openai_llm_context import OpenAILLMContext
+from pipecat.runner.types import RunnerArguments
+from pipecat.runner.utils import create_transport
+from pipecat.services.cartesia.tts import CartesiaTTSService
+from pipecat.services.deepgram.stt import DeepgramSTTService
+from pipecat.services.llm_service import FunctionCallParams
+from pipecat.services.nebius.llm import NebiusLLMService
+from pipecat.transports.base_transport import BaseTransport, TransportParams
+from pipecat.transports.network.fastapi_websocket import FastAPIWebsocketParams
+from pipecat.transports.services.daily import DailyParams
+
+load_dotenv(override=True)
+
+
+async def fetch_weather_from_api(params: FunctionCallParams):
+ await params.result_callback({"conditions": "nice", "temperature": "75"})
+
+
+async def fetch_restaurant_recommendation(params: FunctionCallParams):
+ await params.result_callback({"name": "The Golden Dragon"})
+
+
+# We store functions so objects (e.g. SileroVADAnalyzer) don't get
+# instantiated. The function will be called when the desired transport gets
+# selected.
+transport_params = {
+ "daily": lambda: DailyParams(
+ audio_in_enabled=True,
+ audio_out_enabled=True,
+ vad_analyzer=SileroVADAnalyzer(),
+ ),
+ "twilio": lambda: FastAPIWebsocketParams(
+ audio_in_enabled=True,
+ audio_out_enabled=True,
+ vad_analyzer=SileroVADAnalyzer(),
+ ),
+}
+
+
+async def run_bot(transport: BaseTransport, runner_args: RunnerArguments):
+ stt = DeepgramSTTService(api_key=os.getenv("DEEPGRAM_API_KEY"))
+
+ tts = CartesiaTTSService(
+ api_key=os.getenv("CARTESIA_API_KEY"),
+ voice_id="79a125e8-cd45-4c13-8a67-188112f4dd22", # British Lady
+ )
+
+ tools = ToolsSchema(
+ [
+ FunctionSchema(
+ name="get_current_weather",
+ description="Get the current weather in a given location",
+ properties={
+ "location": {
+ "type": "string",
+ "description": "The city and state, e.g. San Francisco, CA",
+ },
+ "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]},
+ },
+ required=["location"],
+ ),
+ FunctionSchema(
+ name="get_restaurant_recommendation",
+ description="Get a restaurant recommendation",
+ properties={
+ "cuisine": {
+ "type": "string",
+ "description": "The type of cuisine, e.g. Italian, Chinese, Mexican",
+ },
+ "location": {
+ "type": "string",
+ "description": "The city or area to search in",
+ },
+ },
+ required=[],
+ ),
+ ]
+ )
+
+ llm = NebiusLLMService(
+ api_key=os.getenv("NEBIUS_API_KEY"),
+ model="meta-llama/Meta-Llama-3.1-8B-Instruct-fast",
+ )
+
+ # You can also register a function_name of None to get all functions
+ # sent to the same callback with an additional function_name parameter.
+ llm.register_function("get_current_weather", fetch_weather_from_api)
+ llm.register_function("get_restaurant_recommendation", fetch_restaurant_recommendation)
+
+ @llm.event_handler("on_function_calls_started")
+ async def on_function_calls_started(service, function_calls):
+ await tts.queue_frame(TTSSpeakFrame("Let me check on that."))
+
+ messages = [
+ {
+ "role": "system",
+ "content": "You are a helpful LLM in a WebRTC call. Your goal is to demonstrate your capabilities in a succinct way. Your output will be converted to audio so don't include special characters in your answers. Respond to what the user said in a creative and helpful way.",
+ },
+ ]
+
+ context = OpenAILLMContext(messages, tools)
+ context_aggregator = llm.create_context_aggregator(context)
+
+ pipeline = Pipeline(
+ [
+ transport.input(), # Transport user input
+ stt, # Speech-to-text
+ context_aggregator.user(), # User responses
+ llm, # LLM
+ tts, # Text-to-speech
+ transport.output(), # Transport bot output
+ context_aggregator.assistant(), # Assistant spoken responses
+ ]
+ )
+
+ task = PipelineTask(
+ pipeline,
+ PipelineParams(
+ allow_interruptions=True,
+ enable_metrics=True,
+ enable_usage_metrics=True,
+ ),
+ )
+
+ @transport.event_handler("on_client_connected")
+ async def on_client_connected(transport, client):
+ logger.info(f"Client connected")
+ # Kick off the conversation.
+ await task.queue_frames([context_aggregator.user().get_context_frame()])
+
+ @transport.event_handler("on_client_disconnected")
+ async def on_client_disconnected(transport, client):
+ logger.info(f"Client disconnected")
+ await task.cancel()
+
+ runner = PipelineRunner(handle_sigint=runner_args.handle_sigint)
+
+ await runner.run(task)
+
+
+async def bot(runner_args: RunnerArguments):
+ """Main bot entry point compatible with Pipecat Cloud."""
+ transport = await create_transport(runner_args, transport_params)
+ await run_bot(transport, runner_args)
+
+
+if __name__ == "__main__":
+ from pipecat.runner.run import main
+
+ main()
diff --git a/pyproject.toml b/pyproject.toml
index ce1233e049..a3e36bb46f 100644
--- a/pyproject.toml
+++ b/pyproject.toml
@@ -80,6 +80,7 @@ mistral = []
mlx-whisper = [ "mlx-whisper~=0.4.2" ]
moondream = [ "accelerate~=1.10.0", "einops~=0.8.0", "pyvips[binary]~=3.0.0", "timm~=1.0.13", "transformers>=4.48.0" ]
nim = []
+nebius = []
neuphonic = [ "pipecat-ai[websockets-base]" ]
noisereduce = [ "noisereduce~=3.0.3" ]
openai = [ "pipecat-ai[websockets-base]" ]
diff --git a/src/pipecat/services/nebius/__init__.py b/src/pipecat/services/nebius/__init__.py
new file mode 100644
index 0000000000..eabaf76adf
--- /dev/null
+++ b/src/pipecat/services/nebius/__init__.py
@@ -0,0 +1,13 @@
+#
+# Copyright (c) 2024–2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+import sys
+
+from pipecat.services import DeprecatedModuleProxy
+
+from .llm import *
+
+sys.modules[__name__] = DeprecatedModuleProxy(globals(), "nebius", "nebius.llm")
diff --git a/src/pipecat/services/nebius/llm.py b/src/pipecat/services/nebius/llm.py
new file mode 100644
index 0000000000..5a27196748
--- /dev/null
+++ b/src/pipecat/services/nebius/llm.py
@@ -0,0 +1,51 @@
+#
+# Copyright (c) 2024–2025, Daily
+#
+# SPDX-License-Identifier: BSD 2-Clause License
+#
+
+"""Nebius AI Studio LLM service implementation using OpenAI-compatible interface."""
+
+from loguru import logger
+
+from pipecat.services.openai.llm import OpenAILLMService
+
+
+class NebiusLLMService(OpenAILLMService):
+ """A service for interacting with Nebius AI Studio's API using the OpenAI-compatible interface.
+
+ This service extends OpenAILLMService to connect to Nebius AI Studio's API endpoint while
+ maintaining full compatibility with OpenAI's interface and functionality.
+ """
+
+ def __init__(
+ self,
+ *,
+ api_key: str,
+ base_url: str = "https://api.studio.nebius.ai/v1",
+ model: str = "meta-llama/Meta-Llama-3.1-8B-Instruct-fast",
+ **kwargs,
+ ):
+ """Initialize Nebius AI Studio LLM service.
+
+ Args:
+ api_key: The API key for accessing Nebius AI Studio's API.
+ base_url: The base URL for Nebius AI Studio API. Defaults to "https://api.studio.nebius.ai/v1".
+ model: The model identifier to use.
+ **kwargs: Additional keyword arguments passed to OpenAILLMService.
+ """
+ super().__init__(api_key=api_key, base_url=base_url, model=model, **kwargs)
+
+ def create_client(self, api_key=None, base_url=None, **kwargs):
+ """Create OpenAI-compatible client for Nebius AI Studio API endpoint.
+
+ Args:
+ api_key: The API key to use for the client. If None, uses instance api_key.
+ base_url: The base URL for the API. If None, uses instance base_url.
+ **kwargs: Additional keyword arguments passed to the parent create_client method.
+
+ Returns:
+ An OpenAI-compatible client configured for Nebius AI Studio's API.
+ """
+ logger.debug(f"Creating Nebius AI Studio client with api {base_url}")
+ return super().create_client(api_key, base_url, **kwargs)
- JavaScript
- 
- React
- 
- Swift
- 
- Kotlin
- 
- C++
-