Ten Agent

TEN Agent is a multi-functional AI agent framework that integrates real-time vision, voice recognition, and screen sharing detection capabilities, and supports rapid expansion and development.

Artificial intelligence chatbots Education and learning tools #AI agent #Real-time vision #Voice interaction #Multi-modal .Python

rating : 2 points

downloads : 8

update time : 2025-04-23

What is TEN Agent?

TEN Agent is an open-source AI agent framework that integrates various AI capabilities through a modular design. It supports real-time audio and video processing, multi-modal interaction, and an extensible plugin system, allowing developers to quickly build intelligent applications.

How to use TEN Agent?

You can start the TEN Agent development environment through simple Docker deployment, and then configure different functional modules and extension plugins through the Web interface.

Applicable scenarios

TEN Agent is suitable for various AI interaction scenarios such as intelligent customer service, real-time translation, content creation assistance, and educational tools.

Main features

Multi-modal interactionSupports multiple input and output methods such as text, voice, and vision

Real-time processingBased on Agora RTC's low-latency audio and video processing capabilities

Plugin systemAn extensible plugin architecture that supports custom function development

Advantages and limitations

Advantages

Out-of-the-box multi-modal AI capabilities

Modular design for easy expansion

Supports integration of mainstream AI models

Limitations

Requires API key configuration

Some functions rely on third-party services

Local deployment requires certain hardware resources

How to use

Prepare the environment

Install Docker and Node.js environments

Configure keys

Set the API keys for services such as Agora and OpenAI

Start the service

Use Docker Compose to start the development environment

Usage examples

Real-time visual interaction with GeminiObtain real-time visual input through the camera and interact with the Gemini model

Weather queryQuery the weather conditions of a specified location through voice or text

Frequently Asked Questions

How to obtain Agora's API key?

Does it support local deployment of AI models?

Related resources

Official documentation

Complete development documentation and usage guides

GitHub repository

Source code and issue tracking

Discord community

Developer communication community

🚀 TEN Agent Component

TEN Agent is a real-time voice assistant platform integrating various AI technologies, supporting multimodal interaction and modular expansion.

🚀 Quick Start

✨ Features

Real-time Voice Processing: Achieve voice interaction based on Deepgram's ASR and FishAudio's TTS.
AI Inference Capability: Integrate the OpenAI API for intelligent conversations.
Real-time Audio and Video: Utilize Agora.io to build real-time communication capabilities.
Extension Plugin Support: Modular design for easy function expansion.

Project Architecture

Component Diagram

📦 Installation

System Requirements

Processor: Dual-core or higher
Memory: 4GB or more
Required Tools:
- Docker container runtime environment
- Node.js LTS version 18.x

Notes for macOS Configuration

For Macs with Apple chips, uncheck "Use Rosetta for x86/AMD64 emulation" in Docker settings. Note: This may slow down the build speed on ARM architectures, but the performance will be normal after deployment to an x64 server.

Docker Settings

Installation Steps

Create a .env file
```
cp ./.env.example ./.env
```
Configure Agora Keys Update the following variables in the .env file:
```
AGORA_APP_ID=
AGORA_APP_CERTIFICATE=
```
Start the Development Container
```
docker compose up -d
```
Enter the Container Environment
```
docker exec -it ten_agent_dev bash
```
Build the Agent Service
```
task use
```
Run the Web Interface
```
task run
```
Configure the Playground Open the address: localhost:3000 and select the required modules and plugins.

Module Example

💻 Usage Examples

Basic Usage

Gemini Real-time Plugin

Open the Playground interface.
Select the "voice_assistant_realtime" diagram.
Select the Gemini real-time module.
Add the v2v extension and configure the API key.

Gemini Real-time Console

📚 Documentation

Project Components

Component Architecture Diagram

Join the Community

Discord: Link
GitHub Repository: Link

Contribution Guidelines

Code Style:
- Follow the ES6 standard.
- Use 2-space indentation.
Commit Guidelines:
- Commit messages should include the type and purpose.
- Provide relevant documentation support.