Fm MCP Comfyui Bridge

🚀 Image Generation Tool with Analysis Function

This project is an image generation tool built using ComfyUI and the MCP framework. It can generate corresponding images by inputting specific prompts and provides analysis functions for the generated images.

🚀 Quick Start

1. Initialize the Project

git clone https://github.com/username/repo.git
cd repo
pip install -r requirements.txt

2. Configure Environment Variables

Add the necessary configurations to the config.py file in the project:

# config.py
COMFYUI_API_KEY = "your_api_key_here"
HUGGINGFACE_TOKEN = "your_huggingface_token_here"
OLLAMA_BINARY_PATH = "/path/to/ollama"

3. Start the Service

Run the main script to start the MCP server:

python main.py

✨ Features

1. Image Generation

   • Support generating high-quality images through English prompts (especially Danbooru tags).
   • The generated image links can be directly displayed in Markdown using the ![URL](URL) format.

2. Image Analysis

   • Provide the function to obtain image captions to help users understand the image content.
   • Extract and display the 14chan (WD14) tag information of the image, facilitating the analysis of the image's theme and elements.

3. Automatic Prompt Optimization

   • The system will iteratively optimize the prompts multiple times based on the matching degree between the generated results and the original prompts, and finally generate the image that best meets the user's intention.
   • It can perform up to 8 generations and optimizations to ensure the output quality.

📚 Documentation

Example Workflow

1. The user inputs a prompt in Japanese or Chinese.
2. The system automatically translates it into English and appends the necessary Danbooru tags (e.g., "masterpiece, best quality, newest,").
3. Call ComfyUI to generate an image and obtain the returned image link.
4. Use the get_caption and get_tag tools to analyze the image content.
5. Compare the original prompt with the generated result. If they don't match, optimize the prompt and repeat the generation process.

Project Structure

Core Components

• ComfyUI: A powerful image generation interface that supports multiple models and custom nodes.
• MCP (Magic Command Parser): A framework for parsing and executing specific commands, which is the core operating mechanism of this tool.

Main Interfaces

1. Image Generation

@mcp.tool("image://generate")
def generate_image(
    prompt: str,
    negative_prompt: Optional[str] = None,
    width: int = DEFAULT_WIDTH,
    height: int = DEFAULT_HEIGHT,
    samples: int = DEFAULT_SAMPLES,
    num_inference_steps: int = DEFAULT_STEPS,
) -> str:
    """Image generation interface"""
    return image_url

2. Image Analysis

@mcp.tool("image://analyze")
def analyze_image(
    url: str,
) -> Tuple[str, List[str]]:
    """Get the captions and WD14 tags of the image"""
    pass

3. Prompt Optimization

@mcp.resource("optimizer://prompt")
def optimize_prompt(prompt: str) -> str:
    """Optimize the new prompt based on the previous generation results"""
    pass

Project Dependencies

• mcp[cli] >=1.6.0: MCP server framework
• requests >=2.32.3: HTTP request handling
• huggingface-hub >=0.25.2: Hugging Face model access
• numpy >=2.1.2: Numerical computation library
• ollama >=0.3.3: Local LLM support
• onnxruntime >=1.19.2: ONNX model runtime environment
• pandas >=2.2.3: Data analysis library

📄 License

This project is licensed under the MIT License. For detailed information, please refer to the LICENSE file.

👨‍💻 Author

• rerofumi: GitHub - rero2@yuumu.org

The WD1.4 tag parsing part uses the source code and model data of wd-tagger developed by SmilingWolf. The model data will be automatically downloaded during the first run.