Entity Identificationn

🚀 Entity Recognition

This project aims to identify whether two sets of data come from the same entity. It is a server that supports the MCP (Model Context Protocol).

🚀 Quick Start

This tool offers a comprehensive approach to comparing two sets of data, evaluating both the exact equality and semantic equivalence of their values. It leverages text normalization and language models to determine whether the data originates from the same entity.

✨ Features

• Text Normalization: Converts text to lowercase, removes punctuation, and standardizes whitespace.
• Value Comparison: Performs exact and semantic comparisons on two values (ignoring list order).
• JSON Traversal: Iterates through JSON objects key by key and compares corresponding values.
• Language Model Integration: Utilizes generative language models to assess semantic similarity and ultimately determine whether the data comes from the same entity.

📦 Installation

To use this tool, ensure that the necessary dependencies are installed. You can install them using pip:

pip install genai

💻 Usage Examples

Basic Usage

Functions

1. normalize_text(text):

   • Normalizes the input text, including converting to lowercase, removing punctuation, and standardizing whitespace.

2. compare_values(val1, val2):

   • Performs exact and semantic comparisons on two values.
   • If the values are lists, it performs semantic comparison ignoring the order.

3. compare_json(json1, json2):

   • Compares each key of two JSON objects.
   • Uses compare_values to evaluate the values of each key.
   • Integrates a language model to assess semantic similarity and make a final determination.

Example

import json
import genai
import re

# Define your JSON objects
json1 = {
    "name": "John Doe",
    "address": "123 Main St, Anytown, USA",
    "hobbies": ["reading", "hiking", "coding"]
}

json2 = {
    "name": "john doe",
    "address": "123 Main Street, Anytown, USA",
    "hobbies": ["coding", "hiking", "reading"]
}

# Compare JSON objects
comparison_results = compare_json(json1, json2)

# Generate the final matching result
model1 = genai.GenerativeModel("gemini-2.0-flash-thinking-exp")
result_matching = model1.generate_content("综合这些信息，你认为可以判断两个数据来自同一主体吗？"+json.dumps(comparison_results, ensure_ascii=False, indent=4))
print(result_matching.text)

🤝 Contribution

Contributions are welcome! Please open an issue or submit a pull request.

📄 License

This project is released under the MIT License. For more details, please refer to the LICENSE file.

📞 Contact

If you have any questions or suggestions, please contact me:

• Email: u3588064@connect.hku.hk
• GitHub: u3588064@connect.hku.hk

WeChat