Entityidentification

🚀 Entity Recognition

This project aims to identify whether two sets of data originate from the same entity. It serves as a server supporting the MCP (Model Context Protocol).

📦 Data Comparison Tool

This tool offers a comprehensive approach to compare two sets of data, evaluating both the exact and semantic equality of their values. It leverages text normalization and language models to determine if the data comes from the same entity.

✨ Features

• Text Normalization: Converts text to lowercase, removes punctuation, and standardizes whitespace.
• Value Comparison: Performs exact and semantic comparisons between two values (ignoring order for lists).
• JSON Traversal: Iterates through each key in a JSON object and compares the corresponding values.
• Language Model Integration: Utilizes generative language models to assess semantic similarity and make a final determination on whether the data comes from the same entity.

📦 Installation

To use this tool, ensure that the necessary dependencies are installed. You can install them using pip:

pip install genai

💻 Usage Examples

Functions

1. normalize_text(text):
   • Normalizes the input text by converting it to lowercase, removing punctuation, and standardizing whitespace.
2. compare_values(val1, val2):
   • Performs exact and semantic comparisons between two values.
   • Ignores the order of elements when comparing lists semantically.
3. compare_json(json1, json2):
   • Compares the keys of two JSON objects.
   • Uses compare_values to evaluate the values of each key.
   • Integrates language model evaluation of semantic similarity and makes a final determination on whether the data comes from the same entity.

Basic Usage

import json
import genai
import re

# Define your JSON objects
json1 = {
    "name": "John Doe",
    "address": "123 Main St, Anytown, USA",
    "hobbies": ["reading", "hiking", "coding"]
}

json2 = {
    "name": "john doe",
    "address": "123 Main Street, Anytown, USA",
    "hobbies": ["coding", "hiking", "reading"]
}

# Compare JSON objects
comparison_results = compare_json(json1, json2)

# Generate the final matching result
model1 = genai.GenerativeModel("gemini-2.0-flash-thinking-exp")
result_matching = model1.generate_content("综合这些信息，你认为可以判断两个数据来自同一主体吗？"+json.dumps(comparison_results, ensure_ascii=False, indent=4))
print(result_matching.text)

👥 Contribution

Contributions are welcome! Please open an issue or submit a pull request.

📄 License

This project is licensed under the MIT License. For more details, please refer to the LICENSE file.

📞 Contact

If you have any questions or suggestions, please contact us:

• Email: u3588064@connect.hku.hk
• GitHub: u3588064@connect.hku.hk.

WeChat