Workflowlearner

🚀 WorkflowLearner

This project aims to enable Large Language Models (LLMs) to learn workflows through MHT files recorded by PSR.exe. PSR.exe is the Problem Steps Recorder in the Windows operating system, which can record user operations and generate MHT files. By parsing these MHT files, LLMs can learn and understand user workflows.

🚀 Quick Start

This project allows LLMs to learn workflows by leveraging MHT files recorded via PSR.exe. The following sections will guide you through the installation and usage.

✨ Features

• Record user operations using PSR.exe and generate MHT files.
• Parse MHT files and extract workflow information.
• Enable LLMs to learn and understand workflows.

📦 Installation

Requirements

• Windows operating system
• Python 3.x
• PSR.exe (Problem Steps Recorder, built - in Windows)

Install Steps

Clone the repository:

git clone https://github.com/u3588064/WorkflowLearner
cd llm-workflow-learning

Install dependencies:

pip install -r requirements.txt

💻 Usage Examples

Basic Usage

Recording actions with PSR.exe

1. Open PSR.exe (Problem Steps Recorder), which can be found via the search bar in Win7/Win8/Win10.
2. Click the "Start Recording" button and then start your work. PSR.exe will automatically record mouse actions (such as clicks, drags, scrolls, etc.) and keyboard actions (only actions, not the typed content).
3. After completing the operations, click the "Stop Recording" button. The software will automatically pop up a save window.
4. The result will be stored as a Zip archive. Inside the archive is a webpage in MHT format, containing step - by - step procedures and screenshots.

Parse MHT files

python parse_mht.py path/to/your/file.mht

Learn workflows using LLM

python learn_workflows.py path/to/parSED_DATA.csv

File Structure

• parse_mht.py: Parse MHT files and extract useful information.
• learn_workflows.py: Train or infer LLM models using the extracted information to learn workflows.
• requirements.txt: Python packages required by the project.

📚 Documentation

Online Demo

The project provides an online demo link: WorkflowLearner Online Demo

Contribution

Contributions are welcome! Please refer to the project documentation to learn how to participate in the development.

License

This project is licensed under the MIT License.

Contact

If you have any questions or need further information, please contact: u3588064@connect.hku.hk