Add Customer Support AI Training Environment (OpenEnv-compatible)

[Copilot]

Bootstraps a complete OpenEnv-compatible RL environment simulating a customer support system, built from scratch on an empty repo.

Architecture

• environment.py — Core env with reset() / step() / state() interface. 6 deterministic scenarios (payment failure, delayed delivery, refund, account issues). Shaped reward function with partial credit, penalties, and bonuses. Escalation does not auto-terminate episodes — agent must call mark_resolved.
• tasks.py — Three Task dataclasses (easy / medium / hard) with expected action sequences, max steps, and pass thresholds.
• grader.py — Per-task deterministic graders returning [0.0, 1.0]; weighted final score (easy=0.2, medium=0.3, hard=0.5).
• inference.py — Rule-based deterministic baseline agent; runs all tasks and prints scored summary.
• openenv.yaml — Env metadata, action/task config, resource constraints.
• Dockerfile — python:3.10-slim container; CMD ["python", "inference.py"].

Interface

from environment import CustomerSupportEnv

env = CustomerSupportEnv(seed=42)
obs = env.reset(scenario_index=0)          # structured observation
obs, reward, done, info = env.step("classify_issue")
state = env.state()                        # full internal snapshot for grading

Reward shaping

Action	Condition	Reward
classify_issue	first use	+0.3
classify_issue	repeated	−0.1
respond_with_solution	classified + no escalation needed	+0.5
respond_with_solution	before classify	−0.2
escalate_to_human	required by scenario	+0.5 (+0.1 premium bonus)
escalate_to_human	unnecessary	−0.3
mark_resolved	after proper handling	+0.3
mark_resolved	premature	−0.1 to −0.2

Baseline scores (rule-based agent, seed=42)

Issue Classification  [easy]    1.00
Action Selection      [medium]  0.90
End-to-End Conv.      [hard]    0.90
─────────────────────────────────────
Weighted Average                0.92

Original prompt

You are an expert Python developer and reinforcement learning environment designer. I want you to build a complete OpenEnv-compatible project called "Customer Support AI Training Environment". This project should simulate a real-world customer support system where an AI agent interacts with incoming customer queries and learns to respond appropriately through step-by-step actions. The environment must strictly follow the OpenEnv interface pattern using reset(), step(action), and state() functions, and should be implemented in clean, modular Python code.

The environment should simulate realistic customer support scenarios. Each episode begins with a customer query such as payment failure, delayed delivery, refund request, or account issue. The observation returned to the agent should include structured information such as the customer message, sentiment (e.g., angry, neutral, happy), customer type (premium or regular), and conversation history. The agent must take actions such as "classify_issue", "respond_with_solution", "ask_for_more_info", "escalate_to_human", or "mark_resolved". The environment should update state accordingly after each action.

You must design three tasks with increasing difficulty levels. The easy task should involve simple classification of the issue type. The medium task should require choosing the correct action based on the situation. The hard task should simulate a multi-step conversation where the agent must handle the issue end-to-end, including proper response, escalation decisions, and resolution. Each task must have a deterministic grading function that returns a score between 0.0 and 1.0 based on correctness and quality.

The reward function must be well-designed and provide meaningful signals throughout the interaction. Instead of binary rewards, include partial rewards for partially correct actions, penalties for incorrect or harmful actions, and bonuses for optimal decisions. For example, correctly identifying the issue gives partial reward, choosing the correct action adds more reward, and successfully resolving the issue gives full reward. Penalize rude or irrelevant responses, unnecessary escalations, or repeated incorrect actions.

Create a baseline inference script named inference.py that initializes the environment, runs through all defined tasks, and uses a simple rule-based or random policy to interact with the environment. The script should print scores for each task and a final average score. Ensure that this script is deterministic and reproducible.

Organize the project into multiple files: environment.py (core environment logic), tasks.py (task definitions), grader.py (grading and scoring logic), inference.py (baseline agent runner), openenv.yaml (metadata and configuration), requirements.txt (dependencies), Dockerfile (for containerization), and README.md (documentation). Use clean code practices, type hints where appropriate, and simple class-based structure.

Ensure the project is lightweight and can run within limited resources (2 vCPU, 8GB RAM) and completes execution in under 20 minutes. Avoid unnecessary complexity or heavy dependencies. The implementation should be beginner-friendly but logically strong enough to demonstrate real-world utility.

Finally, include clear comments in the code explaining each component, and ensure the environment can later be deployed easily to Hugging Face Spaces. Do not include any UI or frontend code. Focus entirely on backend logic, environment simulation, and agent interaction.

Generate the full initial version of this project with working code across all files.

[Copilot]

Copilot wasn't able to review any files in this pull request.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.