-
Notifications
You must be signed in to change notification settings - Fork 558
Add ApptainerWorkspace implementation for rootless container support #892
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
31 commits
Select commit
Hold shift + click to select a range
c7926f0
Add ApptainerWorkspace implementation for rootless container support
openhands-agent 98939bf
Update documentation to clarify testing limitations
openhands-agent 040a4b5
Fix ApptainerWorkspace to use native apptainer pull instead of Docker
openhands-agent e462117
Add implementation summary document
openhands-agent d22c27f
Fix ApptainerWorkspace exec mode and RemoteWorkspace authentication
openhands-agent 8b43a7e
Remove unnecessary documentation and test files
openhands-agent c727af7
Merge main branch into openhands/apptainer-workspace-891
openhands-agent b86ffc3
Add openhands-agent-server dependency to workspace package
openhands-agent 157fdeb
Merge branch 'main' into openhands/apptainer-workspace-891
neubig 487ab1e
Merge main branch and resolve conflicts in __init__.py
openhands-agent 2ee7a69
Fix type errors in apptainer example: remove visualize and agent_status
openhands-agent add65b5
Trigger CI re-run
openhands-agent fe579c9
Merge branch 'main' into openhands/apptainer-workspace-891
neubig 4c82a41
Add Apptainer CI workflow with setup-apptainer action
openhands-agent 4907d4e
Enhance ApptainerWorkspace tests to match DockerWorkspace test coverage
openhands-agent 4dd67b1
Fix apptainer example to match docker example style
openhands-agent d002fd3
Remove demo log file from PR
openhands-agent a3773ab
Merge main branch and resolve conflicts in __init__.py
openhands-agent 86e838e
Fix ApptainerWorkspace tests and __del__ cleanup
neubig fc24f67
Merge branch 'main' into openhands/apptainer-workspace-891
neubig 0fbb279
Fix ApptainerWorkspace environment leakage and file ownership issues
openhands-agent e6e5c59
Merge branch 'main' into openhands/apptainer-workspace-891
openhands-agent 2eff934
Fix ApptainerWorkspace tests to mock port availability check
openhands-agent 0b9c129
Merge branch 'main' into openhands/apptainer-workspace-891
neubig 285b3f9
Openhands/apptainer workspace 891 fixes (#1492)
adityasoni9998 e0417a9
Merge branch 'main' into openhands/apptainer-workspace-891
xingyaoww a859ce2
Address PR review comments from xingyaoww
openhands-agent 818885f
Merge branch 'main' into openhands/apptainer-workspace-891
xingyaoww 812df71
Simplify ApptainerWorkspace: remove base_image support
openhands-agent 3d604d2
Update ApptainerWorkspace README to clarify pre-built images only
neubig baf76c9
Fix validation order in ApptainerWorkspace
neubig File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
109 changes: 109 additions & 0 deletions
109
examples/02_remote_agent_server/07_convo_with_apptainer_sandboxed_server.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,109 @@ | ||
| import os | ||
| import platform | ||
| import time | ||
|
|
||
| from pydantic import SecretStr | ||
|
|
||
| from openhands.sdk import ( | ||
| LLM, | ||
| Conversation, | ||
| RemoteConversation, | ||
| get_logger, | ||
| ) | ||
| from openhands.tools.preset.default import get_default_agent | ||
| from openhands.workspace import ApptainerWorkspace | ||
|
|
||
|
|
||
| logger = get_logger(__name__) | ||
|
|
||
| # 1) Ensure we have LLM API key | ||
| api_key = os.getenv("LLM_API_KEY") | ||
| assert api_key is not None, "LLM_API_KEY environment variable is not set." | ||
|
|
||
| llm = LLM( | ||
| usage_id="agent", | ||
| model=os.getenv("LLM_MODEL", "anthropic/claude-sonnet-4-5-20250929"), | ||
| base_url=os.getenv("LLM_BASE_URL"), | ||
| api_key=SecretStr(api_key), | ||
| ) | ||
|
|
||
|
|
||
| def detect_platform(): | ||
| """Detects the correct platform string.""" | ||
| machine = platform.machine().lower() | ||
| if "arm" in machine or "aarch64" in machine: | ||
| return "linux/arm64" | ||
| return "linux/amd64" | ||
|
|
||
|
|
||
| # 2) Create an Apptainer-based remote workspace that will set up and manage | ||
| # the Apptainer container automatically. Use `ApptainerWorkspace` with a | ||
| # pre-built agent server image. | ||
| # Apptainer (formerly Singularity) doesn't require root access, making it | ||
| # ideal for HPC and shared computing environments. | ||
| with ApptainerWorkspace( | ||
| # use pre-built image for faster startup | ||
| server_image="ghcr.io/openhands/agent-server:latest-python", | ||
| host_port=8010, | ||
| platform=detect_platform(), | ||
| ) as workspace: | ||
| # 3) Create agent | ||
| agent = get_default_agent( | ||
| llm=llm, | ||
| cli_mode=True, | ||
| ) | ||
|
|
||
| # 4) Set up callback collection | ||
| received_events: list = [] | ||
| last_event_time = {"ts": time.time()} | ||
|
|
||
| def event_callback(event) -> None: | ||
| event_type = type(event).__name__ | ||
| logger.info(f"🔔 Callback received event: {event_type}\n{event}") | ||
| received_events.append(event) | ||
| last_event_time["ts"] = time.time() | ||
|
|
||
| # 5) Test the workspace with a simple command | ||
| result = workspace.execute_command( | ||
| "echo 'Hello from sandboxed environment!' && pwd" | ||
| ) | ||
| logger.info( | ||
| f"Command '{result.command}' completed with exit code {result.exit_code}" | ||
| ) | ||
| logger.info(f"Output: {result.stdout}") | ||
| conversation = Conversation( | ||
| agent=agent, | ||
| workspace=workspace, | ||
| callbacks=[event_callback], | ||
| ) | ||
| assert isinstance(conversation, RemoteConversation) | ||
|
|
||
| try: | ||
| logger.info(f"\n📋 Conversation ID: {conversation.state.id}") | ||
|
|
||
| logger.info("📝 Sending first message...") | ||
| conversation.send_message( | ||
| "Read the current repo and write 3 facts about the project into FACTS.txt." | ||
| ) | ||
| logger.info("🚀 Running conversation...") | ||
| conversation.run() | ||
| logger.info("✅ First task completed!") | ||
| logger.info(f"Agent status: {conversation.state.execution_status}") | ||
|
|
||
| # Wait for events to settle (no events for 2 seconds) | ||
| logger.info("⏳ Waiting for events to stop...") | ||
| while time.time() - last_event_time["ts"] < 2.0: | ||
| time.sleep(0.1) | ||
| logger.info("✅ Events have stopped") | ||
|
|
||
| logger.info("🚀 Running conversation again...") | ||
| conversation.send_message("Great! Now delete that file.") | ||
| conversation.run() | ||
| logger.info("✅ Second task completed!") | ||
|
|
||
| # Report cost (must be before conversation.close()) | ||
| cost = conversation.conversation_stats.get_combined_metrics().accumulated_cost | ||
| print(f"EXAMPLE_COST: {cost}") | ||
| finally: | ||
| print("\n🧹 Cleaning up conversation...") | ||
| conversation.close() |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
159 changes: 159 additions & 0 deletions
159
openhands-workspace/openhands/workspace/apptainer/README.md
|
neubig marked this conversation as resolved.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,159 @@ | ||
| # Apptainer Workspace | ||
|
|
||
| The `ApptainerWorkspace` provides a container-based workspace using [Apptainer](https://apptainer.org/) (formerly Singularity), which doesn't require root access. This makes it ideal for HPC and shared computing environments where Docker may not be available or permitted. | ||
|
|
||
| Note: This class only works with **pre-built images**. It does not support building images on-the-fly from a base image. For on-the-fly building with Docker, use `DockerDevWorkspace` instead. | ||
|
|
||
| ## Why Apptainer? | ||
|
|
||
| - **No root required**: Unlike Docker, Apptainer doesn't need root/sudo privileges | ||
| - **HPC-friendly**: Designed for high-performance computing environments | ||
| - **Secure**: Better security model for multi-user systems | ||
| - **Compatible**: Can use pre-built Docker images | ||
|
|
||
| ## Prerequisites | ||
|
|
||
| Install Apptainer by following the [official quick start guide](https://apptainer.org/docs/user/main/quick_start.html). | ||
|
|
||
| On Ubuntu/Debian: | ||
| ```bash | ||
| sudo apt-get update | ||
| sudo apt-get install -y apptainer | ||
| ``` | ||
|
|
||
| On CentOS/RHEL: | ||
| ```bash | ||
| sudo yum install -y apptainer | ||
| ``` | ||
|
|
||
| ## Usage | ||
|
|
||
| ### Option 1: Use Pre-built Agent Server Image (Recommended) | ||
|
|
||
| ```python | ||
| from openhands.workspace import ApptainerWorkspace | ||
|
|
||
| # Use a pre-built agent server image | ||
| with ApptainerWorkspace( | ||
| server_image="ghcr.io/openhands/agent-server:latest-python", | ||
| host_port=8010, | ||
| ) as workspace: | ||
| result = workspace.execute_command("echo 'Hello from Apptainer!'") | ||
| print(result.stdout) | ||
| ``` | ||
|
|
||
| ### Option 2: Use Existing SIF File | ||
|
|
||
| ```python | ||
| from openhands.workspace import ApptainerWorkspace | ||
|
|
||
| # Use an existing Apptainer SIF file | ||
| with ApptainerWorkspace( | ||
| sif_file="/path/to/your/agent-server.sif", | ||
| host_port=8010, | ||
| ) as workspace: | ||
| result = workspace.execute_command("ls -la") | ||
| print(result.stdout) | ||
| ``` | ||
|
|
||
| ### Mount Host Directory | ||
|
|
||
| ```python | ||
| from openhands.workspace import ApptainerWorkspace | ||
|
|
||
| # Mount a host directory into the container | ||
| with ApptainerWorkspace( | ||
| server_image="ghcr.io/openhands/agent-server:latest-python", | ||
| host_port=8010, | ||
| mount_dir="/path/to/host/directory", | ||
| ) as workspace: | ||
| result = workspace.execute_command("ls /workspace") | ||
| print(result.stdout) | ||
| ``` | ||
|
|
||
| ## Configuration Options | ||
|
|
||
| | Parameter | Type | Default | Description | | ||
| |-----------|------|---------|-------------| | ||
| | `server_image` | `str \| None` | `None` | Pre-built agent server image (mutually exclusive with `sif_file`) | | ||
| | `sif_file` | `str \| None` | `None` | Path to existing SIF file (mutually exclusive with `server_image`) | | ||
| | `host_port` | `int \| None` | `None` | Port to bind to (auto-assigned if None) | | ||
| | `mount_dir` | `str \| None` | `None` | Host directory to mount into container | | ||
| | `cache_dir` | `str \| None` | `~/.apptainer_cache` | Directory for caching SIF files | | ||
| | `forward_env` | `list[str]` | `["DEBUG"]` | Environment variables to forward | | ||
| | `detach_logs` | `bool` | `True` | Stream logs in background | | ||
| | `platform` | `PlatformType` | `"linux/amd64"` | Platform architecture | | ||
| | `extra_ports` | `bool` | `False` | Expose additional ports (VSCode, VNC) | | ||
| | `use_fakeroot` | `bool` | `True` | Use --fakeroot for consistent file ownership | | ||
|
|
||
| ## How It Works | ||
|
|
||
| 1. **Image Preparation**: Pulls Docker images and converts to Apptainer SIF format, or uses existing SIF files | ||
| 2. **Caching**: SIF files are cached in `~/.apptainer_cache` by default for faster startup | ||
| 3. **Container Execution**: Runs the agent server using `apptainer run` | ||
| 4. **Health Checking**: Waits for the server to become healthy before accepting requests | ||
| 5. **Cleanup**: Automatically stops the container when done | ||
|
|
||
| ## Differences from DockerWorkspace | ||
|
|
||
| | Feature | DockerWorkspace | ApptainerWorkspace | | ||
| |---------|----------------|-------------------| | ||
| | Root required | Yes (typically) | No | | ||
| | Docker daemon | Required | Not required | | ||
| | Port mapping | Native | Host networking | | ||
| | Image format | Docker | SIF (from Docker) | | ||
| | HPC support | Limited | Excellent | | ||
| | Setup complexity | Lower | Slightly higher | | ||
|
|
||
| ## Troubleshooting | ||
|
|
||
| ### Apptainer not found | ||
| ``` | ||
| RuntimeError: Apptainer is not available | ||
| ``` | ||
| **Solution**: Install Apptainer following the [installation guide](https://apptainer.org/docs/user/main/quick_start.html). | ||
|
|
||
| ### Port already in use | ||
| ``` | ||
| RuntimeError: Port 8010 is not available | ||
| ``` | ||
| **Solution**: Either specify a different `host_port` or let the system auto-assign one by not specifying it. | ||
|
|
||
| ### Image pull fails | ||
| ``` | ||
| Failed to pull and convert Docker image | ||
| ``` | ||
| **Solution**: Ensure you have network access to pull images from the Docker registry. Apptainer pulls directly from Docker registries without needing Docker daemon. | ||
|
|
||
| ## Complete Example | ||
|
|
||
| See `examples/02_remote_agent_server/07_convo_with_apptainer_sandboxed_server.py` for a complete working example that demonstrates: | ||
| - Setting up an Apptainer workspace | ||
| - Running agent conversations | ||
| - File operations in the sandboxed environment | ||
| - Proper cleanup | ||
|
|
||
| **To test the example:** | ||
| ```bash | ||
| # Make sure Apptainer is installed | ||
| apptainer --version | ||
|
|
||
| # Run the example | ||
| cd examples/02_remote_agent_server | ||
| python 07_convo_with_apptainer_sandboxed_server.py | ||
| ``` | ||
|
|
||
| ## Performance Notes | ||
|
|
||
| - **First run**: Slower due to image download and SIF conversion | ||
| - **Subsequent runs**: Much faster if the SIF file is cached | ||
| - **Best for**: Long-running workloads, HPC environments, multi-user systems | ||
| - **Cache location**: Check and clean `~/.apptainer_cache` periodically | ||
|
|
||
| ## Security | ||
|
|
||
| Apptainer provides better security isolation for shared systems: | ||
| - Runs as the invoking user (no privilege escalation) | ||
| - No daemon running as root | ||
| - Designed for multi-tenant HPC environments | ||
| - Support for encrypted containers (optional) |
6 changes: 6 additions & 0 deletions
6
openhands-workspace/openhands/workspace/apptainer/__init__.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| """Apptainer workspace implementation.""" | ||
|
|
||
| from .workspace import ApptainerWorkspace | ||
|
|
||
|
|
||
| __all__ = ["ApptainerWorkspace"] |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.