Draft
Conversation
…istening after shutting down api
… S3 object storage functionality. Update BackendRequestModel to use 'compress' instead of 'zlib' for serialization. Refactor model deployment to support compression during save operations. Remove unused sandbox module and streamline protected object handling.
… Processor classes This commit introduces a new configuration module for the queue system, centralizing environment variable management for the Dispatcher and Processor components. The QueueConfig class provides type-safe access to configuration values with defaults and validation. Additionally, the Dispatcher and Processor classes have been refactored to utilize the new configuration module, improving code clarity and maintainability. The changes include enhanced docstrings for better understanding of class functionalities and attributes, as well as adjustments to how environment variables are accessed throughout the codebase.
…ketIO connection handling This commit introduces a new configuration module, AppConfig, which centralizes environment variable management for the API service. The app.py file has been refactored to utilize this configuration for Redis connection, SocketIO settings, and request handling parameters. Additionally, the SocketIO connection handling has been improved with enhanced docstrings for better clarity on function purposes and parameters. This refactor aims to improve code maintainability and readability.
…dundant NNSIGHT_ERROR status handling. This change enhances code clarity and maintains consistent logging practices.
This commit introduces a new endpoint in the API service to fetch Python environment details, including the Python version and installed packages. It implements caching for efficiency and handles timeouts and errors gracefully. Additionally, the Dispatcher class has been updated to manage ENV events, ensuring the environment cache is cleared when necessary. The Controller class now includes a method to gather environment information from the Ray cluster, enhancing the overall functionality and maintainability of the service.
… it consitent with the intuitive understaing of the logs
…econnecting is slow. Log were RUNNING BEFORE deserializeing so its part of the running time calculation
…andling. The traceback formatting has been removed, and only the exception message is now included in the response description, enhancing clarity and consistency in error reporting.
Nnsight v0.5.16
Protections are set via a yml. Pytests for basec nnsihgt usage and security
…so you cant do this
| exit 1 | ||
| fi | ||
|
|
||
| echo "Starting Ray head node..." |
Need to set manual cuda devices and ray env fvar to prevent ray from blovking gpu access on the deployment
Member
Issue:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.