Skip to content

feat(storage): implement chunked streaming for document uploads to prevent memory saturation #103

Closed as not planned

Description

@Diwakar-odds

Problem Statement

In document.routes.ts, multer.memoryStorage() buffers uploaded files entirely in RAM before processing. High concurrency of large file uploads (up to the 10MB limit) will rapidly exhaust container memory, causing application crashes.

Proposed Technical Approach

  • Migrate to multer({ dest: 'uploads/' }) to stream files to disk.
  • Alternatively, implement a streaming engine to pipe upload chunks directly to persistent storage with minimal RAM footprint.

Activity

  1. Diwakar-odds commented on Aug 8, 2026

    @Diwakar-odds
    Author

    I would like to request assignment for this issue. I have thoroughly analyzed the architecture and verified this is an L3 (Core/Architecture/Performance) task. I am ready to implement the changes once assigned.

  2. added
    backendBackend API, services, workers, or domain logic
    performanceMeasurable speed, latency, memory, or scalability work
    triageNeeds initial maintainer triage
    on Aug 8, 2026
  3. github-actions commented on Sep 8, 2026

    @github-actions

    This issue has been inactive for a while. It will close automatically if there is no new activity. Comment with updated context if it is still relevant.

  4. added
    staleInactive issue or pull request pending closure
    on Sep 8, 2026
  5. github-actions commented on Sep 22, 2026

    @github-actions

    Closing as stale. Please open a fresh issue with current reproduction details if this is still needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

ECSoC26-L1backendBackend API, services, workers, or domain logicperformanceMeasurable speed, latency, memory, or scalability workstaleInactive issue or pull request pending closuretriageNeeds initial maintainer triage

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions