Implement lazy-evaluated Polars engine and centralized Terraform orchestration

[BLMgithub]

Context
Migrates assembly/semantic stages to a high-performance Polars engine to resolve memory bottlenecks while centralizing GCP infrastructure via Terraform for state-managed integrity. This update bridges the gap between enterprise-scale data processing and production-grade infrastructure automation.

Implemented

• Engine Transition: Replace Pandas with Polars LazyFrame APIs to enable deferred execution and out-of-core streaming.
• I/O & State: Deploy hybrid read/write handlers for Pandas-Polars interoperability and overhaul logging for deterministic execution tracing.
• IaC Foundation: Migrate local .tfstate to versioned GCS buckets and establish ci-infra.yml to enforce infrastructure gates pre-deployment.
• Resource Decoupling: Configure lifecycle.ignore_changes on compute resources to prevent Terraform from overriding dynamic application tags.
• Scale Validation: Synchronize test suites with new lazy-execution contracts and verify stability against 15M+ row datasets within strict serverless constraints (2 vCPU / 4GB RAM).

Impact

• Scale & Speed: Drastically reduced peak memory footprint and accelerated transformation throughput via query optimization.
• Resiliency: Automated drift correction ensures infrastructure integrity before any application code reaches the execution environment.
• Maintainability: Clear separation between infrastructure provisioning and application deployment eliminates race conditions and configuration drift.