You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Context: New scheduled jobs (#2, #34, plus the extended contract listener from #20) are new failure surfaces. src/monitor.rs and src/alerts.rs already exist for this kind of thing.
Where:
src/monitor.rs, src/alerts.rs
What to do:
Add health checks / alert rules for the new jobs (job hasn't run in N intervals, job erroring repeatedly), following whatever pattern already covers the existing jobs.
Alert on "hasn't run successfully in N expected intervals" and "erroring repeatedly," not just "is the process running."
Watch out for:
Alert fatigue is a real risk if thresholds are too tight for jobs with long intervals (e.g. an hourly job) -- calibrate the "hasn't run" threshold to a few missed intervals, not one.
Definition of done:
New jobs are covered by the same monitoring/alerting as existing jobs, not silently unmonitored
Context: New scheduled jobs (#2, #34, plus the extended contract listener from #20) are new failure surfaces.
src/monitor.rsandsrc/alerts.rsalready exist for this kind of thing.Where:
src/monitor.rs,src/alerts.rsWhat to do:
Suggested approach:
src/monitor.rs/src/alerts.rsalready cover the existing jobs (corridor refresh, anchor refresh, etc.) and add the same shape of health check for Add daily-active-accounts computation job in src/jobs #2's DAA job, Add Soroban contract-invocation ingestion pipeline in src/ingestion #20's extended listener, and Add hourly background job to compute top movers in src/jobs #34's top-movers job.Watch out for:
Definition of done: