ci(fix): refactor serverless-system-tests-build-layer#16348
ci(fix): refactor serverless-system-tests-build-layer#16348rithikanarayan wants to merge 2 commits intomainfrom
Conversation
Codeowners resolved as |
Performance SLOsComparing candidate rithika.narayan/refactor-system-tests (97cc837) with baseline main (229230f) 📈 Performance Regressions (4 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 105.777µs (SLO: <130.000µs 📉 -18.6%) vs baseline: +4.5% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.1% ✅ add_inplace_aspectTime: ✅ 102.669µs (SLO: <130.000µs 📉 -21.0%) vs baseline: +0.3% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ add_inplace_noaspectTime: ✅ 28.217µs (SLO: <40.000µs 📉 -29.5%) vs baseline: -0.2% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.1% ✅ add_noaspectTime: ✅ 49.366µs (SLO: <70.000µs 📉 -29.5%) vs baseline: +1.2% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.0% ✅ bytearray_aspectTime: ✅ 252.946µs (SLO: <400.000µs 📉 -36.8%) vs baseline: +0.7% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.2% ✅ bytearray_extend_aspectTime: ✅ 649.486µs (SLO: <800.000µs 📉 -18.8%) vs baseline: +2.6% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ bytearray_extend_noaspectTime: ✅ 270.272µs (SLO: <400.000µs 📉 -32.4%) vs baseline: +1.7% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ bytearray_noaspectTime: ✅ 140.774µs (SLO: <300.000µs 📉 -53.1%) vs baseline: +1.7% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.1% ✅ bytes_aspectTime: ✅ 222.028µs (SLO: <300.000µs 📉 -26.0%) vs baseline: +2.0% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ bytes_noaspectTime: ✅ 134.661µs (SLO: <200.000µs 📉 -32.7%) vs baseline: -1.0% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ bytesio_aspectTime: ✅ 3.873ms (SLO: <5.000ms 📉 -22.5%) vs baseline: +0.6% Memory: ✅ 43.116MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ bytesio_noaspectTime: ✅ 321.129µs (SLO: <420.000µs 📉 -23.5%) vs baseline: +0.9% Memory: ✅ 43.136MB (SLO: <46.000MB -6.2%) vs baseline: +5.3% ✅ capitalize_aspectTime: ✅ 88.737µs (SLO: <300.000µs 📉 -70.4%) vs baseline: +1.1% Memory: ✅ 43.273MB (SLO: <46.000MB -5.9%) vs baseline: +5.4% ✅ capitalize_noaspectTime: ✅ 261.217µs (SLO: <300.000µs 📉 -12.9%) vs baseline: +3.6% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ casefold_aspectTime: ✅ 89.158µs (SLO: <500.000µs 📉 -82.2%) vs baseline: +1.0% Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ casefold_noaspectTime: ✅ 308.869µs (SLO: <500.000µs 📉 -38.2%) vs baseline: +0.2% Memory: ✅ 43.195MB (SLO: <46.000MB -6.1%) vs baseline: +5.5% ✅ decode_aspectTime: ✅ 86.587µs (SLO: <100.000µs 📉 -13.4%) vs baseline: -0.1% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +4.7% ✅ decode_noaspectTime: ✅ 154.469µs (SLO: <210.000µs 📉 -26.4%) vs baseline: +0.6% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ encode_aspectTime: ✅ 85.514µs (SLO: <200.000µs 📉 -57.2%) vs baseline: +0.3% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.2% ✅ encode_noaspectTime: ✅ 143.198µs (SLO: <200.000µs 📉 -28.4%) vs baseline: +3.1% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ format_aspectTime: ✅ 14.675ms (SLO: <19.200ms 📉 -23.6%) vs baseline: ~same Memory: ✅ 43.234MB (SLO: <46.000MB -6.0%) vs baseline: +5.3% ✅ format_map_aspectTime: ✅ 16.435ms (SLO: <21.500ms 📉 -23.6%) vs baseline: -0.1% Memory: ✅ 43.136MB (SLO: <46.000MB -6.2%) vs baseline: +5.4% ✅ format_map_noaspectTime: ✅ 373.955µs (SLO: <500.000µs 📉 -25.2%) vs baseline: +2.0% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +4.9% ✅ format_noaspectTime: ✅ 305.304µs (SLO: <500.000µs 📉 -38.9%) vs baseline: +2.9% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.0% ✅ index_aspectTime: ✅ 125.373µs (SLO: <300.000µs 📉 -58.2%) vs baseline: +1.6% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ index_noaspectTime: ✅ 39.980µs (SLO: <300.000µs 📉 -86.7%) vs baseline: -0.3% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.9% ✅ join_aspectTime: ✅ 211.983µs (SLO: <300.000µs 📉 -29.3%) vs baseline: -1.7% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.3% ✅ join_noaspectTime: ✅ 142.318µs (SLO: <300.000µs 📉 -52.6%) vs baseline: -1.4% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.0% ✅ ljust_aspectTime: ✅ 608.742µs (SLO: <700.000µs 📉 -13.0%) vs baseline: 📈 +20.4% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ ljust_noaspectTime: ✅ 255.673µs (SLO: <300.000µs 📉 -14.8%) vs baseline: ~same Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.1% ✅ lower_aspectTime: ✅ 296.048µs (SLO: <500.000µs 📉 -40.8%) vs baseline: +0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ lower_noaspectTime: ✅ 235.088µs (SLO: <300.000µs 📉 -21.6%) vs baseline: +1.5% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.9% ✅ lstrip_aspectTime: ✅ 0.277ms (SLO: <3.000ms 📉 -90.8%) vs baseline: +0.2% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +5.0% ✅ lstrip_noaspectTime: ✅ 0.179ms (SLO: <3.000ms 📉 -94.0%) vs baseline: -1.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.9% ✅ modulo_aspectTime: ✅ 14.364ms (SLO: <18.750ms 📉 -23.4%) vs baseline: -0.3% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.2% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.818ms (SLO: <19.350ms 📉 -23.4%) vs baseline: ~same Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.4% ✅ modulo_aspect_for_bytesTime: ✅ 14.423ms (SLO: <18.900ms 📉 -23.7%) vs baseline: ~same Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +4.9% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.644ms (SLO: <19.150ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 43.116MB (SLO: <46.000MB -6.3%) vs baseline: +4.9% ✅ modulo_noaspectTime: ✅ 0.359ms (SLO: <3.000ms 📉 -88.0%) vs baseline: +1.0% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.8% ✅ replace_aspectTime: ✅ 18.456ms (SLO: <24.000ms 📉 -23.1%) vs baseline: ~same Memory: ✅ 43.175MB (SLO: <46.000MB -6.1%) vs baseline: +5.2% ✅ replace_noaspectTime: ✅ 281.338µs (SLO: <300.000µs -6.2%) vs baseline: -1.7% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.1% ✅ repr_aspectTime: ✅ 315.118µs (SLO: <420.000µs 📉 -25.0%) vs baseline: -2.2% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% ✅ repr_noaspectTime: ✅ 46.355µs (SLO: <90.000µs 📉 -48.5%) vs baseline: -0.5% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.2% ✅ rstrip_aspectTime: ✅ 393.917µs (SLO: <500.000µs 📉 -21.2%) vs baseline: -0.7% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +4.8% ✅ rstrip_noaspectTime: ✅ 185.296µs (SLO: <300.000µs 📉 -38.2%) vs baseline: +1.4% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.7% ✅ slice_aspectTime: ✅ 184.370µs (SLO: <300.000µs 📉 -38.5%) vs baseline: -1.1% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ slice_noaspectTime: ✅ 54.000µs (SLO: <90.000µs 📉 -40.0%) vs baseline: -0.4% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ stringio_aspectTime: ✅ 4.539ms (SLO: <5.000ms -9.2%) vs baseline: 📈 +16.7% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ stringio_noaspectTime: ✅ 357.809µs (SLO: <500.000µs 📉 -28.4%) vs baseline: +1.8% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.9% ✅ strip_aspectTime: ✅ 278.862µs (SLO: <350.000µs 📉 -20.3%) vs baseline: -0.3% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ strip_noaspectTime: ✅ 181.362µs (SLO: <240.000µs 📉 -24.4%) vs baseline: +0.4% Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ swapcase_aspectTime: ✅ 344.036µs (SLO: <500.000µs 📉 -31.2%) vs baseline: +2.4% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ swapcase_noaspectTime: ✅ 279.612µs (SLO: <400.000µs 📉 -30.1%) vs baseline: +2.6% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.9% ✅ title_aspectTime: ✅ 326.140µs (SLO: <500.000µs 📉 -34.8%) vs baseline: +0.8% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.0% ✅ title_noaspectTime: ✅ 264.469µs (SLO: <400.000µs 📉 -33.9%) vs baseline: +3.3% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.0% ✅ translate_aspectTime: ✅ 499.450µs (SLO: <700.000µs 📉 -28.6%) vs baseline: +2.6% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.9% ✅ translate_noaspectTime: ✅ 424.265µs (SLO: <500.000µs 📉 -15.1%) vs baseline: +0.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.0% ✅ upper_aspectTime: ✅ 297.040µs (SLO: <500.000µs 📉 -40.6%) vs baseline: +1.1% Memory: ✅ 43.136MB (SLO: <46.000MB -6.2%) vs baseline: +5.1% ✅ upper_noaspectTime: ✅ 236.898µs (SLO: <400.000µs 📉 -40.8%) vs baseline: +2.1% Memory: ✅ 43.234MB (SLO: <46.000MB -6.0%) vs baseline: +5.6% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 509.532µs (SLO: <700.000µs 📉 -27.2%) vs baseline: 📈 +19.5% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.7% ✅ ospathbasename_noaspectTime: ✅ 428.529µs (SLO: <700.000µs 📉 -38.8%) vs baseline: -1.9% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +5.8% ✅ ospathjoin_aspectTime: ✅ 615.264µs (SLO: <700.000µs 📉 -12.1%) vs baseline: -0.9% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.4% ✅ ospathjoin_noaspectTime: ✅ 624.562µs (SLO: <700.000µs 📉 -10.8%) vs baseline: -0.5% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.9% ✅ ospathnormcase_aspectTime: ✅ 351.230µs (SLO: <700.000µs 📉 -49.8%) vs baseline: -2.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.7% ✅ ospathnormcase_noaspectTime: ✅ 361.284µs (SLO: <700.000µs 📉 -48.4%) vs baseline: -1.5% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +5.7% ✅ ospathsplit_aspectTime: ✅ 492.223µs (SLO: <700.000µs 📉 -29.7%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +5.6% ✅ ospathsplit_noaspectTime: ✅ 496.768µs (SLO: <700.000µs 📉 -29.0%) vs baseline: -0.7% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +5.7% ✅ ospathsplitdrive_aspectTime: ✅ 371.587µs (SLO: <700.000µs 📉 -46.9%) vs baseline: -2.9% Memory: ✅ 43.077MB (SLO: <46.000MB -6.4%) vs baseline: +5.7% ✅ ospathsplitdrive_noaspectTime: ✅ 72.927µs (SLO: <700.000µs 📉 -89.6%) vs baseline: +1.5% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +5.2% ✅ ospathsplitext_aspectTime: ✅ 463.419µs (SLO: <700.000µs 📉 -33.8%) vs baseline: ~same Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +5.3% ✅ ospathsplitext_noaspectTime: ✅ 461.765µs (SLO: <700.000µs 📉 -34.0%) vs baseline: -1.8% Memory: ✅ 43.116MB (SLO: <46.000MB -6.3%) vs baseline: +6.1% 📈 iastaspectssplit - 12/12✅ rsplit_aspectTime: ✅ 167.739µs (SLO: <250.000µs 📉 -32.9%) vs baseline: 📈 +13.6% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.2% ✅ rsplit_noaspectTime: ✅ 166.706µs (SLO: <250.000µs 📉 -33.3%) vs baseline: +8.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +5.5% ✅ split_aspectTime: ✅ 153.743µs (SLO: <250.000µs 📉 -38.5%) vs baseline: +6.3% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +5.9% ✅ split_noaspectTime: ✅ 163.908µs (SLO: <250.000µs 📉 -34.4%) vs baseline: +9.2% Memory: ✅ 43.018MB (SLO: <46.000MB -6.5%) vs baseline: +5.9% ✅ splitlines_aspectTime: ✅ 153.288µs (SLO: <250.000µs 📉 -38.7%) vs baseline: +6.0% Memory: ✅ 42.762MB (SLO: <46.000MB -7.0%) vs baseline: +5.1% ✅ splitlines_noaspectTime: ✅ 156.913µs (SLO: <250.000µs 📉 -37.2%) vs baseline: +4.9% Memory: ✅ 43.155MB (SLO: <46.000MB -6.2%) vs baseline: +6.2% 📈 telemetryaddmetric - 30/30✅ 1-count-metric-1-timesTime: ✅ 3.563µs (SLO: <20.000µs 📉 -82.2%) vs baseline: 📈 +21.2% Memory: ✅ 35.566MB (SLO: <38.000MB -6.4%) vs baseline: +5.7% ✅ 1-count-metrics-100-timesTime: ✅ 208.316µs (SLO: <220.000µs -5.3%) vs baseline: +2.8% Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +5.7% ✅ 1-distribution-metric-1-timesTime: ✅ 3.401µs (SLO: <20.000µs 📉 -83.0%) vs baseline: +3.2% Memory: ✅ 35.291MB (SLO: <38.000MB -7.1%) vs baseline: +5.0% ✅ 1-distribution-metrics-100-timesTime: ✅ 221.229µs (SLO: <230.000µs -3.8%) vs baseline: +2.0% Memory: ✅ 35.566MB (SLO: <38.000MB -6.4%) vs baseline: +5.7% ✅ 1-gauge-metric-1-timesTime: ✅ 2.180µs (SLO: <20.000µs 📉 -89.1%) vs baseline: +0.4% Memory: ✅ 35.507MB (SLO: <38.000MB -6.6%) vs baseline: +5.9% ✅ 1-gauge-metrics-100-timesTime: ✅ 136.877µs (SLO: <150.000µs -8.7%) vs baseline: +0.2% Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +5.7% ✅ 1-rate-metric-1-timesTime: ✅ 3.197µs (SLO: <20.000µs 📉 -84.0%) vs baseline: +4.7% Memory: ✅ 35.547MB (SLO: <38.000MB -6.5%) vs baseline: +6.0% ✅ 1-rate-metrics-100-timesTime: ✅ 222.139µs (SLO: <250.000µs 📉 -11.1%) vs baseline: +1.9% Memory: ✅ 35.586MB (SLO: <38.000MB -6.4%) vs baseline: +6.0% ✅ 100-count-metrics-100-timesTime: ✅ 20.395ms (SLO: <22.000ms -7.3%) vs baseline: +2.3% Memory: ✅ 35.586MB (SLO: <38.000MB -6.4%) vs baseline: +5.4% ✅ 100-distribution-metrics-100-timesTime: ✅ 2.294ms (SLO: <2.550ms 📉 -10.0%) vs baseline: +3.4% Memory: ✅ 35.606MB (SLO: <38.000MB -6.3%) vs baseline: +6.1% ✅ 100-gauge-metrics-100-timesTime: ✅ 1.404ms (SLO: <1.550ms -9.4%) vs baseline: +0.4% Memory: ✅ 35.527MB (SLO: <38.000MB -6.5%) vs baseline: +6.1% ✅ 100-rate-metrics-100-timesTime: ✅ 2.228ms (SLO: <2.550ms 📉 -12.6%) vs baseline: +2.2% Memory: ✅ 35.547MB (SLO: <38.000MB -6.5%) vs baseline: +6.1% ✅ flush-1-metricTime: ✅ 4.606µs (SLO: <20.000µs 📉 -77.0%) vs baseline: +1.6% Memory: ✅ 35.645MB (SLO: <38.000MB -6.2%) vs baseline: +5.0% ✅ flush-100-metricsTime: ✅ 174.420µs (SLO: <250.000µs 📉 -30.2%) vs baseline: -0.1% Memory: ✅ 35.606MB (SLO: <38.000MB -6.3%) vs baseline: +4.8% ✅ flush-1000-metricsTime: ✅ 2.175ms (SLO: <2.500ms 📉 -13.0%) vs baseline: -0.2% Memory: ✅ 36.451MB (SLO: <38.750MB -5.9%) vs baseline: +4.9%
|
Co-authored-by: Florentin Labelle <florentin.labelle@datadoghq.com>
Description
The definition of the ddtrace dependency in datadog-lambda-python is being modified and the build_layers.sh script in that repo is being refactored to deduplicate the use of the sed command injecting the ddtrace wheel in different repos PR. This PR brings dd-trace-py into compatibility with those changes.
Testing
Risks
Additional Notes