Skip to content

Add PIT (Probability Integral Transform) plot to pted_coverage_test#16

Merged
ConnorStoneAstro merged 8 commits into
mainfrom
copilot/add-pit-graph-for-pted-results
Apr 25, 2026
Merged

Add PIT (Probability Integral Transform) plot to pted_coverage_test#16
ConnorStoneAstro merged 8 commits into
mainfrom
copilot/add-pit-graph-for-pted-results

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Apr 23, 2026

  • Add pit_plot(pvals, saveto, confidence=0.95) utility function to utils.py with KS confidence bands
  • Export pit_plot in utils.py __all__
  • Add pit_plot/pit_confidence parameters to pted_coverage_test() in pted.py
  • Update docstrings describing the new parameters
  • Update README.md with description of the new parameters
  • Add tests in tests/test_pted.py and tests/test_utils.py
  • Fix README: pit_confidence default in signature block corrected from 0.9 to 0.95 (matches code)
  • Fix utils.py: ravel pvals once after asarray so sort and kstest always operate on a 1-D array

Copilot AI linked an issue Apr 23, 2026 that may be closed by this pull request
Agent-Logs-Url: https://github.com/ConnorStoneAstro/pted/sessions/c34db391-d3e0-45dc-96af-461051196d9f

Co-authored-by: ConnorStoneAstro <78555321+ConnorStoneAstro@users.noreply.github.com>
Copilot AI changed the title [WIP] Add PIT graph for pted results Add PIT (Probability Integral Transform) plot to pted and pted_coverage_test Apr 23, 2026
Copilot AI requested a review from ConnorStoneAstro April 23, 2026 13:11
@codecov-commenter
Copy link
Copy Markdown

codecov-commenter commented Apr 23, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 98.05%. Comparing base (728bd53) to head (d64e673).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
+ Coverage   97.87%   98.05%   +0.17%     
==========================================
  Files           4        4              
  Lines         329      359      +30     
==========================================
+ Hits          322      352      +30     
  Misses          7        7              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

ConnorStoneAstro and others added 2 commits April 23, 2026 10:20
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
@ConnorStoneAstro
Copy link
Copy Markdown
Owner

@copilot could you add a monkeypatch test for the pit utility to cover the case where matplotlib is not available? It should check that the warning is properly thrown

Agent-Logs-Url: https://github.com/ConnorStoneAstro/pted/sessions/94059e7f-7b1d-4c51-988b-4c79de3c2490

Co-authored-by: ConnorStoneAstro <78555321+ConnorStoneAstro@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 23, 2026

@copilot could you add a monkeypatch test for the pit utility to cover the case where matplotlib is not available? It should check that the warning is properly thrown

Added test_pit_plot_no_matplotlib in tests/test_utils.py (commit 28d31db). It follows the same monkeypatch pattern as the existing test_sbc_histogram test — sets matplotlib.pyplot to None in sys.modules and asserts that a UserWarning matching "matplotlib" is raised.

@ConnorStoneAstro ConnorStoneAstro marked this pull request as ready for review April 23, 2026 20:33
Copilot AI review requested due to automatic review settings April 23, 2026 20:33
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a Probability Integral Transform (PIT) plotting capability intended to help visually assess uniformity/calibration of per-simulation p-values produced by PTED coverage testing.

Changes:

  • Introduces pit_plot(...) in src/pted/utils.py (with KS-style confidence band and KS test annotation) and exports it via __all__.
  • Extends pted_coverage_test() with pit_plot/pit_confidence parameters to optionally write a PIT plot file.
  • Adds/updates tests and README documentation covering the new PIT plotting behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
src/pted/utils.py Adds pit_plot utility and exports it.
src/pted/pted.py Adds optional PIT plot output to pted_coverage_test().
tests/test_pted.py Adds tests for PIT plotting via coverage test and direct utility call.
tests/test_utils.py Adds test asserting a warning when matplotlib is unavailable for PIT plotting.
README.md Documents PIT plot parameters and usage.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md
Comment thread README.md Outdated
Comment thread src/pted/pted.py
Comment thread tests/test_pted.py
Comment thread src/pted/utils.py
Comment thread src/pted/utils.py
Comment thread src/pted/utils.py
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@ConnorStoneAstro ConnorStoneAstro changed the title Add PIT (Probability Integral Transform) plot to pted and pted_coverage_test Add PIT (Probability Integral Transform) plot to pted_coverage_test Apr 24, 2026
Agent-Logs-Url: https://github.com/ConnorStoneAstro/pted/sessions/c63d7cf2-32ae-4f01-9616-46fd8992abe6

Co-authored-by: ConnorStoneAstro <78555321+ConnorStoneAstro@users.noreply.github.com>
Copy link
Copy Markdown
Contributor Author

Copilot AI commented Apr 24, 2026

@copilot could you add a monkeypatch test for the pit utility to cover the case where matplotlib is not available? It should check that the warning is properly thrown

test_pit_plot_no_matplotlib is already in tests/test_utils.py (commit 28d31db). It monkeypatches matplotlib.pyplot to None in sys.modules and asserts a UserWarning matching "matplotlib" is raised.

@ConnorStoneAstro ConnorStoneAstro merged commit 0bc4398 into main Apr 25, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PIT graph for pted results

4 participants