Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2101 commits
Select commit Hold shift + click to select a range
9deb745
Audited the relation.py file's docstrings and corrected some mistakes
Apr 21, 2026
c69a1a9
Updated docstrings and error messages in spec.py to reflect multi-mis…
Apr 21, 2026
57072ef
Updated docstrings and error messages in xga/relations/fit.py to refl…
Apr 21, 2026
58c3b97
Continuing doc string updates
Apr 21, 2026
a9c9c9f
Continuing doc string updates, this time sourcetools/density.py
Apr 21, 2026
881a2a8
Updating docstrings continues. Also, in match.py on_detector... funct…
Apr 22, 2026
9ebc3d2
Updated docstrings in stack.py and temperature.py; stack.py functions…
Apr 22, 2026
a19f085
Updated docstrings in various files of the xspec/fit directory - also…
Apr 23, 2026
72070bb
Even more docstrings updated - they should all have been audited now
Apr 23, 2026
2d41610
Added badge generation to tests run.py and removed old pytest files
Apr 23, 2026
09e5ea1
Made some stylistic changes to __init__.py of tests and added use of …
Apr 23, 2026
d60f1cd
Added TYPE_HINTING control to some imports to allow type hinting with…
Apr 23, 2026
32cde28
Allow an XGA_CONFIG_DIR env variable to be set to override where we n…
Apr 23, 2026
1ec9fb9
Hopefully made use of pandas in census match more robust if no obs en…
Apr 24, 2026
eb8ab6f
Added a utils.py to the tests directory - it contains definitions for…
Apr 24, 2026
f9f72ec
Made the optional XGA_CONFIG_DIR config path absolute after retrieval…
Apr 24, 2026
5ae8c8d
Added 'shims' to the xga overall init, makes sure that from xga.sas i…
Apr 24, 2026
b235faa
Added missing inits to testing subdirectories
Apr 24, 2026
7d35458
Removing an incorrect copyright message
Apr 24, 2026
0065b3e
Trying to get copyright message added properly?
Apr 24, 2026
0856570
Added lazy initialization of XGA, and support for re-initializing the…
Apr 25, 2026
5da9382
changes to the view method
jessicapilling Apr 27, 2026
3863ed6
y axis label changed to residuals
jessicapilling Apr 27, 2026
2663143
added residuals to view
jessicapilling Apr 27, 2026
63a5541
Implement lazy configuration, parallel census building, and dynamic r…
Apr 27, 2026
cfac026
Added copyright notices to the test submodule inits
Apr 27, 2026
d2c99a3
Document lazy loading, dynamic re-init, and parallel census management
Apr 27, 2026
847fffa
Added imports from lightcurve.py to the products/__init__.py, so top …
Apr 27, 2026
3f36e55
Individual test .py files have been altered to use 'get_test_source',…
Apr 27, 2026
0638692
Added pre-setup test setup file for xga, and a template config for DA…
Apr 27, 2026
ee82d54
Updated project info in conf.py in docs/source
Apr 27, 2026
ac3f484
Renamed DAXA template config file
Apr 27, 2026
5b94ca2
Altered the template daxa cfg to set num_cores = -1 for automatic det…
Apr 27, 2026
dcb4099
Fixed a bug where non-lazy-loaded constants defined in xga.utils were…
Apr 27, 2026
62a06ed
Moved information about sources used for tests to a new file - source…
Apr 27, 2026
f20d15e
Added a prepare_data script to get data ready for XGA tests, splittin…
Apr 27, 2026
c53edb9
Remove tests/run.py in favor of standard test runners
Apr 27, 2026
7708062
The prepare_data.py script will now make sure XGA census files are re…
Apr 27, 2026
8e5f2d3
The handling of NullSource ccf generation in execute_cmd got lost at …
Apr 27, 2026
f6597fa
Had to alter the paths in the test xga.cfg - they are now relative to…
Apr 27, 2026
8eb0b25
Changed where BLACKLIST is imported in sources/base.py - it is now in…
Apr 27, 2026
70140cd
Got rid of a stupid print in utils when rebuilding a census. Switched…
Apr 27, 2026
b9e3a86
Introduced a bug to generate/sas/phot.py where it would always error …
Apr 27, 2026
ac2c160
Left a print statement accidentally
Apr 27, 2026
b5db69b
Fixed an error I introduced where if the expmap it was meant to be ge…
Apr 27, 2026
6edaafc
Made changes to prepare_data.py to get around a limitation in how the…
Apr 27, 2026
6634fde
Added the beginnings of a readme for the test suite
Apr 27, 2026
fe038da
Added a warning filter to hopefully avoid a million duplicate depreca…
Apr 27, 2026
890dd2d
Updated the prepare_data.py and template XGA/DAXA configs to once aga…
Apr 27, 2026
08e9e90
Removed the sys call from the test_generate test files - shouldn't be…
Apr 27, 2026
40d036b
Removed the sys call from the quite a few more test files
Apr 27, 2026
4a23bd2
Removed sys calls from more test files in the test_sourcetools directory
Apr 27, 2026
ffed281
Finished removing the sys calls from test files
Apr 27, 2026
6f8c2fb
Fixed the broken signatures of some of the get_... methods of BaseSou…
Apr 28, 2026
7d186ee
Fixed bugs I introduced to the esass/phot.py file. Indexing a single …
Apr 28, 2026
8fa6b77
Fixed a broken test - the test was meant to ensure that a TelescopeNo…
Apr 28, 2026
0c2feb7
Fixed a problem with uncaught product not available error when checki…
Apr 29, 2026
8068325
Tiny typo fix
Apr 29, 2026
fa465ff
Fixed an unintended continue in sas/lightcurve.py that would always o…
Apr 29, 2026
62b22a6
Added a more useful message to the error in AnnularSpectra init that …
Apr 29, 2026
a08de5c
Improved the error message in AnnularSpectra further, as it will also…
Apr 30, 2026
370b81b
Fixed a typo in an error message and added a warning that hopefully n…
Apr 30, 2026
562d8c9
Utils will now lazy-load/search for particular mission's backend anal…
Apr 30, 2026
0af4f89
Adjusted the get combined methods to fix a problem I introduced earli…
Apr 30, 2026
b815920
Replaced the blanket mv command in eSASS generation command strings w…
Apr 30, 2026
277239b
Needed extra {} in some of the move strings in eSASS generation comma…
May 4, 2026
5fbbaec
Made an error string an fstring (it was already meant to be one, I ju…
May 4, 2026
9d2568a
Fixed a bug where no eSASS annular spectra could be successfully gene…
May 5, 2026
b711cef
Altered the astropy Cosmology import so that it is from astropy.cosmo…
May 5, 2026
55292ce
Altered the deprecation warning on find_all_wcs in utils.py, so hopef…
May 5, 2026
de97f71
Fixed how stacked spectra for eROSITA (and non-stacked) or retrieved …
May 5, 2026
7e98edb
Hopefully fixed pkg_resources deprecation warning by using importlib,…
May 5, 2026
bfc31e2
Added a very basic single source global apec test
May 6, 2026
f029dea
Fixed issues where both inst combined and TM1, TM2, etc. spectra were…
May 6, 2026
c9a474c
Replaced hopefully the last hold outs of if telescope == 'erosita', r…
May 6, 2026
d41fcb5
Added erass as a fully realized telescope option to utils, and simpli…
May 6, 2026
45198c5
Should have moved the tests over to using erass rather than erosita
May 6, 2026
a08fb3d
Hopefully removed the last hard coded 'erosita' assumption of erosita…
May 6, 2026
3b6f605
Updated the test xga.cfg to include an eROSITA section (the original …
May 6, 2026
31d45c8
Explicitly added erass_sky and erass_det units to the utils.py
May 6, 2026
06a1937
Updated the get_combined* methods in BaseSource to default to obsid c…
May 6, 2026
9a156c7
Altered the obs_check method in BaseSource so that separate ObsID, co…
May 6, 2026
bec67a6
Fixed Issue #779: Prevented CWD leakage in _existing_xga_products usi…
May 6, 2026
6827c9d
Trying to fix a spectrum retrieval issue I introduced into _spec_obj_…
May 6, 2026
ce96a81
Fixed 'erosita' telescope specification in esass_spectrum_set and add…
May 6, 2026
29a922c
The assert checks in one temperature profile test were wrong - they w…
May 6, 2026
0c10df4
Fixed a problem with the assertions in the test_min_cnt_proj_temp_pro…
May 6, 2026
e6e1a7a
Added a missing erass entry to the pattern variable in get_lightcurves
May 6, 2026
d4eedd5
Fixed a (hopefully) final hard coded erosita issue in the eSASS regio…
May 6, 2026
99c5952
evtool_combine_evts had a bug that at various points was in other eSA…
May 7, 2026
45bdd86
Fixed is-telescope-associated check in the get_luminosities and get_r…
May 7, 2026
01719fa
Made errors in AnnularSpectra add_fit_data more informative by detail…
May 7, 2026
3a1b4bf
Removed left over value error
May 7, 2026
75cbe00
Likely fixed one of the remaining failure points for tests, fetching …
May 8, 2026
084f278
Hopefully fixed a problem with fetching images in _make_spec_cmd_info…
May 8, 2026
577bc39
Added new logic to the conversion factor assignment in the xspec_call…
May 8, 2026
3d0f1f8
Fixed a problem with the _get_phot_prod method in BaseSource, so that…
May 8, 2026
1b27fc4
Improved the test_existing_prods_loaded_in_img test, and the combined…
May 8, 2026
0e01979
More changes to the other tests that check whether products are loade…
May 8, 2026
ed6cfe0
Fixed an issue in erosita spectral generation of combined observation…
May 8, 2026
2af6dd3
Added 'syntax error' to the list of things that will triggered error …
May 8, 2026
c58866c
Sorted out the commands constructed for eSASS lightcurve generation, …
May 8, 2026
27c8111
Improved the test_srctool_lightcurve_combine_insts_f_combine_obs_f te…
May 8, 2026
8dc65f1
Gave test_srctool_lightcurve_combine_insts_t_combine_obs_f the same t…
May 8, 2026
2ac90c4
The test_srctool_lightcurve_combine_insts_f_combine_obs_t test now ha…
May 8, 2026
1086016
Specified the energy bounds of all the light curves to be generated a…
May 8, 2026
7de1810
" king" was being passed to _setup_inv_abel_dens_onion_temp rather th…
May 10, 2026
eee3cb6
Hopefully fixed the TestEsassPhotFuncs.test_evtool_image test (was lo…
May 10, 2026
9ef42f5
Had left a mistake in test_evtool_image
May 10, 2026
31f3f37
Hopefully fixed the test_expmap test for eROSITA, same issue as the i…
May 10, 2026
dee7925
Specified the wrong upper energy limit in the expmap call
May 10, 2026
4504f19
Made the same change to eROSITA generation of individual Obs individ…
May 10, 2026
fdbad11
Added try-excepts to all product retrieval tests to catch NoProductAv…
May 10, 2026
a6d6c71
Added CLAUDE.md to the ignore
May 11, 2026
12829c3
Set hi and lo energy limits in the test_evtool_image_combined_obs tes…
May 11, 2026
f5b7bd5
Moved the code that raises errors identified during product generatio…
May 11, 2026
d695ec4
Added a product not usable error to the exceptions file
May 11, 2026
8d2c150
Added an extra check and error raise to the AnnularSpectra class init…
May 11, 2026
5524311
Combined the mission software decorators, so that esass_call, sas_cal…
May 11, 2026
e104c2b
Fixed indenting of gridspec kw
DavidT3 May 11, 2026
2a714c4
Replaced unnecessary use of subplots with figure()
DavidT3 May 11, 2026
cb7896e
Altered Spectrum view residual plotting to give full control over whi…
DavidT3 May 11, 2026
8fa538f
Removed 'SASERROR_LIST', 'SASWARNING_LIST', 'XSPEC_FIT_METHOD' from t…
May 11, 2026
ce2d04a
Fixed a bug in how esass_spectrum_set constructs file names for backg…
May 11, 2026
1f1ce41
Altered _existing_xga_products to use the correct suffix for eRASS/eR…
May 11, 2026
d7d37ce
Still trying to fix the problem of being unable to read in annular sp…
May 12, 2026
ac85084
_existing_xga_products will now catch ProductNotUsableError when decl…
May 12, 2026
96b375f
Fix model input normalization and subplot sharing
DavidT3 May 13, 2026
8a96d36
Ensured spectrum axis labels show correctly for show_residuals=True
DavidT3 May 13, 2026
b159664
Merge pull request #1498 from DavidT3/feat/addResidToSpecView
DavidT3 May 13, 2026
2ddab58
Screwed up the view method when I merged Jess' changes - unindented i…
May 13, 2026
84293fb
Commit of all changes resulting from the big merge of master into the…
May 18, 2026
5eb4e57
Made some changes to how get_profiles, get_combined_profiles, and _ge…
May 18, 2026
40cd49a
Removed telescope arguments to the get_conv_factor method of Spectrum…
May 18, 2026
e8a3480
Removed a legacy argument from AnnularSpectra generate_profile method…
May 18, 2026
9051c36
Replaced mutable default values in a few methods in products.base
May 18, 2026
b7991ad
Fixed some typos and also added support for generating cross-arfs to …
May 18, 2026
eb91aea
Replaced some string slices in BaseProduct init with .startwith calls
May 19, 2026
daf3170
Fixed a bug in how EventList calls the BaseProduct super, some argume…
May 19, 2026
24cf5bb
Fixing a broken import introduced in dealing with issue #1511
May 19, 2026
565b7c3
As far as I know I have re-implemented double_temp_apec to support mu…
May 19, 2026
7fbbfda
Fixed an indentation problem in the XSPEC power-law convenience funct…
May 19, 2026
ea2e152
Updated license to the correct year (again)
May 19, 2026
52fdc15
Added a 'global_temp_success_check' argument to sourcetools._common._…
May 19, 2026
7cde7dc
Updated the _setup_inv_abel_dens_onion_temp function to include an ar…
May 19, 2026
886fb06
Exposed the option of turning on checks for global temperature fit su…
May 19, 2026
3423f7d
Added entries for the 'stacked_spectra' and 'telescope' arguments to …
May 19, 2026
219125e
Improved the error message raised when XSPEC function signatures and …
May 19, 2026
757d445
Added final missing argument for single_temp_apec to the FIT_FUNC_ARG…
May 19, 2026
f454531
Added a telescope column header to the XSPEC fit inventory file defin…
May 19, 2026
1058df0
Fixed a merge-introduced bug in xspec_call that didn't pass the teles…
May 19, 2026
8dac03f
Altered the xspec_call decorator to be able to find a fit's telescope…
May 19, 2026
443275d
Removed a check on radii input in single_temp_apec_profile that was m…
May 19, 2026
6b7efde
Had to make some fixes where xspec_call was accessing fit configurati…
May 19, 2026
f097b00
Added a load_fits argument to get_test_source, to make it possible fo…
May 19, 2026
d8a6ba0
Added a new test 'test_global_single_temp_apec_diff_fitconf' that fit…
May 19, 2026
baff053
Hopefully dealt with a problem assigning fit results to annular spect…
May 19, 2026
a68242e
Added more to the multiple-fits-of-same-model test, where at the end …
May 19, 2026
d163803
Dictionaries for annular spectrum fit results were not being set up p…
May 19, 2026
a564b78
Added a very barebones test for annular profile fitting with cross-ar…
May 19, 2026
7434a19
Fixed the residual axis that can optionally be added to a figure prod…
May 20, 2026
f797a7b
Added a new test that calls the view method of Spectrum to see if plo…
May 20, 2026
0251a88
Added daxa_output, census + blacklist files, and the .idea hidden dir…
May 20, 2026
7e060f5
Removed a bunch of leftover __pycache__ directories that shouldn't be…
May 20, 2026
19bbc86
Added tests of reloading global XSPEC fit results, both for stacked a…
May 20, 2026
7921b7f
Bodged the cluster_cr_conv function so that it returns the telescope …
May 20, 2026
ef4ea90
Fixed assignment of conv factors to spectrum instances in xspec_call …
May 20, 2026
a37caaf
Fixed the formatting of an error message in get_annular_spectra, and …
May 20, 2026
284ba7e
Added a try except to catch missing annular spectra when loading in a…
May 20, 2026
78da1f7
Changed some asserts in the test_onion_deproj_temp_prof test so that …
May 20, 2026
892cf34
Fixed the draw_rads and draw_vals arguments in get_view of BaseProfil…
May 20, 2026
cddf879
Made the same draw_rads and draw_vals fixes in the view method of Bas…
May 20, 2026
633dff6
Fixed an issue in _get_prof_prod where a variable used by the rest of…
May 20, 2026
96deefd
Altered the signature of _get_spec_based_profiles so that it has obs_…
May 20, 2026
f6fe9c3
Commented out 'missing combined_ prefix' warning in BaseSource._get_p…
May 20, 2026
6543cc6
Tiny typo changes
May 20, 2026
d9d6e67
Fixed a problem in _get_prof_prod where radii was being compared with…
May 20, 2026
375f4a4
Added a new test_products sub-directory to the tests, and the test_pr…
May 20, 2026
a86c6bb
Added a test of drawing rads and vals on an individual profile
May 20, 2026
8d7ded3
Added drawing rads and vals test for aggregate profile views
May 20, 2026
db863fa
Added htmlcov to the gitignore
May 20, 2026
5d086f0
Fixed errors in HydrostaticMass, SpecificEntropy, and ThermalPressure…
May 20, 2026
ab6c015
Fixed a logical hole in the ThermalPressure, SpecificEntropy, and Hyd…
May 21, 2026
024fe6f
Added subtests to the reloading of XSPEC fits from disk tests, so tha…
May 21, 2026
a898e80
Replaced some more array comparisons in test_fit_general.py with nump…
May 21, 2026
8381c26
Fixed importlib use in utils.py (changed import from import importlib…
May 21, 2026
b0ab313
Updated utils, BaseSource, and functions in sourcetools.match to a) s…
May 21, 2026
c970811
Fixed a bug introduced in match.py when checking for true entries in …
May 21, 2026
95c6f62
Added extra checks in BaseSource after the self._initial_products cal…
May 21, 2026
cb68443
Do need the extra checks I introduced last time, but I'm experimentin…
May 21, 2026
1b800b8
Do need the extra checks I introduced last time, but I'm experimentin…
May 21, 2026
c0c0a40
Constructed warning about removed data
May 21, 2026
a3c31cf
Made sure the warning about removed data isn't shown if the source is…
May 22, 2026
c18778e
Fixed a tiny bug in y-axis label of spectrum view() method (a missing…
May 22, 2026
23173ce
Was accidentally setting up a figure twice in the 'view' method of Sp…
May 22, 2026
b9597cb
Split off the utils code that writes new XGA configuration files to i…
May 26, 2026
fcf16b6
Updated very old email addresses
May 28, 2026
3faae7a
Fixed an issue with the new implementation of config file writing I a…
May 28, 2026
a39ba65
Added an OSError catch when assembling observation census(es) - Jess …
May 28, 2026
fdf7b58
Updated the _existing_xga_products method to not call break if an ann…
Jun 2, 2026
838bdf7
Fixed the incorrect comparison to self.instruments rather than self.i…
Jun 2, 2026
cd7c233
Fixed the incorrect comparison to self.instruments rather than self.i…
Jun 2, 2026
f5a31a4
Hopefully fixed the use of contextlib chdir for Python 3.10 and below…
Jun 4, 2026
355fec4
Added mention of the optional contextlib2 dependency (well, optional …
Jun 4, 2026
db75a9d
Hopefully actually fixed the contextlib chdir missing from Python 3.1…
Jun 4, 2026
a1e21bf
Updated the docs to reflect the ACTUAL optional dependency that is re…
Jun 4, 2026
2c136ad
Fixed the hyperlink syntax I used in the docs rst files to link to th…
Jun 4, 2026
6c2ea77
Added a blank line
Jun 4, 2026
a093ee5
Fixed a problem with the final check of 'on_detector_match' function,…
Jun 4, 2026
f44cf97
Put the fix in the wrong place...
Jun 4, 2026
3e511d3
Made sure the blacklist and census constants used in _separation_sear…
Jun 11, 2026
0e394b1
Refactored a LOT of the separation match code in sourcetools/match - …
Jun 11, 2026
1c358c3
Made changes to the on_detector_match function to hopefully improve p…
Jun 12, 2026
38a2dc0
Optimized on_detector_match and separation_match for large samples
Jun 12, 2026
b91b7b4
Optimized on_detector_match performance and communication
Jun 12, 2026
5bf3c82
Added an argument to reinitialise_xga that allows the user to manuall…
Jun 15, 2026
7e38c0f
Added a mifbuild function for XMM SAS, first step in the conversion o…
Jun 15, 2026
0651e5b
Wasn't creating the temporary directory in mifbuild
Jun 15, 2026
7ab812d
Altered the cifbuild function to use a mif rather than looking throug…
Jun 15, 2026
6302c43
Forgot to add the mif path to the new version of the cifbuild command
Jun 15, 2026
f8cc054
Added a one element progress bar to the mifbuild function
Jun 15, 2026
69937b6
Small changes to cifbuild
Jun 15, 2026
4d432b3
Moved the raising of 'no obs date available' errors in cifbuild to ou…
Jun 16, 2026
2292d10
Removed some unnecessary 'does folder exist' checks from generate/sas…
Jun 16, 2026
06d4b11
Made sure that an error will be raised if trying to read the header o…
Jun 16, 2026
67a219e
Should now catch instances where attitude files are not usable in the…
Jun 16, 2026
9539406
Attempted to improve efficiency of the on_detector_match function
Jun 17, 2026
26422d6
Attempted to improve efficiency of the on_detector_match function - a…
Jun 17, 2026
0ebfe39
Hopefully sped up the matching process in on_detector_match even more
Jun 17, 2026
024fb35
Fixed a bug I introduced by deleting ObsIDs with no usable attitude file
Jun 17, 2026
6b28276
Made loading of fit results more robust in the _existing_xga_products…
Jun 23, 2026
505171a
Switched some 'break' calls to 'continue' calls when loading XSPEC fi…
Jun 23, 2026
689c727
Moved the _existing_xga_products method away from the iterrows calls …
Jun 23, 2026
3311ed1
Messed up some of the itertuples calls
Jun 23, 2026
e197399
Removed a leftover line that renamed the 'type' column of the inventory.
Jun 23, 2026
098e2b8
Disabled garbage collection during the _existing_xga_products call in…
Jun 23, 2026
fdd1395
Added an option to XGA products that will stop them from checking the…
Jun 23, 2026
ee08ef9
Disabled gc collect in a couple of places.
Jun 23, 2026
72b28ec
Hopefully basically removed the use of os.path.exists in the initiali…
Jun 23, 2026
31f968d
Switched the caches of file names in _existing_xga_products to sets f…
Jun 23, 2026
bf3114e
Hopefully replaced another os.path.exists call in the BaseSource clas…
Jun 23, 2026
5a2c4ce
Hopefully replaced another os.path.exists call in the BaseSource clas…
Jun 23, 2026
d10eb6c
Hopefully replaced another os.path.exists call in the BaseSource clas…
Jun 23, 2026
792e8b0
Passed check_exists=False to the initial product object set up in _in…
Jun 23, 2026
06a74a1
The ObsID-Inst combinations of combined data products accessed throug…
Jun 24, 2026
5e069d9
Hopefully fixed a problem with how _existing_xga_products retrieves s…
Jun 24, 2026
5e77a2f
Missed a few places where I was accessing 'row' as though it were sti…
Jun 24, 2026
0b2f4c4
Added a try except to catch missing annualr fit result files in the _…
Jun 24, 2026
d3b7dff
Hopefully sorted issue #1526
Jun 24, 2026
537370e
Manually updated the coverage badge to reflect the last coverage I re…
Jun 24, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
17 changes: 15 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,9 +1,22 @@
CLAUDE.md
/docs/source/notebooks/tutorials/whole_sample.csv
/docs/source/notebooks/advanced_tutorials/random_brightness_profile.xga
.coverage
.pytest_cache
xga_output
daxa_output
*census*
*blacklist*
.idea
.ipynb_checkpoints
.vscode
/xga.egg-info/
*__pycache__*
/docs/build
*__pycache__
xga.egg-info/
*noupload*
.DS_Store
/xga/__pycache__
*.pyc
xga/__pycache__/__init__.cpython-39.pyc
claude_context/
htmlcov
2 changes: 1 addition & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
BSD 3-Clause License

Copyright (c) 2024, David J Turner and the Contributors
Copyright (c) 2026, David J Turner and the Contributors
All rights reserved.

Redistribution and use in source and binary forms, with or without
Expand Down
8 changes: 6 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,9 @@ conda activate <YOUR ENVIRONMENT NAME GOES HERE>
pip install pyabel==0.9
```

> [!IMPORTANT]
> For Python 3.10 and earlier, the optional dependency [contextlib-chdir](https://pypi.org/project/contextlib-chdir/) must be installed.

## Required Dependencies
XGA depends on two non-Python pieces of software:
* XMM's Science Analysis System (SAS) - Version 17.0.0, but other versions should be largely compatible with the
Expand Down Expand Up @@ -90,8 +93,9 @@ Before XGA can be used you must fill out a configuration file (a completed examp

Follow these steps to fill out the configuration file:
1. Import XGA to generate the initial, incomplete, configuration file.
2. Navigate to ~/.config/xga and open xga.cfg in a text editor. The .config directory is usually hidden, so it is
2. Navigate to ~/.config/xga and open xga.cfg in a text editor. The .config directory is usually hidden, so it is
probably easier to navigate via the terminal.
* Note: The location of the configuration directory can be overridden by setting the ``XGA_CONFIG_DIR`` environment variable. If set, XGA will look for its configuration and store its census files in exactly that directory.
3. Take note of the entries that currently have /this/is/required at the beginning, without these entries the
module will not function.
4. Set the directory where you wish XGA to save the products and files it generates. I just set it to xga_output,
Expand Down Expand Up @@ -136,7 +140,7 @@ If you encounter a bug, or would like to make a feature request, please use the
[issues](https://github.com/DavidT3/XGA/issues) page, it really helps to keep track of everything.

However, if you have further questions, or just want to make doubly sure I notice the issue, feel free to send
me an email at turne540@msu.edu
me an email at djturner@umbc.edu



Expand Down
8 changes: 8 additions & 0 deletions docs/example_config/xga.cfg
Original file line number Diff line number Diff line change
Expand Up @@ -18,3 +18,11 @@ mos1_expmap = {obs_id}/images/{obs_id}-{lo_en}-{hi_en}keV-mos1_merged_expmap.fit
mos2_expmap = {obs_id}/images/{obs_id}-{lo_en}-{hi_en}keV-mos2_merged_expmap.fits
region_file = /home/dt237/apollo_mnt/xmm_obs/code/xapa/id_results/{obs_id}/final_class_regions_REDO.reg

[EROSITA_FILES]
root_erosita_dir = /home/dt237/apollo_mnt/erosita_obs/data/
clean_erosita_evts = {obs_id}/clean_evts.fits
lo_en = ['0.5']
hi_en = ['2.0']
erosita_image = {obs_id}/images/{obs_id}_combined_{lo_en}-{hi_en}keVimg.fits
erosita_expmap = {obs_id}/images/{obs_id}_combined_{lo_en}-{hi_en}keVexpmap.fits
region_file = /home/dt237/apollo_mnt/erosita_obs/regions/{obs_id}.reg
1 change: 0 additions & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ corner>=2.2.2
emcee>=3.1.6
tabulate>=0.9.0
getdist>=1.4.7
pytest>=6.2.5
cycler>=0.11.0
exceptiongroup>=1.0.4
importlib_resources~=6.5.2
31 changes: 4 additions & 27 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# This code is a part of X-ray: Generate and Analyse (XGA), a module designed for the XMM Cluster Survey (XCS).
# Last modified by David J Turner (turne540@msu.edu) 20/02/2023, 14:04. Copyright (c) The Contributors
# This code is part of X-ray: Generate and Analyse (XGA), a module designed for the XMM Cluster Survey (XCS).
# Last modified by David J Turner (djturner@umbc.edu) 4/27/26, 11:03 AM. Copyright (c) The Contributors.

# Configuration file for the Sphinx documentation builder.
#
Expand All @@ -16,8 +16,6 @@
import os
import sys
sys.path.insert(0, os.path.abspath('../..'))
import shutil


# -- Extra Setup for XGA -----------------------------------------------------

Expand All @@ -38,32 +36,11 @@
# As this file lives in the docs/source folder, we go two levels up for the absolute path the XGA repo folder
xga_path = '/'.join(current_path.split('/')[:-2])

# The test data includes a mostly complete config file, and census
test_cfg_path = os.path.join(xga_path, 'tests', 'test_data', 'xga.cfg')
test_census_path = os.path.join(xga_path, 'tests', 'test_data', 'census.csv')
test_data_path = os.path.join(xga_path, 'tests', 'test_data')

# Copy the config and census over to the place they should be on whatever system is building this
shutil.copy(test_cfg_path, config_file_path)
shutil.copy(test_census_path, census_file_path)

with open(config_file_path, 'r') as cfg:
lines = cfg.readlines()

lines = [line.replace('root_xmm_dir = ./', 'root_xmm_dir = {}'.format(test_data_path)) for line in lines]

with open(config_file_path, 'w') as cfg:
cfg.write(''.join(lines))

# I don't like to import in the code itself, but it has to be here so we can be sure a valid config file exists
# from xga import utils
# utils.XGA_MODE = "DOCS"

# -- Project information -----------------------------------------------------

project = 'X-ray: Generate and Analyse (XGA)'
copyright = '2022, David J Turner'
author = 'David J Turner'
copyright = '2026, The Contributors'
author = 'David J Turner, XGA Team'

# The full version, including alpha/beta/rc tags
# release = '0.1'
Expand Down
15 changes: 5 additions & 10 deletions docs/source/future.rst
Original file line number Diff line number Diff line change
@@ -1,20 +1,15 @@
Planned XGA Features
========================

* **Upper Limit X-ray Luminosities** - These luminosities are measured from photometric products when there are not sufficient X-ray counts to generate a spectrum. This will involve integrating a tool that I have already written for XCS into the structure of XGA, though rather than measuring upper limit luminosities from a single image, I intend to measure them from the combined data.
* **Support for other X-ray telescopes** - XGA's purpose is to facilitate the exploitation of the complete public X-ray archive, hence the aim is to support all X-ray telescopes eventually. Currently XMM and eROSITA are supported, and the software is setup so that integrating new telescope-specific softwares should not be an arduous task. If you are interested in integrating a telescope into XGA please reach out to the development team (:doc:`support`).

* **Source Finder** - It is likely that I will extend my Hierarchical Clustering Peak Finder into a full source finder, as well as completing the implementation of a convolutional peak finder/source finder, which was inspired by my friend and colleague Lucas Porth.
* **Creating a Docker image for users to download** - Creating a Docker environment with all relevant software already installed.

* **More XSPEC Models** - Including a two temperature APEC model, and support for custom user defined models.
* **More XSPEC Models** - Support for custom user defined models.

* **Implement emcee scaling relation fitter** - What it says on the tin really. Implement an emcee fitter for the scaling relations so that there is an MCMC option that doesn't require the installation of a bunch of optional dependencies.
* **Upper Limit X-ray Luminosities** - These luminosities are measured from photometric products when there are not sufficient X-ray counts to generate a spectrum. This will support measurements from combined data, as well as single images.

* **Ability to save ScalingRelation objects** - The ability to save ScalingRelation objects to disk in some way, so that code to generate them doesn't need to be run multiple times.

* **Support for other X-ray telescopes** - Support for generation and analysis of data products from other telescopes.

* **Creating a Docker image for users to download** - Creating a Docker environment with SAS and HEASoft already installed, for ease of use.
* **Implement emcee scaling relation fitter** - Implement an emcee fitter for the scaling relations so that there is an MCMC option that doesn't require the installation of optional dependencies.

* **Method for finding poorly removed point sources** - An attempt to mitigate occasional mistakes by source finders that produce regions that don't necessarily remove the entire point source.

* **Overdensity radius measurement** - Using HydrostaticMass profiles to measure the overdensity radius of a cluster.
Binary file added docs/source/images/flowchart.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/source/images/mulitmission.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
71 changes: 54 additions & 17 deletions docs/source/install.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,13 @@ Installing and Configuring XGA
==============

This is a slightly more complex installation than many Python modules, but shouldn't be too difficult. If you're
having issues feel free to contact me.
having issues feel free to get in contact (see :doc:`Getting Help` section).

Data Required For Using This Module
-----------------------------------

**This is very important** - Currently, to make use of this module, you **must** have access to cleaned XMM-Newton
event lists, as XGA is not yet capable of producing them itself.
**This is very important** - Currently, to make use of this module, you **must** have access to **either** cleaned XMM-Newton or eROSITA
event lists. For aquiring and cleaning X-ray data, the Python module `DAXA <https://github.com/DavidT3/DAXA>`_ is recommended.

Region Files
------------
Expand Down Expand Up @@ -50,19 +50,29 @@ We also provide a Conda lock file in the conda_envs directory (see `conda-lock G
conda activate <YOUR ENVIRONMENT NAME GOES HERE>
pip install pyabel==0.9

.. important::
For Python 3.10 and earlier, the optional dependency `contextlib-chdir <https://pypi.org/project/contextlib-chdir/>`_ must be installed.

Required Dependencies
---------------------

XGA depends on two non-Python pieces of software:
XGA depends on some non-Python pieces of software, dependent on the telescope from which your data originates:

For all telescopes:
* HEASoft's XSPEC - Version 12.10.1 - It isn't guaranteed later versions will work.

For XMM data:
* XMM's Science Analysis System (SAS) - Version 17.0.0, but other versions should be largely compatible with the software. SAS version 14.0.0 however, does not support features that PSF correction of images depends on.
* HEASoft's XSPEC - Version 12.10.1 - I can't guarantee later versions will work.

For eROSITA data:
* eROSITA Science Analysis Software System (eSASS) - EDR version. It must be installed on the system, we do not support interaction through a Docker container.

All required Python modules can be found in requirements.txt, and should be added to your system during the installation of XGA.

Excellent installation guides for `SAS <https://www.cosmos.esa.int/web/xmm-newton/sas-installation>`_ and
`HEASoft <https://heasarc.gsfc.nasa.gov/lheasoft/install.html>`_ already exist, so I won't go into that here.
XGA will not generate XMM products without detecting SAS, and will not fit spectra without detecting XSPEC.
Excellent installation guides for `SAS <https://www.cosmos.esa.int/web/xmm-newton/sas-installation>`_,
`HEASoft <https://heasarc.gsfc.nasa.gov/lheasoft/install.html>`_, and `eSASS <https://erosita.mpe.mpg.de/edr/DataAnalysis/esassinstall.html>`_ already exist, please see them for detailed installation instructions.
XGA will not generate XMM products without detecting SAS, or generate eROSITA products without detecting eSASS, or fit
spectra without detecting XSPEC.

Optional Dependencies
---------------------
Expand All @@ -85,35 +95,62 @@ Before XGA can be used you must fill out a configuration file (a completed examp
Follow these steps to fill out the configuration file:

1. Import XGA to generate the initial, incomplete, configuration file.
* Note: XGA uses a **lazy loading** architecture. Importing the module does not perform any filesystem checks or software discovery immediately. These only occur when a configuration variable or data-dependent function is first accessed.
2. Navigate to ~/.config/xga and open xga.cfg in a text editor. The .config directory is usually hidden, so it is probably easier to navigate via the terminal.
* Note: The location of the configuration directory can be overridden by setting the ``XGA_CONFIG_DIR`` environment variable. If set, XGA will look for its configuration and store its census files in exactly that directory. Because of XGA's lazy loading, you can even set this environment variable **after** importing XGA in a session.
* Note: If you need to switch configuration directories mid-session, you can use the ``reinitialise_xga()`` function:

.. code-block:: python

import xga
# Change to a new config directory
xga.reinitialise_xga('/path/to/new/config/')
3. Take note of the entries that currently have /this/is/required at the beginning, without these entries the module will not function.
4. Set the directory in which XGA will save the products and files it generates. I just set it to xga_output, so wherever I run a script that imports XGA it will create a folder called xga_output there. You could choose to use an absolute path and have a global XGA folder however, it would make a lot of sense.
4. Set the directory in which XGA will save the products and files it generates. It is advised to just set it to xga_output, so wherever you run a script that imports XGA it will create a folder called xga_output there. You could choose to use an absolute path and have a global XGA folder however, it wouldn't make a lot of sense.
5. You may also set an optional parameter in the [XGA_SETUP] section, 'num_cores'. If you wish to manually limit the number of cores that XGA is allowed to use, then set this to an integer value, e.g. num_cores = 10. You can also set this at runtime, by importing NUM_CORES from xga and setting that to a value.
6. The root_xmm_dir entry is the path of the parent folder containing all of your observation data.
6. The root_<telescope>_dir entry is the path of the parent folder containing all of your observation data for <telescope>. It is not necessary to have data for all telescopes that XGA supports.
7. Most of the other entries tell XGA how different files are named. clean_pn_evts, for instance, gives the naming convention for the cleaned PN events files that XGA generates products from.
8. Bear in mind when filling in the file fields that XGA uses the Python string formatting convention, so **anywhere you see {obs_id} will be filled formatted with the ObsID of interest when XGA is actually running**.
9. The lo_en and hi_en entries can be used to tell XGA what images and exposure maps you may already have. For instance, if you already had 0.50-2.00keV and 2.00-10.00keV images and exposure maps, you could set lo_en = ['0.50', '2.00'] and hi_en = ['2.00', '10.00'].
10. Finally, the region_file entry tells XGA where region files for each observation are stored (if they exist).

**Disclaimer: If region files are supplied, XGA also expects at least one image per instrument per observation, for WCS information.**
**Disclaimer: If region files are supplied for XMM data, XGA also expects at least one image per instrument per observation, for WCS information.**

I have tried to make this section as general as possible, but I am biased by how my research group generates and
This section aims to be as general as possible, but is biased by how our research group generates and
stores our data products. If you are an X-ray astronomer who wishes to use this module, but it seems to be incompatible
with your setup, please get in touch or raise an issue.

**Remote Data Access:** If your data lives on a remote server, and you want to use XGA on a local machine, I recommend
setting up an SFTP connection and mounting the server as an external volume. Then you can fill out the configuration
file with paths going through the mount folder - its how I use it a lot of the time.
**Remote Data Access:** If your data lives on a remote server, and you want to use XGA on a local machine, it is recommended
to set up an SFTP connection and mounting the server as an external volume. Then you can fill out the configuration
file with paths going through the mount folder.
To mount a server, one can follow the steps detailed in this `guide <https://sftptogo.com/blog/how-to-mount-sftp-as-a-drive-on-mac/>`_.

XGA's First Run After Configuration
-----------------------------------

The first time you import any part of XGA, it will create an 'observation census', where it will search through
The first time you access any configuration-dependent part of XGA, it will create an 'observation census', where it will search through
all the observations it can find (based on your entries in the configuration file), check that there are events
lists present, and record the pointing RA and DEC. *This can take a while*, but will only take that long on the first
run. The module will check the census against your observation directory and see if it needs to be updated on
run. This process is fully parallelized and will utilize your available CPU cores to speed up the filesystem I/O.

The module will check the census against your observation directory and see if it needs to be updated on
every run.

Census Management
-----------------

XGA provides the ``rebuild_census()`` function for manual management of the observation census. This is useful if you have deleted data from your disk or want to force a fresh scan of your archives.

* **Standard Update**: Calling ``rebuild_census()`` without arguments is equivalent to the update check performed at startup.
* **Full Rebuild**: ``rebuild_census(full_rebuild=True)`` deletes the existing census and parses every FITS header from scratch.
* **Cleanup**: ``rebuild_census(clean_dead=True)`` checks the current census against the filesystem and removes entries for ObsIDs that no longer exist.

.. code-block:: python

import xga
# Prune census entries for data that has been deleted from disk
xga.rebuild_census(clean_dead=True)

Blacklisting ObsIDs
-------------------

Expand Down
Loading