From 757d571a007d7455326df9644686a1b5b16f1937 Mon Sep 17 00:00:00 2001 From: Spencer Schwarz Date: Thu, 23 Jul 2026 01:45:59 -0400 Subject: [PATCH 1/2] Migrate GUI from PyQt5 to PyQt6 for napari 0.7 / Python 3.14 WHY --- The HCS GUI embeds napari viewers (Live, Multichannel, and the unified Mosaic/Plate view) for image display. napari 0.7 targets PyQt6, but the app was pinning the Qt binding to PyQt5 via `QT_API=pyqt5`. Running napari 0.7 + vispy 0.16 under PyQt5 made the Mosaic view glitchy/broken and blocked moving to Python 3.14 (PyQt5 has no 3.14 wheels; old napari pulled in pydantic v1, which is also incompatible with 3.14). See "Python 3.14 Compatibility Check.md" for the full dependency analysis behind these decisions. This migrates the app to PyQt6 so the embedded napari 0.7 viewers render correctly and the stack can run on Python 3.14 (napari 0.7.0, PyQt6 6.11, numpy 2.4, pydantic v2). WHAT CHANGED ------------ 1. Qt binding flipped PyQt5 -> PyQt6. `os.environ["QT_API"]` is set to "pyqt6" in every entry module that configures it before the first qtpy import: main_hcs.py, control/gui_hcs.py, control/widgets.py, control/console.py, control/single_instance.py, control/core/core.py, control/core_volumetric_imaging.py, control/core_usbspectrometer.py, control/core_PDAF.py, control/core_displacement_measurement.py, control/widgets_usbspectrometer.py, scripts/run_acquisition.py (child env), and tests/control/test_single_instance.py. Almost all Qt access already goes through qtpy, which shims the common PyQt5->PyQt6 differences (unscoped enums, .exec_(), QAction/QShortcut re-exports), so most call sites needed no change. 2. The handful of real PyQt6 breakers that qtpy does NOT shim: - QDesktopWidget (removed in Qt6) -> QApplication.primaryScreen() in control/gui_hcs.py and control/core_volumetric_imaging.py. - QVariant (removed in PyQt6): dropped the now-dead isinstance(x, QVariant) unwrap guards in control/core/core.py and control/gui_hcs.py (PyQt6 hands back plain Python objects). - QComboBox.activated[str] overload removed in Qt6 -> use the dedicated `textActivated` signal in control/widgets.py. - tools/view_laser_af_reference_image.py: converted its direct PyQt5 imports to qtpy and matplotlib backend_qt5agg -> backend_qtagg (binding-agnostic). 3. Shared OpenGL context for the multiple embedded napari viewers. main_hcs.py now sets AA_ShareOpenGLContexts before the QApplication is constructed. Several vispy/OpenGL canvases in one process must share a GL context; without it, rendering fails with "Cannot SIZE object N ..." / GL_INVALID_FRAMEBUFFER_OPERATION. 4. Single Qt binding requirement + setup scripts. The blank-canvas rendering seen during bring-up was root-caused to PyQt5 and PyQt6 being installed side by side (conflicting Qt libraries break napari's OpenGL rendering). setup_22.04.sh and setup_cuda_22.04.sh now install exactly one binding: PyQt6 (+ PyQt6-Qt6, PyQt6-sip) via pip, remove any pre-existing PyQt5, move pyqtgraph to pip (apt's python3-pyqtgraph drags in python3-pyqt5), bump napari to >=0.7,<0.8, and drop the "numpy<2" pin (napari 0.7 needs numpy>=2). Comments explain the one-binding rule so it isn't reintroduced. 5. Mosaic view: defer GL redraws while its tab is hidden (control/widgets_mosaic.py). Under PyQt6, painting a napari/vispy canvas that lives on a hidden QTabWidget page fails because the hidden QOpenGLWidget has an incomplete framebuffer, which corrupts the shared GL program and cascades into repeated "Cannot SIZE object N because it does not exist" errors during acquisition. The widget now writes tile pixel data unconditionally (so on-disk saves stay correct) but skips the GL refresh/reset/fit calls while hidden, records a pending-refresh flag, and flushes it in showEvent once the tab is visible. This makes live-during-acquisition mosaic updates work regardless of which tab is in front. TESTING ------- - Verified against the live dev environment: Python 3.14, napari 0.7.0, PyQt6 6.11, numpy 2.4.4, pydantic 2.13.4, qtpy 2.4.3, vispy 0.16.1. - qtpy resolves to PyQt6; full test suite collects cleanly (1435 tests). - GUI/pytest-qt suites pass, including tests/control/test_unified_mosaic_widget.py (its fixture builds a real napari.Viewer). The pre-existing full-suite single-process teardown crash (napari/vispy OpenGL shutdown) is unrelated to the binding change; run napari-heavy test files separately. - App launches and the Mosaic/Live/Multichannel napari views render; a full plate acquisition completes and the mosaic populates in both Full View and Plate View. KNOWN FOLLOW-UPS (pre-existing, not introduced here) ---------------------------------------------------- - Memory: full-view mosaic of a multi-well plate builds a large in-memory canvas (~2GB+), which napari displays at ~4x overhead, and the end-of-run save copies the whole canvas; back-to-back runs can stack these. Not an unbounded leak (canvas is reused), but peak RAM is high and causes GUI-thread stalls. Candidate fixes: bound the save-snapshot copy, pre-allocate the full-view canvas, optionally coarser full-view render resolution. - A napari 0.7 / vispy 0.16 layer-teardown ValueError ("list.remove(x): x not in list") can fire when clearing layers on a mode switch. Needs a guard. Co-Authored-By: Claude Opus 4.8 (1M context) --- software/Python 3.14 Compatibility Check.md | 66 +++++++++++++++++++ software/control/console.py | 2 +- software/control/core/core.py | 5 +- software/control/core_PDAF.py | 2 +- .../control/core_displacement_measurement.py | 2 +- software/control/core_usbspectrometer.py | 2 +- software/control/core_volumetric_imaging.py | 6 +- software/control/gui_hcs.py | 8 +-- software/control/single_instance.py | 2 +- software/control/widgets.py | 4 +- software/control/widgets_mosaic.py | 63 ++++++++++++++++-- software/control/widgets_usbspectrometer.py | 2 +- software/main_hcs.py | 11 +++- software/scripts/run_acquisition.py | 2 +- software/setup_22.04.sh | 24 ++++--- software/setup_cuda_22.04.sh | 2 - .../tests/control/test_single_instance.py | 2 +- .../tools/view_laser_af_reference_image.py | 10 ++- 18 files changed, 171 insertions(+), 44 deletions(-) create mode 100644 software/Python 3.14 Compatibility Check.md diff --git a/software/Python 3.14 Compatibility Check.md b/software/Python 3.14 Compatibility Check.md new file mode 100644 index 000000000..33804f13a --- /dev/null +++ b/software/Python 3.14 Compatibility Check.md @@ -0,0 +1,66 @@ +# Python 3.14 Compatibility -- Squid Dependencies + +## Supported (confirmed) +| Package | Notes | +|---------|-------| +| numpy | Wheels for 3.14 since v2.3+ | +| scipy | Supported | +| pandas | Supported | +| pydantic (v2) | Supported -- but **pydantic v1** (used by old napari) is **NOT** | +| Pillow | Supported | +| matplotlib | Supported | +| PyYAML | Supported | +| filelock | Supported | +| platformdirs | Supported | +| scikit-image | Wheels for 3.14 since v0.26 | +| napari (latest) | Supported -- requires PyQt6, not PyQt5 | +| opencv-python-headless | Uses stable ABI (cp37-abi3), works | +| opencv-contrib-python-headless | Same stable ABI, works | +| torch (PyTorch) | Supported since ~2.13 | +| dask | Active releases in 2026 | +| zarr | Supported | +| tifffile | Supported | +| imageio | Supported | +| pyqtgraph | Pure Python, likely works | +| qtpy | Pure Python abstraction layer, works | + +## Blocked / Incompatible +| Package | Issue | +|---------|-------| +| **PyQt5** | **No Python 3.14 support -- this is the main blocker** | +| **pydantic v1** | Explicitly incompatible with 3.14 (the `__slots__` error) | + +## Uncertain / Likely OK but unconfirmed +| Package | Notes | +|---------|-------| +| pyserial | Pure Python, last release 2020 -- probably works but untested | +| pyvisa | Likely works | +| hidapi | Cython extension -- no 3.14 wheels confirmed | +| GitPython | Marked as NOT supporting 3.14 on pyreadiness.org | +| aicsimageio | Unclear, niche package | +| basicpy | Unclear | +| ome-zarr | Unclear | +| pydantic-xml | Likely works if pydantic v2 is used | +| pyreadline3 | Windows-specific, unclear | +| qtconsole | Depends on Qt backend | +| lxml / lxml_html_clean | C extension -- needs checking | +| mcp | Unclear | +| Hardware drivers (PySpin, pyAndorSDK3, ids_peak, pyvcam, pm16) | Vendor-specific, unlikely to have 3.14 builds yet | + +## Bottom Line + +The **two hard blockers** are **PyQt5** and **pydantic v1** (via old napari). Migrating to Python 3.14 would require: +1. Switching from PyQt5 to PyQt6 (non-trivial API changes) +2. Upgrading napari to a version that uses pydantic v2 +3. Verifying all hardware vendor SDKs have 3.14 builds + +**Recommendation: Stay on Python 3.10** for production use. It's the sweet spot where everything works. + +## Sources +- [Python 3.14 Readiness](https://pyreadiness.org/3.14/) +- [NumPy 2.3.0 Release Notes](https://numpy.org/devdocs/release/2.3.0-notes.html) +- [PyQt5 on PyPI](https://pypi.org/project/PyQt5/) +- [napari Installation](https://napari.org/stable/getting_started/installation.html) +- [PyTorch torch.compile Python 3.14 support](https://dev-discuss.pytorch.org/t/torch-compile-support-for-python-3-14-completed/3276) +- [Pydantic v1 Python 3.14 issue](https://github.com/pydantic/pydantic/issues/12618) +- [Anaconda Python 3.14 overview](https://www.anaconda.com/blog/python-3-14-what-data-scientists-developers-need-know) diff --git a/software/control/console.py b/software/control/console.py index fed32978e..9accbb423 100644 --- a/software/control/console.py +++ b/software/control/console.py @@ -1,7 +1,7 @@ import os # set QT_API environment variable -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy from qtpy.QtWidgets import QApplication, QMainWindow, QWidget, QVBoxLayout, QPushButton, QLabel diff --git a/software/control/core/core.py b/software/control/core/core.py index 64f030f04..e1e045748 100644 --- a/software/control/core/core.py +++ b/software/control/core/core.py @@ -4,7 +4,7 @@ import tempfile # qt libraries -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy import pyqtgraph as pg from qtpy.QtCore import * @@ -1677,9 +1677,6 @@ def update_wellplate_settings( rows, cols, ): - if isinstance(sample_format, QVariant): - sample_format = sample_format.value() - if sample_format == "glass slide": if IS_HCS: sample = "4 glass slide" diff --git a/software/control/core_PDAF.py b/software/control/core_PDAF.py index 935955b81..f89e7f6d8 100644 --- a/software/control/core_PDAF.py +++ b/software/control/core_PDAF.py @@ -1,7 +1,7 @@ # set QT_API environment variable import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" # qt libraries from qtpy.QtCore import * diff --git a/software/control/core_displacement_measurement.py b/software/control/core_displacement_measurement.py index c86794d3b..55336f55c 100644 --- a/software/control/core_displacement_measurement.py +++ b/software/control/core_displacement_measurement.py @@ -1,7 +1,7 @@ # set QT_API environment variable import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy # qt libraries diff --git a/software/control/core_usbspectrometer.py b/software/control/core_usbspectrometer.py index e67f2bd31..733065ec7 100644 --- a/software/control/core_usbspectrometer.py +++ b/software/control/core_usbspectrometer.py @@ -1,7 +1,7 @@ # set QT_API environment variable import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy # qt libraries diff --git a/software/control/core_volumetric_imaging.py b/software/control/core_volumetric_imaging.py index 9c15c81aa..ef9689522 100644 --- a/software/control/core_volumetric_imaging.py +++ b/software/control/core_volumetric_imaging.py @@ -1,7 +1,7 @@ # set QT_API environment variable import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy # qt libraries @@ -173,8 +173,8 @@ def __init__(self, window_title=""): self.setCentralWidget(self.widget) # set window size - desktopWidget = QDesktopWidget() - width = min(desktopWidget.height() * 0.9, 1000) # @@@TO MOVE@@@# + screen_height = QApplication.primaryScreen().size().height() + width = int(min(screen_height * 0.9, 1000)) # @@@TO MOVE@@@# height = width self.setFixedSize(width, height) diff --git a/software/control/gui_hcs.py b/software/control/gui_hcs.py index 04222a107..282d65f15 100644 --- a/software/control/gui_hcs.py +++ b/software/control/gui_hcs.py @@ -15,7 +15,7 @@ ClearedScanCoordinates, ) -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import re import time from enum import Enum, auto @@ -1426,8 +1426,7 @@ def _getMainWindowMinimumSize(self): We want our main window to fit on the primary screen, so grab the users primary screen and return something slightly smaller than that. """ - desktop_info = QDesktopWidget() - primary_screen_size = desktop_info.screen(desktop_info.primaryScreen()).size() + primary_screen_size = QApplication.primaryScreen().size() height_min = int(0.9 * primary_screen_size.height()) width_min = int(0.96 * primary_screen_size.width()) @@ -2386,9 +2385,6 @@ def onDisplayTabChanged(self, index): self.toggleWellSelector(False) def onWellplateChanged(self, format_): - if isinstance(format_, QVariant): - format_ = format_.value() - # TODO(imo): Not sure why glass slide is so special here? It seems like it's just a "1 well plate". if format_ == "glass slide": self.toggleWellSelector(False) diff --git a/software/control/single_instance.py b/software/control/single_instance.py index 309e2c896..60f2eee7c 100644 --- a/software/control/single_instance.py +++ b/software/control/single_instance.py @@ -11,7 +11,7 @@ # Pin the Qt binding before importing qtpy, matching the convention used by # every other Qt-using module in this codebase. -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import getpass from typing import NamedTuple, Optional diff --git a/software/control/widgets.py b/software/control/widgets.py index 440883a71..0109fb798 100644 --- a/software/control/widgets.py +++ b/software/control/widgets.py @@ -31,7 +31,7 @@ from squid.config import CameraPixelFormat # set QT_API environment variable -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" # qt libraries import qtpy @@ -4170,7 +4170,7 @@ def add_components(self, show_trigger_options, show_display_options, show_autole self.entry_displayFPS.valueChanged.connect(self.streamHandler.set_display_fps) self.slider_resolutionScaling.valueChanged.connect(self.streamHandler.set_display_resolution_scaling) self.slider_resolutionScaling.valueChanged.connect(self.liveController.set_display_resolution_scaling) - self.dropdown_modeSelection.activated[str].connect(self.select_new_microscope_mode_by_name) + self.dropdown_modeSelection.textActivated.connect(self.select_new_microscope_mode_by_name) self.dropdown_triggerManu.currentIndexChanged.connect(self.update_trigger_mode) self.btn_live.clicked.connect(self.toggle_live) self.entry_exposureTime.valueChanged.connect(self.update_config_exposure_time) diff --git a/software/control/widgets_mosaic.py b/software/control/widgets_mosaic.py index 17a6ae662..068e5d9aa 100644 --- a/software/control/widgets_mosaic.py +++ b/software/control/widgets_mosaic.py @@ -135,6 +135,13 @@ def __init__(self, objectiveStore, camera, contrastManager, parent=None): self.mode = _load_last_view_mode() self.layers_initialized = False + # Deferred-redraw flag. Painting this napari/vispy canvas while its tab is + # hidden fails on PyQt6 (the hidden QOpenGLWidget has an incomplete + # framebuffer → GL_INVALID_FRAMEBUFFER_OPERATION, which corrupts the shared + # GL program and cascades into "Cannot SIZE object N" on later frames). Tile + # numpy data is still written while hidden (so saves stay correct); only the + # GL refresh is deferred and flushed in showEvent once the tab is visible. + self._pending_refresh = False self.mosaic_dtype = None self.viewer_pixel_size_mm = None @@ -282,12 +289,39 @@ def setPlateLayout(self, plate_view_init): if PLATE_BOUNDARIES_LAYER in self.viewer.layers: self.viewer.layers.remove(self.viewer.layers[PLATE_BOUNDARIES_LAYER]) if canvas_changed and self.mode == DisplayMode.PLATE: - self._fit_view_to_plate() + if self._display_active(): + self._fit_view_to_plate() + else: + self._pending_refresh = True def _image_layers(self): """Iterate napari image layers, skipping shape/boundary overlays.""" return [lyr for lyr in self.viewer.layers if lyr.name not in NON_IMAGE_LAYERS and hasattr(lyr, "data")] + def _display_active(self) -> bool: + """Whether the mosaic canvas is on-screen and safe to repaint. + + A napari/vispy canvas embedded in a hidden QTabWidget page cannot be + painted on PyQt6 (the hidden QOpenGLWidget has no valid framebuffer), so + callers defer GL refreshes to showEvent when this returns False. + """ + return self.isVisible() + + def showEvent(self, event): + """Flush any GL refreshes that were deferred while the tab was hidden. + + Tile data written while hidden is already in each layer's array; here we + repaint once the framebuffer is valid so the tab shows the latest mosaic. + """ + super().showEvent(event) + if self._pending_refresh: + self._pending_refresh = False + for lyr in self._image_layers(): + lyr.refresh() + if self.mode == DisplayMode.PLATE: + self._draw_plate_boundaries() + self.resetView() + def enable_shape_drawing(self, enable): """Set Manual-ROI drawing on/off. Idempotent: the upstream signal only emits True on entry to Manual mode (never False on exit), so @@ -535,8 +569,11 @@ def updateTile(self, update): layer = self.viewer.layers[channel_name] blit_tiles_to_canvas(layer.data, [(image, y_px, x_px)]) - layer.refresh() - self._draw_plate_boundaries() + if self._display_active(): + layer.refresh() + self._draw_plate_boundaries() + else: + self._pending_refresh = True # The fit-the-whole-plate camera reset only fires when the canvas # is (re)allocated — in setPlateLayout when slot dims/coverage # change and in _create_channel_layer when a layer is created. @@ -556,6 +593,8 @@ def _update_mosaic_layer(self, layer, image, tl_x_mm, tl_y_mm, prev_top_left): mosaic_height = int(math.ceil((self.viewer_extents[1] - self.viewer_extents[0]) / self.viewer_pixel_size_mm)) mosaic_width = int(math.ceil((self.viewer_extents[3] - self.viewer_extents[2]) / self.viewer_pixel_size_mm)) + display_active = self._display_active() + if layer.data.shape[:2] != (mosaic_height, mosaic_width): y_offset = int(math.floor((prev_top_left[0] - self.top_left_coordinate[0]) / self.viewer_pixel_size_mm)) x_offset = int(math.floor((prev_top_left[1] - self.top_left_coordinate[1]) / self.viewer_pixel_size_mm)) @@ -567,14 +606,20 @@ def _update_mosaic_layer(self, layer, image, tl_x_mm, tl_y_mm, prev_top_left): x_end = min(x_offset + lyr.data.shape[1], new_data.shape[1]) new_data[y_offset:y_end, x_offset:x_end] = lyr.data[: y_end - y_offset, : x_end - x_offset] lyr.data = new_data - self.resetView() # Keep ROI vertices anchored to their stage-coordinate positions after the shift. self._update_shape_layer_position() + if display_active: + self.resetView() y_pos = int(math.floor((tl_y_mm - self.top_left_coordinate[0]) / self.viewer_pixel_size_mm)) x_pos = int(math.floor((tl_x_mm - self.top_left_coordinate[1]) / self.viewer_pixel_size_mm)) blit_tiles_to_canvas(layer.data, [(image, y_pos, x_pos)]) - layer.refresh() + # Painting a hidden tab's canvas fails on PyQt6 (see _display_active); defer + # the GL refresh to showEvent. The tile is already in layer.data either way. + if display_active: + layer.refresh() + else: + self._pending_refresh = True def _create_channel_layer(self, channel_name, reference_image): """Create a new napari image layer for a channel. @@ -613,9 +658,13 @@ def _create_channel_layer(self, channel_name, reference_image): layer.mouse_double_click_callbacks.append(self._on_double_click) # Fit the view when the first plate-sized canvas is created so the user # immediately sees the full plate; subsequent tiles preserve any - # pan/zoom they've made since. + # pan/zoom they've made since. Deferred to showEvent when the tab is + # hidden — painting a hidden canvas fails on PyQt6 (see _display_active). if self.mode == DisplayMode.PLATE and self.num_rows > 0 and self.num_cols > 0: - self._fit_view_to_plate() + if self._display_active(): + self._fit_view_to_plate() + else: + self._pending_refresh = True def _convert_image_dtype(self, image, target_dtype): """Convert image to target dtype with range scaling.""" diff --git a/software/control/widgets_usbspectrometer.py b/software/control/widgets_usbspectrometer.py index 358348eb6..e5ff9c234 100644 --- a/software/control/widgets_usbspectrometer.py +++ b/software/control/widgets_usbspectrometer.py @@ -1,7 +1,7 @@ # set QT_API environment variable import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import qtpy # qt libraries diff --git a/software/main_hcs.py b/software/main_hcs.py index dddc81816..412f3a766 100644 --- a/software/main_hcs.py +++ b/software/main_hcs.py @@ -3,11 +3,12 @@ import logging import os -os.environ["QT_API"] = "pyqt5" +os.environ["QT_API"] = "pyqt6" import signal import sys # qt libraries +from qtpy.QtCore import Qt from qtpy.QtWidgets import * from qtpy.QtGui import * @@ -54,6 +55,14 @@ ) args = parser.parse_args() + # This GUI embeds several napari viewers (Live, Multichannel, Mosaic), each + # backed by its own vispy/OpenGL canvas, in a single process. Multiple vispy + # canvases must share one OpenGL context or vispy misroutes GLIR draw commands + # between them and rendering fails with "Cannot SIZE object N because it does + # not exist" (the mosaic image never appears). This attribute MUST be set + # before the QApplication is constructed. See napari's multiple-viewer example. + QApplication.setAttribute(Qt.ApplicationAttribute.AA_ShareOpenGLContexts, True) + # Construct QApplication first so the single-instance check can show a # QMessageBox before any other startup side effects (logging, migration). app = QApplication(["Squid"]) diff --git a/software/scripts/run_acquisition.py b/software/scripts/run_acquisition.py index aa5f824ca..47e90a79b 100755 --- a/software/scripts/run_acquisition.py +++ b/software/scripts/run_acquisition.py @@ -113,7 +113,7 @@ def launch_gui(simulation: bool = False, verbose: bool = False) -> subprocess.Po cmd.append("--verbose") env = os.environ.copy() - env["QT_API"] = "pyqt5" + env["QT_API"] = "pyqt6" if verbose: print(f"Launching GUI: {' '.join(cmd)}") diff --git a/software/setup_22.04.sh b/software/setup_22.04.sh index 2899416cb..d3aa6f756 100755 --- a/software/setup_22.04.sh +++ b/software/setup_22.04.sh @@ -37,8 +37,11 @@ sudo apt update # install packages sudo apt install python3-pip -y -sudo apt install python3-pyqtgraph python3-pyqt5 -y -sudo apt install python3-pyqt5.qtsvg +# NOTE: do NOT install apt's python3-pyqtgraph / python3-pyqt5 here. python3-pyqtgraph +# pulls in python3-pyqt5 as a dependency, and having BOTH PyQt5 and PyQt6 present in the +# same environment causes conflicting Qt libraries that break napari's OpenGL rendering +# (blank/failed canvases, "Cannot SIZE object N because it does not exist"). We install +# pyqtgraph and PyQt6 via pip below so exactly one Qt binding is present. sudo apt-get install git -y ## clone the repo if we don't already have it. @@ -54,15 +57,20 @@ fi cd "$SQUID_SOFTWARE_ROOT" mkdir -p "$SQUID_SOFTWARE_ROOT/cache" -# Ubuntu 22.04 ships pip 22.0.2, whose resolver can't handle the -# napari==0.5.4 dependency graph on current PyPI (hits ResolutionTooDeep -# after hours of backtracking). Upgrade pip before installing libraries. +# Ubuntu 22.04 ships an old pip; upgrade it before resolving the dependency graph. python3 -m pip install --upgrade pip -# install libraries -pip3 install qtpy pyserial pandas imageio crc==1.3.0 lxml "numpy<2" tifffile scipy pyreadline3 +# Qt binding: napari 0.7 requires PyQt6. Exactly ONE Qt binding may be installed — +# PyQt5 and PyQt6 in the same environment conflict and break napari/vispy OpenGL +# rendering. Remove any pre-existing PyQt5 (e.g. pulled in by an apt package) first. +sudo apt remove -y python3-pyqt5 python3-pyqt5.qtsvg 2>/dev/null || true +pip3 uninstall -y PyQt5 PyQt5-Qt5 PyQt5-sip 2>/dev/null || true +pip3 install PyQt6 PyQt6-Qt6 PyQt6-sip + +# install libraries. napari 0.7 requires numpy>=2, so no "numpy<2" pin here. +pip3 install pyqtgraph qtpy pyserial pandas imageio crc==1.3.0 lxml numpy tifffile scipy pyreadline3 pip3 install opencv-python-headless opencv-contrib-python-headless -pip3 install napari==0.5.4 scikit-image dask_image ome_zarr aicsimageio basicpy pytest pytest-qt pytest-xvfb gitpython matplotlib pydantic_xml pyvisa hidapi filelock lxml_html_clean psutil mcp ndv +pip3 install "napari>=0.7,<0.8" scikit-image dask_image ome_zarr aicsimageio basicpy pytest pytest-qt pytest-xvfb gitpython matplotlib pydantic_xml pyvisa hidapi filelock lxml_html_clean psutil mcp ndv # Optional: PI V-308 / C-414 focus stage (USE_PI_FOCUS_STAGE). Safe to skip if unused; # squid.stage.pi imports it lazily and only needs it to connect to real hardware, so diff --git a/software/setup_cuda_22.04.sh b/software/setup_cuda_22.04.sh index 744763fb3..b0047d2ba 100755 --- a/software/setup_cuda_22.04.sh +++ b/software/setup_cuda_22.04.sh @@ -10,6 +10,4 @@ pip install cuda-python pip install cupy-cuda12x pip3 install torch torchvision torchaudio -pip3 install "numpy<2" - sudo apt autoremove -y diff --git a/software/tests/control/test_single_instance.py b/software/tests/control/test_single_instance.py index 4a05435ff..e021feb81 100644 --- a/software/tests/control/test_single_instance.py +++ b/software/tests/control/test_single_instance.py @@ -71,7 +71,7 @@ def test_stale_lock_from_dead_process_is_reclaimed(tmp_path): # Pin QT_API so qtpy in the child selects the same binding as the parent # regardless of what other Qt bindings are installed in the environment. - child_env = {**os.environ, "QT_API": "pyqt5"} + child_env = {**os.environ, "QT_API": "pyqt6"} result = subprocess.run( [sys.executable, "-c", child_script], capture_output=True, diff --git a/software/tools/view_laser_af_reference_image.py b/software/tools/view_laser_af_reference_image.py index fe6507c09..ae944235a 100644 --- a/software/tools/view_laser_af_reference_image.py +++ b/software/tools/view_laser_af_reference_image.py @@ -1,11 +1,15 @@ +import os + +os.environ["QT_API"] = "pyqt6" + import sys import json import base64 import numpy as np -from PyQt5.QtWidgets import QApplication, QMainWindow, QLabel, QVBoxLayout, QWidget, QPushButton, QFileDialog -from PyQt5.QtCore import Qt +from qtpy.QtWidgets import QApplication, QMainWindow, QLabel, QVBoxLayout, QWidget, QPushButton, QFileDialog +from qtpy.QtCore import Qt import matplotlib.pyplot as plt -from matplotlib.backends.backend_qt5agg import FigureCanvasQTAgg +from matplotlib.backends.backend_qtagg import FigureCanvasQTAgg try: import yaml From 067c684e2fda4748b82b769996cfb376e35b7098 Mon Sep 17 00:00:00 2001 From: Spencer Schwarz Date: Thu, 23 Jul 2026 02:42:21 -0400 Subject: [PATCH 2/2] Performance mode: defer mosaic rendering during acquisition, render once at completion MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit WHY --- With the Mosaic view active, rendering every tile into the napari canvas during a multi-well acquisition stalls the GUI thread (repeated GL texture rebuilds and canvas reallocations), so the run visibly stutters and slows as it progresses. See "Mosaic Memory Diagnosis.md" for the full analysis. Previously "Performance Mode" avoided this by disconnecting the mosaic/multichannel data feeds entirely — the run stayed fast, but the mosaic was never populated (the image data only existed in the saved files on disk). This changes Performance Mode so the mosaic still assembles during the run but without per-tile rendering, then renders the finished mosaic once when the run completes — you get the speed of not rendering mid-run and still see the result. WHAT CHANGED (control/gui_hcs.py) --------------------------------- 1. updateNapariConnections: in performance mode the mosaic's data feed (mosaic_tile_update -> updateTile, plate_view_init -> setPlateLayout) now stays CONNECTED, so its canvas keeps building during acquisition. Rendering is what gets suppressed: UnifiedMosaicWidget already defers all GL refresh/reset/fit work while its tab is hidden (see UnifiedMosaicWidget.showEvent / _pending_refresh), and performance mode keeps that tab hidden for the whole run. The remaining napari views (e.g. multichannel) are still disconnected in performance mode as before. 2. toggleAcquisitionStart: the start/finish hook now manages the heavy tabs in performance mode. - start: re-apply toggleNapariTabs() so the mosaic/multichannel tabs are hidden and disabled for the duration of the run (a previous run's completion may have re-enabled them), guaranteeing rendering stays deferred. - finish: re-enable the napari tabs and switch to the Mosaic view; becoming visible fires showEvent, which flushes the single deferred refresh — the one-shot render of the assembled mosaic. TEST ---- tests/control/test_performance_mode.py (new): builds the simulated GUI with the mosaic view forced on, enters performance mode, and asserts the mosaic feed remains connected, the mosaic tab is disabled during the run, and is re-enabled + made current at completion. It lives in its own module on purpose: forcing the mosaic napari.Viewer on alongside the other full-GUI tests accumulates enough napari/vispy viewers to trigger the known STATUS_HEAP_CORRUPTION on OpenGL teardown at process exit (documented; run napari-heavy GUI test files separately). Split out, it and test_HighContentScreeningGui.py each exit cleanly. NOTES / TRADE-OFFS ------------------ - Performance mode now builds the full mosaic canvas in RAM during the run (same peak as normal mode), trading the old mode's lower memory for a usable final render and no mid-run stalls. Reducing that peak is separate follow-up work (see "Mosaic Memory Diagnosis.md"). - The acquisition RAM pre-check is still skipped in performance mode (check_ram_available_with_error_dialog(..., performance_mode=...)); since the canvas is now built, re-enabling that check for performance mode is a reasonable follow-up (left unchanged here to avoid scope creep and an existing test that asserts the skip). Co-Authored-By: Claude Opus 4.8 (1M context) --- software/control/gui_hcs.py | 69 +++++++++++------ .../tests/control/test_performance_mode.py | 74 +++++++++++++++++++ 2 files changed, 121 insertions(+), 22 deletions(-) create mode 100644 software/tests/control/test_performance_mode.py diff --git a/software/control/gui_hcs.py b/software/control/gui_hcs.py index 282d65f15..fde0a10da 100644 --- a/software/control/gui_hcs.py +++ b/software/control/gui_hcs.py @@ -1818,33 +1818,42 @@ def makeNapariConnections(self): self.updateNapariConnections() def updateNapariConnections(self): - # Update Napari connections based on performance mode. Live widget connections are preserved + # Update Napari connections based on performance mode. # Connection tuples can be: # (signal, slot) - uses default Qt.AutoConnection # (signal, slot, connection_type) - uses specified connection type (e.g., Qt.QueuedConnection) + # + # The live widget is always kept connected. In performance mode the mosaic's + # data feed is ALSO kept connected: its canvas still builds during acquisition, + # but rendering is deferred while its tab is hidden and flushed once when the run + # finishes (see UnifiedMosaicWidget.showEvent and toggleAcquisitionStart). Only + # the remaining napari views (e.g. multichannel) are disconnected in performance mode. for widget_name, connections in self.napari_connections.items(): - if widget_name != "napariLiveWidget": # Always keep the live widget connected - widget = getattr(self, widget_name, None) - if widget: - for conn in connections: - signal = conn[0] - slot = conn[1] - connection_type = conn[2] if len(conn) > 2 else None - if self.performance_mode: - try: - signal.disconnect(slot) - except TypeError: - # Connection might not exist, which is fine - pass + if widget_name == "napariLiveWidget": # always kept connected (wired at init) + continue + keep_connected = (not self.performance_mode) or widget_name == "unifiedMosaicWidget" + widget = getattr(self, widget_name, None) + if not widget: + continue + for conn in connections: + signal = conn[0] + slot = conn[1] + connection_type = conn[2] if len(conn) > 2 else None + if keep_connected: + try: + if connection_type is not None: + signal.connect(slot, connection_type) else: - try: - if connection_type is not None: - signal.connect(slot, connection_type) - else: - signal.connect(slot) - except TypeError: - # Connection might already exist, which is fine - pass + signal.connect(slot) + except TypeError: + # Connection might already exist, which is fine + pass + else: + try: + signal.disconnect(slot) + except TypeError: + # Connection might not exist, which is fine + pass def toggleNapariTabs(self): # Enable/disable Napari tabs based on performance mode @@ -2497,6 +2506,13 @@ def toggleAcquisitionStart(self, acquisition_started): self.live_scan_grid_was_on = False # NOTE: RAM monitor widget is connected via multipointController.signal_acquisition_start # which fires AFTER the memory monitor is created (see make_connections) + + # Performance mode: keep the mosaic/multichannel napari tabs hidden and + # disabled for the duration of the run so the mosaic canvas builds without + # rendering (deferred), avoiding the per-tile GL stalls. A previous run's + # completion may have re-enabled/shown them, so re-apply the hidden state here. + if self.performance_mode: + self.toggleNapariTabs() else: self.log.info("FINISHED ACQUISITION") if self.live_scan_grid_was_on: @@ -2504,6 +2520,15 @@ def toggleAcquisitionStart(self, acquisition_started): self.live_scan_grid_was_on = False # NOTE: RAM monitor widget is disconnected via multipointController.acquisition_finished + # Performance mode: render the assembled mosaic once, now that the run is + # done. Re-enable the napari tabs and switch to the mosaic view; becoming + # visible flushes the single deferred refresh (UnifiedMosaicWidget.showEvent). + if self.performance_mode: + for i in range(self.imageDisplayTabs.count()): + self.imageDisplayTabs.setTabEnabled(i, True) + if self.unifiedMosaicWidget is not None: + self.imageDisplayTabs.setCurrentWidget(self.unifiedMosaicWidget) + # click to move off during acquisition self.navigationWidget.set_click_to_move(not acquisition_started) diff --git a/software/tests/control/test_performance_mode.py b/software/tests/control/test_performance_mode.py new file mode 100644 index 000000000..07d0f19f0 --- /dev/null +++ b/software/tests/control/test_performance_mode.py @@ -0,0 +1,74 @@ +import pytest + +import control.gui_hcs +import control.microscope +from qtpy.QtWidgets import QMessageBox + + +@pytest.fixture +def confirm_exit_yes(monkeypatch): + """Auto-accept the 'Confirm Exit' dialog GUI shutdown shows, or teardown hangs forever.""" + + def confirm_exit(parent, title, text, *args, **kwargs): + if title == "Confirm Exit": + return QMessageBox.Yes + raise RuntimeError(f"Unexpected QMessageBox: {title} - {text}") + + monkeypatch.setattr(QMessageBox, "question", confirm_exit) + + +def test_performance_mode_defers_mosaic_and_renders_at_completion(qtbot, confirm_exit_yes, monkeypatch): + """Performance mode keeps the mosaic's data feed connected (so its canvas still + builds during acquisition) but holds its tab hidden/disabled so rendering is + deferred; on completion the mosaic tab is re-enabled and shown so the single + deferred refresh flushes. + + Lives in its own module because it forces the mosaic napari.Viewer on; grouping + it with the other full-GUI tests accumulates enough napari/vispy viewers to hit + the known STATUS_HEAP_CORRUPTION on OpenGL teardown at process exit (documented; + run napari-heavy GUI test files separately). See gui_hcs.updateNapariConnections + and gui_hcs.toggleAcquisitionStart. + """ + # gui_hcs star-imports _def; force the mosaic view on before construction so the + # widget exists regardless of this machine's cached [VIEWS] config. + monkeypatch.setattr(control.gui_hcs, "USE_NAPARI_FOR_MOSAIC_DISPLAY", True) + + scope = control.microscope.Microscope.build_from_global_config(True) + win = control.gui_hcs.HighContentScreeningGui(microscope=scope, is_simulation=True) + qtbot.add_widget(win) + + mosaic = win.unifiedMosaicWidget + assert mosaic is not None, "mosaic widget should be created when USE_NAPARI_FOR_MOSAIC_DISPLAY is on" + mosaic_idx = win.imageDisplayTabs.indexOf(mosaic) + + def mosaic_feed_connected(): + """True if the controller's mosaic_tile_update is still wired to updateTile. + Probes by disconnect (raises TypeError if not connected) then restores.""" + try: + win.multipointController.mosaic_tile_update.disconnect(mosaic.updateTile) + except TypeError: + return False + win.multipointController.mosaic_tile_update.connect(mosaic.updateTile) + return True + + assert mosaic_feed_connected(), "mosaic feed should be connected in normal mode" + + # Enter performance mode via the toggle-button path. + win.performanceModeToggle.setChecked(True) + win.togglePerformanceMode() + assert win.performance_mode is True + + # Key change: the mosaic's data feed stays CONNECTED in performance mode (so its + # canvas still builds during acquisition; rendering is what gets deferred). + assert mosaic_feed_connected(), "mosaic feed must remain connected in performance mode" + + # Start of run: the mosaic tab is hidden/disabled so rendering defers (no per-tile GL). + win.toggleAcquisitionStart(True) + assert win.imageDisplayTabs.isTabEnabled(mosaic_idx) is False + + # Completion: the mosaic tab is re-enabled and made current so its showEvent + # flushes the single deferred render. + win.toggleAcquisitionStart(False) + qtbot.wait(20) + assert win.imageDisplayTabs.isTabEnabled(mosaic_idx) is True + assert win.imageDisplayTabs.currentWidget() is mosaic