Skip to content

Commit 402858b

Browse files
author
tester
committed
Add imageio-native-vips backend and backend priority system
New module: imageio-native-vips - Panama FFM downcalls to libvips for cross-platform image decoding - Supports HEIC, AVIF, WebP, JP2, PDF, SVG, EXR, FITS, Netpbm, HDR - Hardcoded format list in SPI, runtime probe via vips_foreign_find_load_buffer - Decode pipeline: colourspace(sRGB) -> premultiply -> cast_uchar -> RGBA->ARGB repack - Path-based fast path (zero Java heap copy) via vips_image_new_from_file - Library discovery: MacPorts, Homebrew, Linux paths, system property override - 17 tests covering canDecode, getSize, decode, quadrant color verification - Not included in imageio-native aggregator (opt-in only) - Respects imageio.native.formats supplemental mode Backend priority system (BackendPriority in imageio-native-common): - System property imageio.native.backend.priority for global ordering - Per-format overrides via imageio.native.backend.priority.<format> - Default: native first, then vips, then magick - NativeImageReaderSpi uses BackendPriority in onRegistration() for SPI ordering - Backward compatible: no properties set = identical to previous behavior
1 parent 58e4413 commit 402858b

File tree

13 files changed

+1258
-9
lines changed

13 files changed

+1258
-9
lines changed

‎README.md‎

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,50 @@ Class.forName("io.github.ghosthack.imageio.apple.AppleImageReaderSpi");
111111
Class.forName("io.github.ghosthack.imageio.windows.WicImageReaderSpi");
112112
```
113113

114+
## Optional backends
115+
116+
The `imageio-native-vips` module is an optional backend that delegates to [libvips](https://www.libvips.org/) for image decoding. It is **not** included in the `imageio-native` aggregator -- add it explicitly to opt in.
117+
118+
```xml
119+
<dependency>
120+
<groupId>io.github.ghosthack</groupId>
121+
<artifactId>imageio-native-vips</artifactId>
122+
<version>1.0.2</version>
123+
</dependency>
124+
```
125+
126+
Requires libvips installed on the system:
127+
128+
```sh
129+
# macOS (MacPorts)
130+
sudo port install vips
131+
132+
# macOS (Homebrew)
133+
brew install vips
134+
135+
# Debian/Ubuntu
136+
sudo apt install libvips-dev
137+
```
138+
139+
The vips backend adds cross-platform support (macOS, Linux, Windows) for HEIC, AVIF, WebP, JPEG 2000, PDF, SVG, EXR, FITS, Netpbm, HDR, and more -- depending on the libvips build configuration. It respects the `imageio.native.formats` property (supplemental mode by default).
140+
141+
The SPI declares a fixed set of common formats. Formats not in the list but supported by the installed libvips can still be decoded via the direct `VipsNative` API -- they just won't be auto-discovered by `ImageIO.read()`.
142+
143+
### Backend priority
144+
145+
When multiple backends are on the classpath (e.g. platform-native + vips), the consumer controls which backend handles each format via system properties:
146+
147+
```
148+
# Global ordering (left = highest priority). Default: native,vips,magick
149+
-Dimageio.native.backend.priority=native,vips,magick
150+
151+
# Per-format override
152+
-Dimageio.native.backend.priority.jpeg=vips,native
153+
-Dimageio.native.backend.priority.tiff=vips
154+
```
155+
156+
With no properties set, the default ordering is: platform-native first, then vips. This means existing users see no change when adding `imageio-native-vips` to the classpath -- it only activates for formats the platform-native backend can't handle.
157+
114158
## Video poster frames
115159

116160
The optional `imageio-native-video` module extracts a **single still image** from a video file -- the same way the image modules decode a still image from a HEIC or WebP file. The output is always a `BufferedImage`; no video playback, no audio, no frame sequences.
@@ -221,6 +265,7 @@ Both `getSize()` and `decode()` are orientation-aware: dimensions are swapped fo
221265
├── imageio-native-video-apple/ macOS video module (AVFoundation)
222266
├── imageio-native-video-windows/ Windows video module (Media Foundation)
223267
├── imageio-native-video/ cross-platform video aggregator
268+
├── imageio-native-vips/ optional libvips backend
224269
├── scripts/ test fixture generators
225270
└── example-consumer/ standalone demo (not in reactor)
226271
```

‎TODO-vips.md‎

Lines changed: 182 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,182 @@
1+
# TODO: imageio-native-vips + Backend Priority System
2+
3+
## Overview
4+
5+
Add `imageio-native-vips` as an optional backend module that delegates to
6+
libvips via Panama FFM. Also add a `BackendPriority` system so consumers
7+
can control which backend handles which formats.
8+
9+
`imageio-native-vips` is NOT added to the `imageio-native` aggregator --
10+
users opt in explicitly. It requires libvips installed on the system.
11+
12+
---
13+
14+
## Step 1: BackendPriority class in imageio-native-common
15+
16+
New class: `io.github.ghosthack.imageio.common.BackendPriority`
17+
18+
System properties:
19+
```
20+
-Dimageio.native.backend.priority=native,vips,magick # global ordering
21+
-Dimageio.native.backend.priority.jpeg=vips,native # per-format override
22+
```
23+
24+
API:
25+
- `priority(String backend)` -> int (lower = higher priority, from position in list)
26+
- `isAllowed(String backend, String format)` -> boolean
27+
28+
Defaults (no properties set): `native=0, vips=1, magick=2`. All backends allowed.
29+
Parsing happens once (static init), cached.
30+
31+
---
32+
33+
## Step 2: Retrofit existing SPIs with priority
34+
35+
`AppleImageReaderSpi` and `WicImageReaderSpi`:
36+
- `onRegistration()`: use `BackendPriority.priority("native")` + `setOrdering()`
37+
- `canDecodeInput()`: add `BackendPriority.isAllowed("native", format)` check
38+
39+
Backward compatible -- no properties set = identical to current behavior.
40+
41+
---
42+
43+
## Step 3: New module imageio-native-vips
44+
45+
### 3a. pom.xml
46+
47+
- Parent: imageio-native-parent
48+
- ArtifactId: imageio-native-vips
49+
- Dependencies: imageio-native-common, test-jar, junit
50+
- Added to parent POM modules list (NOT to imageio-native aggregator)
51+
52+
### 3b. VipsNative.java
53+
54+
Panama downcalls to libvips + GLib.
55+
56+
Library discovery order:
57+
1. `System.getProperty("imageio.native.vips.lib")`
58+
2. `/opt/local/lib/libvips.dylib` (MacPorts)
59+
3. `/usr/local/lib/libvips.dylib` (Homebrew)
60+
4. `/usr/lib/*/libvips.so` (Linux)
61+
5. `SymbolLookup.libraryLookup("vips", ...)` fallback
62+
63+
Also loads libgobject-2.0 / libglib-2.0 for g_object_unref / g_free.
64+
65+
Downcall handles:
66+
67+
| Function | Purpose |
68+
|----------|---------|
69+
| `vips_init` | One-time init |
70+
| `vips_image_new_from_file` | Load from path (variadic) |
71+
| `vips_image_new_from_buffer` | Load from bytes (variadic) |
72+
| `vips_image_get_width/height/bands` | Dimensions |
73+
| `vips_image_hasalpha` | Alpha detection |
74+
| `vips_colourspace` | Convert to sRGB |
75+
| `vips_premultiply` | Premultiply alpha |
76+
| `vips_cast_uchar` | Cast to 8-bit |
77+
| `vips_image_write_to_memory` | Get pixel buffer |
78+
| `vips_foreign_find_load_buffer` | Probe format support |
79+
| `vips_error_buffer` / `vips_error_clear` | Error handling |
80+
| `g_object_unref` / `g_free` | Cleanup |
81+
82+
Public methods:
83+
- `isAvailable()` -- load lib + vips_init, cache result
84+
- `canDecode(byte[], int)` -- vips_foreign_find_load_buffer
85+
- `getSize(byte[])` / `getSizeFromPath(String)` -- header only
86+
- `decode(byte[])` / `decodeFromPath(String)` -- full pipeline
87+
88+
Decode pipeline:
89+
```
90+
vips_image_new_from_file(path, "access", VIPS_ACCESS_SEQUENTIAL, NULL)
91+
-> vips_colourspace(VIPS_INTERPRETATION_sRGB)
92+
-> vips_premultiply() (if hasalpha)
93+
-> vips_cast_uchar()
94+
-> vips_image_write_to_memory()
95+
-> repack RGBA -> 0xAARRGGBB int[]
96+
-> BufferedImage(TYPE_INT_ARGB_PRE)
97+
cleanup: g_free(buf), g_object_unref(each image)
98+
```
99+
100+
RGBA -> ARGB repacking:
101+
```java
102+
dest[i] = (a << 24) | (r << 16) | (g << 8) | b; // 4-band
103+
dest[i] = 0xFF000000 | (r << 16) | (g << 8) | b; // 3-band
104+
```
105+
106+
### 3c. VipsImageReader.java
107+
108+
Extends NativeImageReader. Overrides all 4 hooks:
109+
- nativeGetSize(byte[]) -> VipsNative.getSize()
110+
- nativeDecode(byte[]) -> VipsNative.decode()
111+
- nativeGetSizeFromPath(String) -> VipsNative.getSizeFromPath()
112+
- nativeDecodeFromPath(String) -> VipsNative.decodeFromPath()
113+
114+
### 3d. VipsImageReaderSpi.java
115+
116+
Constructor: hardcoded format names and suffixes:
117+
- Formats: HEIC, HEIF, AVIF, WebP, JPEG2000, JP2, TIFF, OpenEXR, EXR,
118+
PDF, SVG, GIF, FITS, PBM, PGM, PPM, PFM
119+
- Suffixes: heic, heif, avif, webp, jp2, j2k, tif, tiff, exr, pdf, svg,
120+
gif, fits, fit, pbm, pgm, ppm, pfm
121+
- MIME types: corresponding standard types
122+
123+
canDecodeInput():
124+
1. BackendPriority.isAllowed("vips", format) -- bail if excluded
125+
2. VipsNative.isAvailable() -- bail if lib not found
126+
3. FormatDetector.isJavaNativeFormat() -- bail if supplemental mode excludes it
127+
4. VipsNative.canDecode(header, len) -- probe
128+
129+
onRegistration():
130+
- BackendPriority.priority("vips") for SPI ordering
131+
132+
### 3e. Service file
133+
134+
META-INF/services/javax.imageio.spi.ImageReaderSpi:
135+
io.github.ghosthack.imageio.vips.VipsImageReaderSpi
136+
137+
### 3f. Tests
138+
139+
VipsImageReaderTest:
140+
- Decode test8x8.heic, .avif, .webp, .png via VipsNative
141+
- Verify 8x8 dimensions + quadrant colors (tolerance for lossy)
142+
- Gated by assumeTrue(VipsNative.isAvailable())
143+
144+
VipsImageioTest:
145+
- ImageIO.read() via SPI
146+
- Supplemental mode respected
147+
148+
### 3g. README
149+
150+
New "Optional backends" section:
151+
- What imageio-native-vips adds
152+
- Requires libvips installed (MacPorts, Homebrew, apt)
153+
- Hardcoded format list decision documented
154+
- Backend priority configuration + examples
155+
156+
---
157+
158+
## Step 4: Build and verify
159+
160+
1. Compile all modules including imageio-native-vips
161+
2. Existing 126 tests pass unchanged
162+
3. New vips tests pass (on machines with libvips)
163+
4. example-consumer still works
164+
165+
---
166+
167+
## Files
168+
169+
| File | Change |
170+
|------|--------|
171+
| pom.xml (parent) | Add imageio-native-vips to modules |
172+
| common/.../BackendPriority.java | NEW |
173+
| apple/.../AppleImageReaderSpi.java | Add BackendPriority checks |
174+
| windows/.../WicImageReaderSpi.java | Add BackendPriority checks |
175+
| imageio-native-vips/pom.xml | NEW module POM |
176+
| vips/.../VipsNative.java | NEW -- Panama downcalls |
177+
| vips/.../VipsImageReader.java | NEW -- extends NativeImageReader |
178+
| vips/.../VipsImageReaderSpi.java | NEW -- SPI with priority |
179+
| vips/...services/...ImageReaderSpi | NEW -- service file |
180+
| vips/.../VipsImageReaderTest.java | NEW |
181+
| vips/.../VipsImageioTest.java | NEW |
182+
| README.md | Optional backends section |

‎imageio-native-common/src/main/java/io/github/ghosthack/imageio/common/BackendPriority.java‎

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
package io.github.ghosthack.imageio.common;
2+
3+
import java.util.*;
4+
5+
/**
6+
* Controls the priority ordering of image decoding backends and per-format
7+
* backend routing.
8+
* <p>
9+
* When multiple backends are on the classpath (e.g. platform-native + libvips),
10+
* this class determines which backend gets first crack at each format.
11+
* <p>
12+
* Configured via system properties:
13+
* <ul>
14+
* <li>{@code imageio.native.backend.priority} — global ordering, comma-separated,
15+
* left = highest priority. Default: {@code native,vips,magick}</li>
16+
* <li>{@code imageio.native.backend.priority.<format>} — per-format override,
17+
* e.g. {@code -Dimageio.native.backend.priority.jpeg=vips,native}</li>
18+
* </ul>
19+
* <p>
20+
* When no properties are set, all backends are allowed and the default ordering
21+
* is: {@code native} (platform-native) first, then {@code vips}, then {@code magick}.
22+
*/
23+
public final class BackendPriority {
24+
25+
/** System property for global backend ordering. */
26+
public static final String PROPERTY = "imageio.native.backend.priority";
27+
28+
/** Default ordering when no system property is set. */
29+
private static final String DEFAULT_ORDER = "native,vips,magick";
30+
31+
/** Global ordering: backend name → priority index (lower = higher priority). */
32+
private static final Map<String, Integer> GLOBAL_PRIORITY;
33+
34+
/** Per-format overrides: format (lower-case) → ordered list of backend names. */
35+
private static final Map<String, List<String>> FORMAT_OVERRIDES;
36+
37+
static {
38+
// Parse global priority
39+
String order = System.getProperty(PROPERTY, DEFAULT_ORDER);
40+
GLOBAL_PRIORITY = parseOrder(order);
41+
42+
// Parse per-format overrides
43+
Map<String, List<String>> overrides = new HashMap<>();
44+
Properties props = System.getProperties();
45+
String prefix = PROPERTY + ".";
46+
for (String key : props.stringPropertyNames()) {
47+
if (key.startsWith(prefix) && key.length() > prefix.length()) {
48+
String format = key.substring(prefix.length()).toLowerCase(Locale.ROOT);
49+
String value = props.getProperty(key, "");
50+
List<String> backends = new ArrayList<>();
51+
for (String s : value.split(",")) {
52+
String trimmed = s.strip().toLowerCase(Locale.ROOT);
53+
if (!trimmed.isEmpty()) backends.add(trimmed);
54+
}
55+
if (!backends.isEmpty()) {
56+
overrides.put(format, List.copyOf(backends));
57+
}
58+
}
59+
}
60+
FORMAT_OVERRIDES = Map.copyOf(overrides);
61+
}
62+
63+
private BackendPriority() {}
64+
65+
/**
66+
* Returns the priority index for the given backend (lower = higher priority).
67+
* Backends not listed in the global ordering get {@code Integer.MAX_VALUE}.
68+
*
69+
* @param backend backend name (e.g. {@code "native"}, {@code "vips"}, {@code "magick"})
70+
* @return priority index, 0-based
71+
*/
72+
public static int priority(String backend) {
73+
Integer p = GLOBAL_PRIORITY.get(backend.toLowerCase(Locale.ROOT));
74+
return p != null ? p : Integer.MAX_VALUE;
75+
}
76+
77+
/**
78+
* Returns {@code true} if the given backend is allowed to handle the given
79+
* format.
80+
* <p>
81+
* If a per-format override exists (e.g.
82+
* {@code -Dimageio.native.backend.priority.jpeg=vips,native}), the backend
83+
* must appear in that list. Otherwise, the backend must appear in the
84+
* global ordering.
85+
*
86+
* @param backend backend name (e.g. {@code "native"}, {@code "vips"})
87+
* @param format format name (e.g. {@code "jpeg"}, {@code "heic"})
88+
* @return {@code true} if the backend is allowed for this format
89+
*/
90+
public static boolean isAllowed(String backend, String format) {
91+
String backendLower = backend.toLowerCase(Locale.ROOT);
92+
if (format != null) {
93+
String formatLower = format.toLowerCase(Locale.ROOT);
94+
List<String> override = FORMAT_OVERRIDES.get(formatLower);
95+
if (override != null) {
96+
return override.contains(backendLower);
97+
}
98+
}
99+
// No per-format override — check global list
100+
return GLOBAL_PRIORITY.containsKey(backendLower);
101+
}
102+
103+
private static Map<String, Integer> parseOrder(String order) {
104+
Map<String, Integer> map = new HashMap<>();
105+
int index = 0;
106+
for (String s : order.split(",")) {
107+
String trimmed = s.strip().toLowerCase(Locale.ROOT);
108+
if (!trimmed.isEmpty() && !map.containsKey(trimmed)) {
109+
map.put(trimmed, index++);
110+
}
111+
}
112+
return Map.copyOf(map);
113+
}
114+
}

0 commit comments

Comments
 (0)