Skip to content

Conversation

@marktsuchida
Copy link
Member

@marktsuchida marktsuchida commented Jul 17, 2025

MMCoreJ is currently built with SWIG 2-3 (2 on Windows, 3 on macOS, but it makes little difference).

SWIG 4 replaced the std::vector and std::map wrappers with versions that fit better into Java by extending java.util.List and java.util.Map.

This results in the generated Java API from being incompatible with that generated by SWIG 2-3, in the following ways (based on japicmp output; see #37 for full details):

  • *Vector, with * = Boolean, Char, Double, Long, Str, Unsigned

    • The constructor taking Java long (and producing that many empty/zero elements) was removed
    • Parameter type for add() and set() was changed from primitive to boxed (for all but StrVector); these methods also newly have a return value
    • get() now returns boxed, rather than primitive, type
    • size() and capacity() now return int rather than long
    • reserve() now takes int rather than long
  • StrMap

    • get() takes Object instead of String (source-compatible but not binary-compatible)
    • set() is replaced by put()
    • del() is removed (in favor of remove())
    • has_key() is replaced by containsKey()
    • empty() is replaced by isEmpty()
    • size() now returns int rather than long

(I'm ignoring backward-compatible changes, and incompatible changes to generated classes that should not be considered part of the API, such as MMCoreJJNI.)

This PR explicitly adds back the *Vector constructors taking long. This was enough to get all of our Java modules in micro-manager to compile (as of this writing).

For the remaining incompatibilities, I suggest that we leave them alone:

  • The changes from unboxed to boxed types and from long to int may cause compile errors under certain usage, but they do not cause any silent change in semantics. And they are hopefully even less likely to cause errors in BeanShell scripts.

  • I'm inclined leave StrMap alone, since StrMap is not actually used as a parameter or return value to any API methods (currently, at least). Even if there is code that uses it, it will at most get a compile error that should be easy to fix.


Before merging (let's do some minor "breaking" cleanup while we're at it):

  • Should BooleanVector be removed? (Not used in MMCore API)
  • Just remove StrMap and pair_ss? (Also not used in MMCore API)
  • Remove istringstream wrappers (%ignore the MetadataTag methods that take istringstream); these were never usable in Java
  • Remove %rename(eql) operator= (doesn't generate any Java code; not sure how you'd use it -- was it meant to be operator==?)
  • Remove (%ignore) MetadataError (exceptions are converted so not used)
  • Iterators no longer need remove() (Java 8 default implementation suffices)
  • Does swig-doc-converter (used by apidoc CI) work with SWIG 4 output?
  • Increment major version number (perhaps together with MMCore to avoid confusion)
  • Windows build machine image needs to have SWIG 4 available
  • macOS build machine image needs to have SWIG 4 available
  • apidoc CI job on micro-manager repo needs SWIG 4
  • Re-check that all of our Java modules build with the new code
  • Check that pycro-manager is not broken by this change
  • Check that AcqEngJ is not broken by this change
  • Update build instructions wherever we mention SWIG

Closes #37.

SWIG 4 replaced the `std::vector` and `std::map` wrappers with versions
that fit better into Java by extending `java.util.List` and
`java.util.Map`.

This results in the generated Java API from being incompatible with that
generated by SWIG 2-3, in the following ways (based on japicmp output):

- `*Vector`, with `*` = `Boolean`, `Char`, `Double`, `Long`, `Str`,
  `Unsigned`
  - The constructor taking Java `long` (and producing that many
    empty/zero elements) was removed
  - Parameter type for `add()` and `set()` was changed from primitive to
    boxed (for all but `StrVector`); these methods also newly have a
    return value
  - `get()` now returns boxed, rather than primitive, type
  - `size()` and `capacity()` now return `int` rather than `long`
  - `reserve()` now takes `int` rather than `long`

- `StrMap`
  - `get()` takes `Object` instead of `String` (source-compatible but
    not binary-compatible)
  - `set()` is replaced by `put()`
  - `del()` is removed (in favor of `remove()`)
  - `has_key()` is replaced by `containsKey()`
  - `empty()` is replaced by `isEmpty()`
  - `size()` now returns `int` rather than `long`

(I'm ignoring backward-compatible changes, and incompatible changes to
generated classes that should not be considered part of the API, such as
`MMCoreJJNI`.)

This PR explicitly adds back the `*Vector` constructors taking `long`.
This was enough to get all of our Java modules in `micro-manager` to
compile (as of this writing).

For the remaining incompatibilities, I suggest that we leave them alone:

- The changes from unboxed to boxed types and from `long` to `int` may
  cause compile errors under certain usage, but they do not cause any
  silent change in semantics. And they are hopefully even less likely to
  cause errors in BeanShell scripts.

- I'm inclined leave `StrMap` alone, since `StrMap` is not actually used
  as a parameter or return value to any API methods (currently, at
  least). Even if there is code that uses it, it will at most get a
  compile error that should be easy to fix.
Java 8 added a default implementation for Iterator#remove().
Did not seem to be generating any Java code. No use spotted either.
Like its already-hidden subclasses, it is not thrown by MMCoreJ.
Not used anywhere in MMCore API; we don't need it, so remove while we're
already making minor breaking changes.
Not used in MMCore API.
@marktsuchida
Copy link
Member Author

marktsuchida commented Dec 9, 2025

The changes to *Vector, although small, do require recompilation of code that uses these classes (hence the major version bump).

The classes removed above are not used by any of the jars currently shipped with Micro-Manager (according to jdeps).

However, the *Vector classes are, of course, used, and when they are used by jars that are not compiled as part of the Micro-Manager build, they will break:

  • PycroManagerJava uses StrVector.size().
  • AcqEngJ uses DoubleVector and StrVector, including size() and get().
  • PyJavaZ, NDTiffStorage, and NDViewer do not use these Vector classes.
  • Pycro-manager does not hard-code these types when accessing from Python, so Python code is not expected to break (there is no distinction between int/long, primitive/boxed).

So we need to release new versions of PycroManagerJava and AcqEngJ and ship those together with the new MMCoreJ.

@marktsuchida
Copy link
Member Author

marktsuchida commented Dec 9, 2025

It would be easier to manage the releases if MMCoreJ were already releasing to Maven Central, but that is stalled on (among other things) javadoc generation, which (under the current plan) requires SWIG 4 (#696). So one way to break the cycle is probably to manually deploy a (Java-only) preliminary MMCoreJ to Maven (built from this PR but before merging), so that we can update the dependencies for pycro-manager and AcqEngJ. The other way would be to work on #696 first without SWIG 4, which is also doable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Swig 4.0 makes incompatible changes to MMCoreJ API (and also fails build)

1 participant