After registration, is the order of transforms composition specified in the documentation correct?

[sdiebolt]

Describe the bug

The order of transform composition after registration might be wrong in most of the documentation and tutorial notebooks.

To Reproduce

In the 05_basic_registration.ipynb notebook, compare the resampling that uses the composite transform created by the current version of the notebook, i.e.

final_transform_v4 = sitk.CompositeTransform(
    [registration_method.Execute(fixed_image, moving_image), initial_transform]
)

with the composite transform that uses the correct order (following ITK's book):

final_transform_v4 = sitk.CompositeTransform(
    [initial_transform, registration_method.Execute(fixed_image, moving_image)]
)

The second resampling seems better to me than the first.

To see the error even more clearly, we can use a moving initial transform that performs a flip along one axis. For example, add the following to the 7th code cell before executing the registration:

initialization = np.eye(4)
initialization[0, 0] = -1
initialization[0, 3] = moving_image.GetSize()[0] * moving_image.GetSpacing()[0]

moving_initial_transform = sitk.AffineTransform(
    initialization[:3, :3].flatten(), initialization[:3, 3], (0,) * 3
)
registration_method.SetMovingInitialTransform(moving_initial_transform)

The registration should still be satisfactory due to the symmetry of the brain image.

Composing the optimized and moving initial transforms as described in SimpleITK's documentation leads to an incorrect result:

However, composing these transforms using

composite_transform = sitk.CompositeTransform([moving_initial_transform, final_transform])

as is described in section 3.2 of ITK's book leads to the correct result:

1. Operating system, version, and architecture

• OS: Ubuntu 22.04.3 LTS
• Architecture: x86_64

2. Programming language, and version: Python 3.11.
3. Version of SimpleITK: 2.3.1
4. How was SimpleITK installed? pip
5. A minimal working example which causes the error. See above.

Expected behavior

The resampling should be accurate using the moving initial transforms above.

Additional context

I originally opened a topic on ITK's Discourse as I was confused about the discrepancies between SimpleITK's documentation and the resampling results I got when using axis flips in the moving initial transform. After further research, I believe there is a discrepancy between ITK's documentation on how to compose transforms after registration and SimpleITK's. Notably, most of the public GitHub repositories that use SimpleITK already apply the correct composition order (i.e. the inverse order of what the documentation specifies). The ImageRegistrationMethodDisplacement1.py example from SimpleITK is also using the correct order (sparking two questions on Discourse, here and here, that got no explicit answers).

Actually, I think the error is illustrated by this figure from one of SimpleITK's courses:

Shouldn't $T_\text{opt}$ actually go from virtual image space to virtual image space, since the registration is performed in the virtual space? Then the final output transform would be $T_\text{m}\circ T_\text{opt}\circ T_\text{f}^{-1}$ like described in ITK's guide, and not $T_\text{opt}\circ T_\text{m}\circ T_\text{f}^{-1}$ like in SimpleITK's documentation.

I hope I'm wrong however, and would very much appreciate your help in understanding this problem!

[blowekamp]

Thank you for pointing this out. It looks like the Registration Overview documentation in this repository needs to be updated.

Another issue has also been created in the Notebooks repository. Please let us know if you have encountered other places (that can be updated) which container this wrong order of transformations.

[sdiebolt]

I've found this error in two main places: the "Registration Overview" section of the documentation that you already pointed out, and all versions of the XX_basic_registration.ipynb notebook (the error in present in both R and Python notebooks for all such courses, i.e. SPIE2018, ISBI2020, the SimpleITK TUTORIAL repository, etc.).

I think most of these notebooks are in archived repositories, but since they often come up highly ranked in e.g. Google searches (the SPIE2019 notebook comes up as one of the top results when looking for "SimpleITK registration" on a fresh browser), maybe they should still be updated?

I'd be happy to open PRs to fix these issues if that's of any help!

[blowekamp]

Thank you for offering to contribute. Identifying the locations of the places that need to be updated is a very important and critical step.

For this repository, we are currently looking for an original copy of the figure used in the RtD documentation to update the transformation figure and the the formula. @zivy

[zivy]

Addressed by PR #2055