Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Catch and fix Google Healthcare API errors #149

Merged
merged 1 commit into from
Feb 12, 2024

Conversation

psavery
Copy link
Contributor

@psavery psavery commented Jan 30, 2024

This change adds support for Google Healthcare API DICOMweb servers, such as the NCI's Imaging Data Commons.

The problem: Google Healthcare API raises an error if AvailableTransferSyntaxUID is a field, or if SOPClassUID is used as a search filter.

The SOPClassUID should definitely be allowed as an instance-level search filter, as documented in Table 10.6.1-5. Required Matching Attributes. However, this has apparently been a long-standing problem of nearly four years (see here), so it may not be fixed anytime soon. And even if it is fixed, the Imaging Data Commons may not update their software anytime soon. It would be highly advantageous to support such a large DICOMweb repository by working around the issue.

The fix in this PR is as follows:

  1. The two search_for_instances() calls are still performed identically as before, as long as there are no HTTP errors.
  2. If there is an HTTP error with a 400 status_code, and a message is present matching the errors from Google Healthcare API, then the search_for_instances() arguments are patched to work for Google Healthcare API, as follows: a) AvailableTransferSyntaxUID is simply removed, if present. b) SOPClassUID is manually filtered, if present (meaning it is not supplied in the search_filters, but only instances with a matching SOPClassUID are returned).

These changes shouldn't have any impact on any situations except where an error occurs from a Google Healthcare API server. And in that case, the function calls are patched and then work properly.

The following example works after this fix:

from wsidicom import WsiDicom, WsiDicomWebClient

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.227261840503961430496812955999336758586'
series_uid = '1.3.6.1.4.1.5962.99.1.1334438926.1589741711.1637717011470.2.0'

client = WsiDicomWebClient.create_client(url)

slide = WsiDicom.open_web(client, study_uid, series_uid)

Fixes: #141

psavery added a commit to psavery/large_image that referenced this pull request Jan 30, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Jan 30, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
This change adds support for Google Healthcare API DICOMweb servers, such
as the NCI's [Imaging Data Commons](https://datacommons.cancer.gov/repository/imaging-data-commons).

The problem: Google Healthcare API raises an error if `AvailableTransferSyntaxUID` is a
field, or if `SOPClassUID` is used as a search filter.

The `SOPClassUID` should definitely be allowed as an instance-level search
filter, as documented in [Table 10.6.1-5. Required Matching Attributes](https://dicom.nema.org/medical/dicom/current/output/chtml/part18/sect_10.6.html).
However, this has apparently been a long-standing problem of nearly four
years (see [here](GoogleCloudPlatform/healthcare-dicom-dicomweb-adapter#30 (comment))),
so it may not be fixed anytime soon. And even if it is fixed, the Imaging Data
Commons may not update their software anytime soon. It would be highly
advantageous to support such a large DICOMweb repository by working around
the issue.

The fix in this PR is as follows:

1. The two `search_for_instances()` calls are still performed identically as before, as long as there are no HTTP errors.
2. If there is an HTTP error with a 400 status_code, and a message is present matching the errors from Google Healthcare API, then the `search_for_instances()` arguments are patched to work for Google Healthcare API, as follows:
a) `AvailableTransferSyntaxUID` is simply removed, if present.
b) `SOPClassUID` is manually filtered, if present (meaning it is not supplied in the `search_filters`, but only instances with a matching `SOPClassUID` are returned).

These changes shouldn't have any impact on any situations except where an error
occurs from a Google Healthcare API server. And in that case, the function calls
are patched and then work properly.

The following example works after this fix:

```python
from wsidicom import WsiDicom, WsiDicomWebClient

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.227261840503961430496812955999336758586'
series_uid = '1.3.6.1.4.1.5962.99.1.1334438926.1589741711.1637717011470.2.0'

client = WsiDicomWebClient.create_client(url)

slide = WsiDicom.open_web(client, study_uid, series_uid)
```

Fixes: imi-bigpicture#141

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Jan 31, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Jan 31, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Feb 1, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Feb 1, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Feb 6, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
@erikogabrielsson
Copy link
Collaborator

Hi @psavery

Thanks for your pull request. When parsing dicom files we have tried to handle as many implementations error as possible, so of coarse we should strive to do the same when reading from DICOM web.

Your approach inspired me to make some changes that would enables us to (hopefully) re-use the _search_for_instances() method also for other implementation errors. Is it ok if I push to your branch?

@psavery
Copy link
Contributor Author

psavery commented Feb 12, 2024

@erikogabrielsson Sure, feel free to push to it! 🙂

We would love to have these changes in soon, so that we can start utilizing wsidicom with that large database!

@psavery
Copy link
Contributor Author

psavery commented Feb 12, 2024

Another possibility would be to merge these changes, and add new features as a separate PR.

@erikogabrielsson erikogabrielsson merged commit 31819d5 into imi-bigpicture:main Feb 12, 2024
13 checks passed
@psavery psavery deleted the google-healthcare-api branch February 12, 2024 17:14
@psavery
Copy link
Contributor Author

psavery commented Feb 12, 2024

Thanks so much, @erikogabrielsson! Can we get a new release so we can start using these features?

psavery added a commit to psavery/large_image that referenced this pull request Feb 12, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
@erikogabrielsson
Copy link
Collaborator

Thanks so much, @erikogabrielsson! Can we get a new release so we can start using these features?

Released in 0.19.0

psavery added a commit to psavery/large_image that referenced this pull request Feb 13, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
psavery added a commit to psavery/large_image that referenced this pull request Feb 13, 2024
Google Healthcare API, used by Imaging Data Commons, does not allow
filtering by SOPClassUID. So we cannot use that in the search filter.
We should think of alternatives so that we can include only WSI results.

These changes were needed alongside [this wsidicom PR](imi-bigpicture/wsidicom#149)
in order to view an example dataset.

The following were used for testing:

url = 'https://proxy.imaging.datacommons.cancer.gov/current/viewer-only-no-downloads-see-tinyurl-dot-com-slash-3j3d9jyp/dicomWeb'
study_uid = '2.25.25644321580420796312527343668921514374'
series_uid = '1.3.6.1.4.1.5962.99.1.3205815762.381594633.1639588388306.2.0'

Signed-off-by: Patrick Avery <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

HTTP Errors for Imaging Data Commons
2 participants