Improve color table node format

[lassoan]

Is your feature request related to a problem? Please describe.

Slicer's Colors module handles color tables but it can only store label value, name, color, and opacity. There is a growing need for standardization of specifying content of a segmentation. Terminology support was added to segmentation but its adoption was slow, most likely because it was hard to create and interpret terminology json files.

Describe the solution you'd like

Improve the color table file format in Slicer:

• Use standard CSV file format
• Allow storage of standard terminology codes: to facilitate robust data exchange, DICOM storage, etc.
• Proposed field names:
  • LabelValue
  • Name
  • Color.R, Color.G, Color.B, Color.A
  • Category.CodingSchemeDesignator, Category.CodeValue, Category.CodeMeaning
  • Type.CodingSchemeDesignator, Type.CodeValue, Type.CodeMeaning
  • TypeModifier.CodingSchemeDesignator, TypeModifier.CodeValue, TypeModifier.CodeMeaning
  • AnatomicRegion.CodingSchemeDesignator, AnatomicRegion.CodeValue, AnatomicRegion.CodeMeaning
  • AnatomicRegionModifier.CodingSchemeDesignator, AnatomicRegionModifier.CodeValue, AnatomicRegionModifier.CodeMeaning

Describe alternatives you've considered

JSON would be a more robust and flexible storage format, but it was proven to be hard for users to edit them. Users are comfortable editing tables in Excel, copy/paste content, etc. while there are no similarly universally accessible editor/viewer for JSON files.

Additional context

See discussion: lassoan/SlicerMONAIAuto3DSeg#15

Slicer3 label file format:

(Note that space is used as separator and instead of escaping with quotes, space is replaced by underscore):

#WARNING: This file is automatically generated.
#WARNING: Any edits made to this file may be overwritten.
#
#Filename: Slicer3_2010_Brain_Labels.txt
#Title: Slicer3:2010 Brain Labels
#PageName: Slicer3:2010_Brain_Labels
#Source: https://www.slicer.org/w/index.php/Slicer3:2010_Brain_Labels
#
#Begin table data:
0 Background 0 0 0 0
1 Bone 231 197 108 255
2 Skin 177 122 101 255
3 Muscles 205 92 92 255
4 Fat 238 232 170 255
5 CSF 0 191 255 255
6 Ventricles 65 105 225 255
7 Arteries 250 128 114 255
8 Veins 70 130 180 255
9 Gray_matter 211 211 211 255
10 White_matter 255 255 240 255
11 Tumor 144 238 144 255
12 Edema 224 255 255 255
13 Necrosis 216 191 216 255
14 Structure_1 238 130 238 255
15 Structure_2 255 215 0 255
#EOF 

ITK-Snap label file format:

################################################
# ITK-SnAP Label Description File
# File format: 
# IDX   -R-  -G-  -B-  -A--  VIS MSH  LABEL
# Fields: 
#    IDX:   Zero-based index 
#    -R-:   Red color component (0..255)
#    -G-:   Green color component (0..255)
#    -B-:   Blue color component (0..255)
#    -A-:   Label transparency (0.00 .. 1.00)
#    VIS:   Label visibility (0 or 1)
#    IDX:   Label mesh visibility (0 or 1)
#  LABEL:   Label description 
################################################
    0     0    0    0        0  0  0    "Clear Label"
    1   255    0    0        1  1  1    "Label 1"
    2     0  255    0        1  1  1    "Label 2"
    3     0    0  255        1  1  1    "Label 3"
    4   255  255    0        1  1  1    "Label 4"
    5     0  255  255        1  1  1    "Label 5"
    6   255    0  255        1  1  1    "Label 6"

[pieper]

Yes, this is a great idea. Can they be backward compatible or does this need to be a new format?

[lassoan]

We would of course be able to read the old files but the current file format has irreparable CSV-incompatibilites: space is used as separator (and therefore space in label name is converted to underscore when writing the file, and underscore is replaced by space when reading the file) and it uses comment lines. Comment lines are not part of the RFC 4180 standard and therefore they cause inconvenience and incompatibility issues in many use cases, while comments provides very little value (as by definition its content should be ignored by machine processing).

[pieper]

Okay, then we should version it somehow like with a new file extension. It sounds like this would be a useful format to try to promote for tools like dcmqi, highdicom, ohif/dcmjs, etc too.

[jcfr]

Related:

• Consolidate ColorFiles documentation and hosting #6305

[jcfr]

JSON would be a more robust and flexible storage format, but it was proven to be hard for users to edit them.

Submitting the schema to https://www.schemastore.org so that standard editor are aware and offer auto-completion may help.

[pieper]

👍 for standardizing and clearly the meaning of the color files. Perhaps that meaning can have more than one standardized representation (i.e. json and csv - and dicom SEG?) with a well defined lossless conversion.

[CPBridge]

In highdicom, this information goes into the Segment Description class:

https://highdicom.readthedocs.io/en/latest/package.html#highdicom.seg.SegmentDescription

Generally the user is responsible for creating a list of these objects using the python API, and then passing this into the Segmentation construction. This is a little tedious and it would make sense to add a higher level way to create this list automatically from some standardized definition file such as the one proposed. I can imagine a small helper function could be added to highdicom to do this, and it may be useful to people.

A couple of things to note:

• In principle DICOM allows multiple anatomic regions per segment, whereas the proposed scheme only allows for one. But this could just be a limitation of the format and I don't think it would be too limiting in practice.
• There is also the information about the algorithm type and algorithm identification, which this does not address. But perhaps this could be passed into the helper function and assumed to be the same for all segments.
• In addition to the "anatomic regions", there are also the "primary anatomic structures", which it may be good to support: https://dicom.nema.org/medical/dicom/current/output/chtml/part03/sect_10.5.html#table_10-7b

[lassoan]

Thank you @CPBridge you bring up very good points. For storage of all these additional data that you mentioned we would not need to change the proposed format, just specify some more column names. For now, I would go without these extra columns to keep things simpler. Algorithm type and identification is essential, but probably it is better to specify that outside this table to make the table simpler and easier to reuse between different algorithms.

[muratmaga]

JSON would be a more robust and flexible storage format, but it was proven to be hard for users to edit them. Users are comfortable editing tables in Excel, copy/paste content, etc. while there are no similarly universally accessible editor/viewer for JSON files.

I would encourage JSON. I think the issue is creating the JSON. Perhaps the solution is a small tool that takes a data table in Slicer with relevant fields (can be even copy/pasted from Excel or loaded into Slicer), and then converts it to JSON while doing the scheme checks etc...

We did create a small tool like that, and since then people had been (almost) OK with terminologies. Maybe a version of this, can be turned into a utility module within Slicer? https://github.com/Imageomics/slicerMorph_JSON_generator

[muratmaga]

One more issue, perhaps tangential but related: When the order of the segments are changed (but the terminologies are the same), and one exports a labelmap (because they want to train a ML mode), the assigned label indexs depends on the order of the segments. This is a confusing behavior. If I called my segments bone, tissue, skin, etc... I would expect the indices assigned them to be unique and consistent whether the order is tissue, bone, skin, or skin, tissue, bone.

I understand the ordering matter for flattening the segmentation, but the indices I believe remain the same.

When your updating color table / segmentation terminologies, it would be good to consider this too as well. it is a bit awkward at the moment that you have to explicitly specify a color table to deal with this. Particularly weird, if someone has gone into the trouble of using Terminology module to assign names/colors to segments and yet they have to create their own custom color table.

[lassoan]

the assigned label indexs depends on the order of the segments

The most important application of color tables is to prevent this. Voxel values in the exported segmentation are independent of the order of segments if a color table is selected for the export.

I would encourage JSON

It would be great, but users and software toolkits are not ready for this. The fact that you had to create a tool proves this. The format is not just for Slicer but for all AI segmentation tools and applications, so we would need tools on several platforms. There are few solutions for rendering json to html table, but I'm not aware of any commonly available tool for editing. Excel can import json but you need to go through a long manual process each time to define how to convert the tree to a table. We would need to develop, maintain, advertise convenience tools on several platforms, which would take a lot of time. I would be happy to reconsider if you have a good editing solution.

[muratmaga]

The most important application of color tables is to prevent this. Voxel values in the exported segmentation are independent of the order of segments if a color table is selected for the export.

But as it stands, I have to specify the color table at the export time explicitly, right? To me this just seems awkward, particularly if I chose to name my segments using a standardized terminology and coloring. What I am trying to get at, the part of that terminology DB should have field like "LabelMap Export Index" or something that should define consistently what the index number of that named segment should be, without having to specify a color table. I may not be seeing the whole picture here, but this seems like a reasonable expectation on user end. Color table is great, if I need change that indices, or there are gaps in sequence etc...

The format is not just for Slicer but for all AI segmentation tools and applications, so we would need tools on several platforms.

If the format is public, I am not sure why the tool support needs to be on many different platform, if I our goal is to increase the adoption of usage of Terminologies in Slicer. But, anyways, if the tool need to support different communities, it can be a web based application. It shouldn't be difficult to turn what we have to a web based front end, I can ask the people who help developed for us for idea.

I still think the issue is not the tool, but what the format will contain, particularly if it is meant to serve the whole clinical imaging community.