Difference in clusters sorted by spykingcircus2 and mountainsort5

[kshtjkumar]

Hi ,
I have a question regarding the significant disparity in the number of clusters identified by MountainSort5 and SpykingCircus2. I have a tetrode recording that lasts for 1277 seconds. When I use SpykingCircus2 for sorting, I get 153 clusters, of which 78 have an ISI violation ratio below 0.5. In contrast, the same recording sorted with MountainSort5 results in only 4 clusters, with just 1 cluster exceeding the ISI threshold.

What could be causing such a large difference in the number of clusters?

here is what the probe looks like:

this is the raster form the mountainsort5:

[yger]

I don't know which version of spikeinterface you are using, but lots of improvements have been made to circus2 in the main branch. Since it is unlikely that you can find so many clusters on few channels, either there is a mismatch in the parameters, or something weird with the data for sc2. Are they properly filtered/preprocessed? Are you setting apply_preprocessing=False if you have preprocessing the data with your own filters?

[zm711]

Also I'll tag @magland, so he can comment if he wants about MS5 stuff.

[kshtjkumar]

I don't know which version of spikeinterface you are using, but lots of improvements have been made to circus2 in the main branch. Since it is unlikely that you can find so many clusters on few channels, either there is a mismatch in the parameters, or something weird with the data for sc2. Are they properly filtered/preprocessed? Are you setting apply_preprocessing=False if you have preprocessing the data with your own filters?

I am using v0.100.6, I am passing the preprocessed data to the sc2 sorter, it is re-referenced, notch filtered and bandpassed(300-6000).

[magland]

@kshtjkumar I would play with the ms5 parameters. I'm thinking especially about the detect threshold.

[kshtjkumar]

@kshtjkumar I would play with the ms5 parameters. I'm thinking especially about the detect threshold.

Could you please suggest on how to go with that ?

[magland]

@kshtjkumar I would play with the ms5 parameters. I'm thinking especially about the detect threshold.

Could you please suggest on how to go with that ?

Could you provide the code/script you are using to run this? Then we can show how to adjust the parameters.

[kshtjkumar]

@kshtjkumar I would play with the ms5 parameters. I'm thinking especially about the detect threshold.

Could you please suggest on how to go with that ?

Could you provide the code/script you are using to run this? Then we can show how to adjust the parameters.

sure:

recording_ecog = spre.bandpass_filter(recording_resampled_ecog, freq_min=300, freq_max = 6000 ) #bandpass filter
recording_notch_ecog = spre.notch_filter(recording_ecog,q = 50)  #notch_filter 
rec_ecog_ref = spre.common_reference(recording_notch_ecog, operator="median", reference="global")  #rereferencing the data 
rec_normed = spre.zscore(recording=rec_ecog_ref)
rec_2 = spre.whiten(rec_normed) 
sorting_rec = ms5.sorting_scheme2(rec_2,
            sorting_parameters=ms5.Scheme2SortingParameters(
                phase1_detect_channel_radius=150,
                detect_channel_radius=50,
                training_duration_sec=60
            )
        )
print(output_folder)
print("Sorter found", len(sorting_rec.get_unit_ids()), "units")
sorting_rec = sorting_rec.remove_empty_units()
print("Sorter found", len(sorting_rec.get_unit_ids()), "non empty units")

[magland]

The parameter to add to Scheme2SortingParameters is detect_threshold. You can see ms5 parameters here

https://github.com/flatironinstitute/mountainsort5/blob/main/docs/scheme2.md

And the defaults are here

https://github.com/flatironinstitute/mountainsort5/blob/main/mountainsort5/schemes/Scheme2SortingParameters.py