Feature Maps ft. Patch Size

[abergsneider]

Alright, let's get this discussion started! I have a few concepts that I've kind of gone along without fully understanding the reasoning behind and seeking to get them clarified. Hopefully, this can help other people in the same boat as me and we can help each other.

Fig. 1 from An attempt at beating the 3D U-Net: (my notes in green and red)

My question which I've answered with what I've discovered is below. (Seeking for any corrections to it if I'm mistaken.)

Why are there 30 feature maps at the beginning of the architecture?

• Considering the data from KiTS19 as an example.
• Image slices are in the size of 512x512 with varying depths as high as ~610 and as low as ~29 slices
• Case 0: 610x512x512 = 159,907,840
• Case 61: 29x512x512 = 7,602,176
• Given the patch size of 80x160x160, we need around 30 feature maps (30 patch repetitions) to analyze the median size of the dataset images.

Is my understanding correct?

Fabian: Let me know if this is not the right place for this kind of posts, you're the boss at the end of the day. Thanks for everything again.

[FabianIsensee]

Hi,
I think you are mixing things up a little :-) The number of feature maps does not relate to the dataset size. It is one of nnU-Net's fixed parameters and is therefore the same for all datasets.
But honestly - thanks for bringing this to my attention! This is an error in the figure. The initial number of feature maps is 32, not 30. This is one of the changes I made when improving upon the initial release of nnU-Net. Note that this decision was in no way related to segmentation performance - the only reason I changed from 30 to 32 is because 32 is divisible by 8 and that is a requirement for automated mixed precision to work well!
Best,
Fabian

[abergsneider]

Hi Fabian,

Thanks for your fast response, as always... Looks like I'm helping somehow without even knowing haha

In regards to the number of features. I have just been meaning to try and familiarize myself with where it comes from. By digging around this is what I got:

"As a tradeoff between performance and memory consumption, the initial number of feature maps is set to 32 and doubled (halved) with each downsampling (upsampling) operation."

-Page 16 from: Automated Design of Deep Learning Methods for Biomedical Image Segmentation

"All networks start with some number of feature maps at the highest resolution...... This architecture uses 30 feature maps at the highest resolution."

Page 2-3 from: An attempt at beating the 3D U-Net

And by the way, after doing a grep -R "30" . on the source code, I noticed this line also states "30":

nnUNet/nnunet/network_architecture/generic_UNet.py

Line 171 in 251ab3d

BASE_NUM_FEATURES_3D = 30

But given that 32 is the correct number as you stated, where does it come from? Why not any other multiple of 8? I'm just trying to educate myself with this so far magical number.

Best,

Andres

P.S. Maybe I'm missing something key here, any resource you could point me at would me immensely appreciated.

[FabianIsensee]

This is where is comes from:

nnUNet/nnunet/training/network_training/nnUNetTrainerV2.py

Line 154 in 251ab3d

self.network = Generic_UNet(self.num_input_channels, self.base_num_features, self.num_classes,

self.base_num_features is what you are looking for. This number is saved in the plans file, see here:

nnUNet/nnunet/training/network_training/nnUNetTrainer.py

Line 363 in 251ab3d

self.base_num_features = plans['base_num_features']

It was set to 30 in the initial nnU-Net (which did not support mixed precision yet) and was later changed to 32 when mixed precision support was added. 30 and 32 are very similar, there was no segmentation performance difference between them. 30/32 was chosen as trade-off between performance and memory consumption, with smaller numbers resulting in faster training but potentially (not necessarily!) reduced segmentation accuracy.

Best,
Fabian

[abergsneider]

Hi Fabian, sorry for my delayed response.

Yes, this helps. Thank you very much for your guidance.

Have a great weekend!

Andres