Add densenet models by gpleiss · Pull Request #116 · pytorch/vision (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation16 Commits2 Checks0 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

gpleiss

See issue #97.

It's not super-memory optimized (i.e. there's a concatenation at every layer). This is consistent with the original Torch implementation, and prevents some gross autograd hacks.

Pretrained models for the model zoo are available here. They're converted over from the original Torch implementation (ported from LuaTorch).

@soumith

awesome. do the DenseNet pre-trained models expect the same normalization as the rest of the models?

I'm going to upload your models to the pytorch s3 bucket so that you can make them available via the pretrained=True

@soumith

fmassa
# First convolution
self.features = nn.Sequential()
self.features.add_module('conv0', nn.Conv2d(3, num_init_features, kernel_size=7, stride=2, padding=3, bias=False))

This comment was marked as off-topic.

fmassa

drop_rate (float) - dropout rate after each dense layer
num_classes (int) - number of classification classes
"""
def __init__(self, growth_rate=32, block_config=(6,12,24,16), num_init_features=64, bn_size=4, drop_rate=0, num_classes=1000):

This comment was marked as off-topic.

@gpleiss

@soumith - yeah, same normalization as the other ImageNet models. I'll rename the files

@gpleiss

@gpleiss

@soumith

They are now uploaded to the bucket and available via URLs:

https://download.pytorch.org/models/densenet121-1e136e00.pth
https://download.pytorch.org/models/densenet*

@fmasa

@gpleiss

@soumith

@gpleiss the new files have been uploaded to the same place i.e. https://download.pytorch.org/models/

@gpleiss

@gpleiss

pretrained=True is now ready

@soumith

Thank you! this is good stuff.

@trypag

@gpleiss

@trypag the cifar models use 3x3 convolution for the first layer, but the ImageNet models use 7x7 convolution. The author's implementation is only for the CIFAR models. However, if you download their pretrained imagenet models the first layer is a 7x7 convolution.

@trypag

@farleylai

The authors address the memory efficiency in the followup paper and updated repo with underlying shared memory and re-computation on bp. The current pytorch code just uses torch.cat() directly. Any plan to formalize the technique? It is likely densenet based CNNs are going to prosper for other applications.

rajveerb pushed a commit to rajveerb/vision that referenced this pull request

Nov 30, 2023

@christ1ne @petermattson