LazyLinear (original) (raw)

class torch.nn.LazyLinear(out_features, bias=True, device=None, dtype=None)[source]#

A torch.nn.Linear module where in_features is inferred.

In this module, the weight and bias are of torch.nn.UninitializedParameterclass. They will be initialized after the first call to forward is done and the module will become a regular torch.nn.Linear module. The in_features argument of the Linear is inferred from the input.shape[-1].

Check the torch.nn.modules.lazy.LazyModuleMixin for further documentation on lazy modules and their limitations.

Parameters

• out_features (int) – size of each output sample
• bias (UninitializedParameter) – If set to False, the layer will not learn an additive bias. Default: True

Variables

• weight (torch.nn.parameter.UninitializedParameter) – the learnable weights of the module of shape(out_features,in_features)(\text{out\_features}, \text{in\_features}). The values are initialized from U(−k,k)\mathcal{U}(-\sqrt{k}, \sqrt{k}), wherek=1in_featuresk = \frac{1}{\text{in\_features}}
• bias (torch.nn.parameter.UninitializedParameter) – the learnable bias of the module of shape (out_features)(\text{out\_features}). If bias is True, the values are initialized fromU(−k,k)\mathcal{U}(-\sqrt{k}, \sqrt{k}) wherek=1in_featuresk = \frac{1}{\text{in\_features}}

cls_to_become[source]#

alias of Linear

initialize_parameters(input)[source]#

Infers in_features based on input and initializes parameters.

reset_parameters()[source]#

Resets parameters based on their initialization used in __init__.