fastnas — Model Optimizer 0.27.1 (original) (raw)

Module implementing fasnas pruning algorithm for search.

Classes

Functions

class BinarySearcher

Bases: IterativeSearcher

An iterative searcher that uses binary search to find the best configuration.

after_step()

Update boundaries of the interval after recursing.

Return type:

None

before_search()

Build sensitivity map before search that we use to approximate the cost function.

Return type:

None

before_step()

Check what the middle value is to determine where we recurse.

Return type:

None

property default_state_dict_: dict[str, Any]_

We also store the sensitivity map and related arguments.

early_stop()

Early stop if the interval is small enough.

Return type:

bool

property hparam_names_for_search_: set[str]_

We can only optimize over certain types of hparams in binary search.

property hparam_types_for_search_: tuple[type]_

We can only optimize over certain types of hparams in binary search.

load_search_checkpoint()

We only want to load sensitivity map and original_score here and keep the rest.

Return type:

bool

max_degrade_: float_

middle_value_: float_

min_degrade_: float_

original_score_: float_

sample()

Check in which interval we should recurse and sets the corresponding subnet.

Return type:

_dict_[str, _Any_]

sensitivity_map_: dict[str, dict[int, float]]_

class FastNASPatchManager

Bases: AutoNASPatchManager

A patch manager for FastNAS (same as AutoNAS except no sampling during training).

property sample_during_training

Indicates whether we should sample a new subnet during training.

convert_fastnas_searchspace(model, config)

Convert search space for FastNAS mode with correct patch manager.

Parameters:

• model (Module) –
• config (ModeloptBaseConfig) –

Return type:

_tuple_[Module, _dict_[str, _Any_]]

restore_fastnas_searchspace(model, config, metadata)

Restore search space for FastNAS mode with correct patch manager.

Parameters:

• model (Module) –
• config (ModeloptBaseConfig) –
• metadata (dict [ str , Any ]) –

Return type:

Module