Cheat Sheet — Intel&#174 Extension for PyTorch* 2.7.0+cpu documentation (original) (raw)

Basic CPU Installation

python -m pip install intel_extension_for_pytorch

Import Intel® Extension for PyTorch*

import intel_extension_for_pytorch as ipex

Capture a Verbose Log (Command Prompt)

export ONEDNN_VERBOSE=1

Optimization During Training

model = ...
optimizer = ...
model.train()
model, optimizer = ipex.optimize(model, optimizer=optimizer)

Optimization During Inference

model = ...
model.eval()
model = ipex.optimize(model)

Optimization Using the Low-Precision Data Type bfloat16
During Training (Default FP32)

model = ...
optimizer = ...
model.train()

model, optimizer = ipex.optimize(model, optimizer=optimizer, dtype=torch.bfloat16)

with torch.no_grad():
with torch.cpu.amp.autocast():
model(data)

Optimization Using the Low-Precision Data Type bfloat16
During Inference (Default FP32)

model = ...
model.eval()

model = ipex.optimize(model, dtype=torch.bfloat16)

with torch.cpu.amp.autocast():
model(data)

[Prototype] Fast BERT Optimization

from transformers import BertModel
model = BertModel.from_pretrained("bert-base-uncased")
model.eval()

model = ipex.fast_bert(model, dtype=torch.bfloat16)

Run CPU Launch Script (Command Prompt):
Automate Configuration Settings for Performance

ipexrun [knobs] <your_pytorch_script> [args]

[Prototype] Run HyperTune to perform hyperparameter/execution configuration search

python -m intel_extension_for_pytorch.cpu.hypertune --conf-file <your_conf_file> <your_python_script> [args]

[Prototype] Enable Graph capture

model = …
model.eval()
model = ipex.optimize(model, graph_mode=True)

Post-Training INT8 Quantization (Static)

model = …
model.eval()
data = …

qconfig = ipex.quantization.default_static_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data, anyplace=False)

for d in calibration_data_loader():
prepared_model(d)

converted_model = ipex.quantization.convert(prepared_model)

Post-Training INT8 Quantization (Dynamic)

model = …
model.eval()
data = …

qconfig = ipex.quantization.default_dynamic_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data)

converted_model = ipex.quantization.convert(prepared_model)

[Prototype] Post-Training INT8 Quantization (Tuning Recipe):

model = …
model.eval()
data = …

qconfig = ipex.quantization.default_static_qconfig

prepared_model = ipex.quantization.prepare(model, qconfig, example_inputs=data, inplace=False)

tuned_model = ipex.quantization.autotune(prepared_model, calibration_data_loader, eval_function, sampling_sizes=[100],
accuracy_criterion={'relative': .01}, tuning_time=0)

convert_model = ipex.quantization.convert(tuned_model)