Save Variables in parfor-Loops - MATLAB & Simulink (original) (raw)
Main Content
This example shows how to use the save function in a parfor-loop to save one or more variables to a file.
To use the save function in the body of a parfor-loop, you must use the "-fromstruct" option. If you do not use this option, MATLAB® produces an error due to a transparency violation.
You must also ensure that each worker or parfor iteration has unique access to any files it writes or saves data to. When multiple workers attempt to write to the same file, MATLAB can produce an error or corrupted data, or one worker might overwrite the data from another worker. These issues are more likely to occur when:
- There is more than one worker per machine, and the workers attempt to write to the same file.
- The workers have a shared file system and use the same path to identify a file for writing.
Start a parallel pool of process workers.
pool = parpool("Processes");
Starting parallel pool (parpool) using the 'Processes' profile ... Connected to parallel pool with 6 workers.
Use a parfor-loop to generate data and save it to separate MAT files in each iteration. Temporarily store the results variable in a scalar structure. Use the "-fromstruct" option of the save function to save the field and value from the structure to a file as a variable.
parfor idx = 1:pool.NumWorkers x = rand(1000*idx); s = struct("x",x); save(sprintf("output_%d.mat",idx),"-fromstruct",s); end
By default, parpool sets the working folder on the workers to match that of the MATLAB client session. So, the client current working folder becomes the default folder where the workers save the files.
View the contents of one of the files using the whos function.
whos("-file","output_1.mat");
Name Size Bytes Class Attributes
x 1000x1000 8000000 double
To save multiple variables, create a structure with multiple fields and then save the fields and values as individual variables.
parfor idx = 1:pool.NumWorkers x = rand(1000*idx); y = eye(idx); z = magic(idx); s = struct("x",x,"y",y,"z",z); save(sprintf("output_%d.mat",idx),"-fromstruct",s); end
View the contents of one of the files.
whos("-file","output_2.mat");
Name Size Bytes Class Attributes
x 2000x2000 32000000 double
y 2x2 32 double
z 2x2 32 double
Delete the files and shut down the parallel pool.
delete output* delete(pool);
Parallel pool using the 'Processes' profile is shutting down.