ak.from_buffers — Awkward Array 2.8.2 documentation (original) (raw)

Defined in awkward.operations.ak_from_buffers on line 30.

ak.from_buffers(form, length, container, buffer_key='{form_key}-{attribute}', *, backend='cpu', byteorder='<', allow_noncanonical_form=False, highlevel=True, behavior=None, attrs=None)#

Parameters:

• form (ak.forms.Form or str/dict equivalent) – The form of the Awkward Array to reconstitute from named buffers.
• length (int) – Length of the array. (The output of this function is always single-partition.)
• container (Mapping , such as dict) – The str → Python buffers that represent the decomposed Awkward Array. This container is only assumed to have a __getitem__ method that accepts strings as keys.
• buffer_key (str or callable) – Python format string containing"{form_key}" and/or "{attribute}" or a function that takes these as keyword arguments and returns a string to use as a key for a buffer in the container.
• backend (str) – Library to use to generate values that are put into the new array. The default, cpu, makes NumPy arrays, which are in main memory (e.g. not GPU). If all the values incontainer have the same backend as this, they won’t be copied.
• byteorder ("<", ">") – Endianness of buffers read from container. If the byteorder does not match the current system byteorder, the arrays will be copied.
• allow_noncanonical_form (bool) – If True, non-canonical forms will be simplified to produce arrays with canonical layouts; otherwise, an exception will be thrown for such forms.
• highlevel (bool) – If True, return an ak.Array; otherwise, return a low-level ak.contents.Content subclass.
• behavior (None or dict) – Custom ak.behavior for the output array, if high-level.
• attrs (None or dict) – Custom attributes for the output array, if high-level.

Reconstitutes an Awkward Array from a Form, length, and a collection of memory buffers, so that data can be losslessly read from file formats and storage devices that only map names to binary blobs (such as a filesystem directory).

The first three arguments of this function are the return values ofak.to_buffers, so a full round-trip is

reconstituted = ak.from_buffers(*ak.to_buffers(original))

The container argument lets you specify your own Mapping, which might be an interface to some storage format or device (e.g. h5py). It’s okay if the container dropped NumPy’s dtype and shape information, leaving raw bytes, since dtype and shape can be reconstituted from theak.forms.NumpyForm. If the values of container are recognised as arrays by the given backend, a view over their existing data will be used, where possible. The container values are allowed to be callables with no arguments. If that’s the case, they will be turned into VirtualArray buffers whose generator function is the callable and is used to materialize the buffer when required.

The buffer_key should be the same as the one used in ak.to_buffers.

When allow_noncanonical_form is set to True, this function readily accepts non-simplified forms, i.e. forms which will be simplified by Awkward Array into “canonical” representations, e.g. option[option[...]] → option[...]. Such forms can be produced by the low-level ArrayBuilder snapshot() method. Given that Awkward Arrays must have canonical layouts, it follows that invoking this function with allow_noncanonical_form may produce arrays whose forms differ to the input form.

In order for a non-simplified form to be considered valid, it should be one that the ak.contents.Content layout classes could produce iff. the simplification rules were removed.

See ak.to_buffers for examples.