Memref assumptions about underlying memory (original) (raw)

December 7, 2025, 3:41am 1

Hey everyone,

I am a bit confused about the semantics of the memref type and was hoping for some clarifications.

What are the assumptions made about the underlying region of memory a memref points to?

There are 2 thoughts I had:

• A memref assumes a 1D linear structure of memory
• A memref assumes the underlying memory structure is exactly the same as its shape

The latter makes more sense to me since lowerings may assume different models of memory either logically or physically.

I would think memrefs abstract notion of memory would not be restricted to 1D pointers, but the docs did not make this clear to me.

Have you ever heard the physics definition of a tensor?

“A tensor is something that transforms like a tensor” - Anthony Zee

I would propose a similar sort of definition of memref:

“A memref is something that lowers like a memref”

By which I mean, the quickest way to understand the semantics (IMHO) is to see what it lowers to for e.g., LLVM:

template <typename T, int N>
struct StridedMemRefType {
  T *basePtr;
  T *data;
  int64_t offset;
  int64_t sizes[N];
  int64_t strides[N];
  ...
}

template <typename T>
struct UnrankedMemRefType {
  int64_t rank;
  void *descriptor;
};

See StridedMemRefType, UnrankedMemRefType.

If you inspect various snippets of LLVM (dialect or IR), post lowering, they will reflect these structs (this is the unofficial memref ABI). Possibly there are MLIR compilers out there which do something dramatically different when lowering memref to their target backend - I would be surprised, but it is doable to hook the LLVMTranslationInterface and/or the various memref-to-llvm passes.

ftynse December 9, 2025, 1:03pm 3

Historically, we have been pushing back on defining semantics through lowerings. For memref, I think, the semantics are that it’s a buffer that can be addressed in a multi-dimensional way. There is a lot of code that at least implicitly assumes that the underlying memory is flat and one-dimensional, e.g., the fact that the lack of layout is understood as “contiguous row-major” layout.

Alborz December 9, 2025, 3:32pm 4

The problem I see with this definition is that memref would not be complete. By this I mean you cannot use the dialect on its own. Imagine if someone is creating an interpreter for memref. How would this be possible without the semantics defined within the dialect itself? The interpreter would have to make a (possibly important) underlying assumption about memref which would not be a part of its semantics.

This is what I gathered, but the docs did not make this clear. I’m not sure if adding some semantic information for memref in the docs would break any assumptions about the dialect since operations that care about this specify they assume a flat 1D memory

ftynse December 9, 2025, 10:15pm 5

Being indexed in a multi-dimensional way doesn’t necessarily mean the backing memory can’t be linear though. If you have specific examples that would become incorrect, let’s look at them in detail.

Multi-dimensional affine forms are interpreted in row-major fashion.
In absence of an explicit layout, a memref is considered to have a multi-dimensional identity affine map layout.

Interpreting these two sentences from the documentation, if the layout is left as the default, then the addressing of a memref seems to be interpreted as contiguous row-major memory access. If a layout is present, then it should determine how logical indices are mapped to physical locations in memory.

p.s. Before reading the documentation, I thought that the addressing scheme of a memref with the default layout was undefined and left up to the compiler implementation.

This is a very accurate description of what a memref is, without mixing it up with lowerings. It is a reference to memory that can be addressed/derefenced using a multi-dimensional subscript. There is a backing buffer/memory that gets accessed. But what part of that buffer is accessed and how depends on how the memref was created and what the attached layout is.