MLIR Open Meeting: Tensor Compiler WG, 2025-04-29 (original) (raw)

This Tuesday, April 29, (note that weekday change) we will hold a Tensor Compiler Working Group meeting that is open to the public.

The call starts at 9am PDT, 4pm UTC, 6pm CEST.

Zoom Meeting Link

Meeting ID: 851 5109 0498
Passcode: 828404

This meeting replaces the Open Design Meeting that would have been held on Thursday, May 1 as per TIme slots for Open MLIR Meetings

cc: @jpienaar @banach-space @dcaballe @rengolin @kuhar @sjarus @matthias-springer @javedabsar @Groverkss @qed for agenda.

rengolin April 25, 2025, 9:43am 2

Can we add this to the LLVM calendar?

ftynse April 28, 2025, 7:23am 3

Done

Hi all,

Hereby the agenda thus far:

• [Andrzej] make progress on Should We Restrict the Usage of 0-D Vectors in the Vector Dialect? given that relevant RFC participants are in attendance.
• [Matthias] Removing the rollback from the dialect conversion driver. It’s related to tensor compilers insofar as most of them probably use dialect conversions at some point.
• [Alex] maintainer nomination for the components constituting the tensor compiler
• [Rolf] How will we make progress on …
  • the tensor compiler-spanning charter?
  • updating out-dated documentation?
  • [Andrzej] end-to-end examples?
• [Andrzej] go through topics in MLIR TCDG - Vector Dialect: Refactoring + Re-design ideas

The above is copied & pasted from the Tensor Compiler design group’s meetings document: MLIR TCDG - Meetings - Google Docs

Looking forward to seeing you later today!

On tensor compiler, the question that occurred to me during the meeting,

@rengolin do you envision tensor compiler to be an actual compiler (as in “it is able to code generate X”) or as a compiler framework (if I may) which is in a sense MLIR itself? I guess this would ultimately define the charter.

p.s.: perhaps this is a bad place to discuss this but i figured it’s relevant enough to just start questioning here

rengolin April 29, 2025, 6:33pm 6

As a compiler framework. Enough to be able to write E2E tests to show-case the tensor compiler dialects, their lowering, the upstream transforms and how they combine together. This also works as a “documentation” on how to create your own downstream compiler, based on the same infrastructure, without having to reimplement the world.

Writing charter documents is akin to de-juri consensus by theoretical design. Having a framework is like a de-facto standard that encodes the relationships. The former leaves all the implementation details to the user, and we end up with N+1 compilers. The latter encourages all those projects to reuse the code, and to improve them, upstream.

If one day this becomes a proper upstream compiler or not is orthogonal and should be considered once we’re all happy with the infra and have enough bandwidth to do more. But it should not be the first step, or we’ll create unnecessary contention from the start and won’t get anywhere.

Here’s a copy of the notes for today’s meeting:

• No quorum for 0D-vector discussion
• Dialect conversion rollback feature removal procedure?
  • All upstream patterns seem fine without the rollback feature, except for some SPRIV, TOSA, SCF-to-GPU (?) ones.
  • New flag “allow_rollback” with a default value of “true”. This maintains the old behaviour.
  • Downstream users should try to run with “allow_rollback” flag set to “false” and report back if this causes problems.
    * Recommended procedure: globally change the default value in the right file.
  • Next step would be to set the allow_rollback to false upstream.
  • Final step is to remove the flag as well (with the remaining behaviour being that of allow_rollback = false)
  • Post as a deprecation / PSA?
    * Suraj: how about a warning in the code indicating the upcoming deprecation?
    * Alex: maybe, but -Werror is going to cause problems for downstreams - current practice is to expect users to keep up-to-date with deprecation messaging
    * Andrzej: advertising on Discourse should be enough
  • [Matthias]: checking in with developers of a downstreams which are big users, e.g. IREE, to confirm that nothing breaks for them helps be confident a change will not produce many issues
  • Consensus seems to be that we do not need heavier/more formalized processes at this time: this kind of gradual deprecation with messaging on Discourse is sufficient
• Tensor compiler components maintainers
  • Role of maintainer is to be a go-to person for a part of llvm/mlir to answer questions on that component and to be available to help make sure patches get reviewers which are competent w.r.t. that component
  • It needs to be clear who to talk to regarding separate components when someone wants to change that component
  • [Alex]: are there people, especially here, who would like to step up for Tensor Compiler related components (i.e. dialects)?
  • [Renato]: it can be problematic to have singular maintainers for contentious components - group of people being maintainer for a group of dialects might be a better set-up
  • [Renato]: the tensor compiler design group members already are available for discussions on the tensor compiler components
  • [Alex]: the maintainers’ main responsibility is to make sure patches are reviewed by competent people (need not be the maintainers themselves) and RFCs get responses from relevant stakeholders (e.g. maintainers ping stakeholders they know about).
  • [Renato]: for the hardware specific dialects, getting maintainers should be straightforward. Design groups for dialect groupings should be the default for those dialects and else the fallback is the area team.
  • AR: Alex asks everyone in the call/in the Tensor Compiler Design Group if they are okay to be signed up as maintainers for Tensor Compiler dialects and other dialects
• Making progress on charter, documentation, end-to-end examples:
  • [Andrzej]: spent time on getting others’ perspectives on Vector dialect. Upshot: Vector actually seems mostly fine. Difference between Vector and Tensor types is still an interesting topic though.
  • [Andrzej & Alex]: nobody has been bothered enough to work on a charter document. Probably only if someone picks up the reins and starts writing will others be bothered. Maybe.
  • Alex: individual dialect charters are in various states, though in general do specify a rationale, even if a bit outdated
  • The separate dialect-level charters/documentation do not lay out a comprehensive view of how the dialects are meant to fit together though
  • Renato: Note that showing how the pieces fit together does not need to happen via a document - we can document this using tests
  • Andrzej: stoked to have end-to-end examples / integration tests
    * In the sense of “here are the components and here is how you connect them”
  • Renato: the “guiding light” could be a collection of (sub-)schedules and some infrastructure for connecting them together
  • Matthias: integration tests in the form of schedules
    * test/Dialect/Linalg/matmul-shared-memory-padding.mlir
  • It appears many are in favour of more tests that span end-to-end flows or at least a chunk of the tensor compiler dialects
  • AR proposed by Alex: somebody creates a charter document stating “the tensor compiler dialects are …”
    * If nobody else steps in, Alex does it

And, as is tradition, we didn’t get to Andrzej’s vector topics…