NPU Adaption for FLUX by leisuzz · Pull Request #9751 · huggingface/diffusers (original) (raw)

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service andprivacy statement. We’ll occasionally send you account related emails.

Already on GitHub?Sign in to your account

Conversation17 Commits15 Checks30 Files changed

Conversation

This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters

[ Show hidden characters]({{ revealButtonHref }})

@leisuzz

What does this PR do?

Implement Flash Attention for NPU machine
Add cleaning memory function for NPU machine

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@leisuzz

yiyixuxu

@leisuzz

@leisuzz

@leisuzz

@yiyixuxu @sayakpaul Please let me know if there is any improvement for this PR, and thanks for your help!

@leisuzz

yiyixuxu

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks!

@yiyixuxu

can you run make style and make fix-copies to the CI would pass?

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@leisuzz

@leisuzz

@yiyixuxu I've fixed the code quality issue, please approve the workflow. Thanks

@leisuzz

@leisuzz

@yiyixuxu Sorry about my mistake, could you please approve the workflow? Thanks

@leisuzz

@yiyixuxu

@leisuzz

@leisuzz

@leisuzz

@leisuzz

Hi @yiyixuxu ,
Sorry for the trouble, I think there is something wrong with the ruff in my env as there is no code quality in my end. But I think I've fix it manually. Please approve the workflow. Thanks for your help

@leisuzz

@leisuzz

@yiyixuxu
Could you please redo the approval of the workflow? Thanks for your help and sorry for all the troubles!

@leisuzz

@leisuzz

Hi @yiyixuxu ,
There was a copyright / consistency issue with "def fuse_qkv_projections" as it has to match models.unets.unet_2d_condition.UNet2DConditionModel.fuse_qkv_projections. Does that mean I have to change both of these at the same time? For now, please accept this workflows as fuse_qkv_projections can be added later in another PR.

@leisuzz

@leisuzz

@sayakpaul

Failing tests are unrelated.

a-r-r-o-w pushed a commit that referenced this pull request

Nov 1, 2024

@leisuzz @a-r-r-o-w

Co-authored-by: 蒋硕 jiangshuo9@h-partners.com

sayakpaul pushed a commit that referenced this pull request

Dec 23, 2024

@leisuzz @sayakpaul

Co-authored-by: 蒋硕 jiangshuo9@h-partners.com

@a-r-r-o-w

@yiyixuxu I'm not completely sure why we merged this PR with the following changes:

image

I believe we should default to just the normal SDPA processors otherwise there is inconsistencies introduced. Users can already set custom attention processors, no?

@yiyixuxu