CUDA: FA support for Deepseek (Ampere or newer) by JohannesGaessler · Pull Request #13306 · ggml-org/llama.cpp (original) (raw)
Issues specific to Nvidia GPUs
python script changes
changes relating to the ggml tensor library for machine learning
labels
that may beclosed by this pull request
gabe-l-hart added a commit to gabe-l-hart/llama.cpp that referenced this pull request
- origin/master: (39 commits) server : vision support via libmtmd (ggml-org#12898) sycl : implementation of reordered Q4_0 MMVQ for Intel GPUs (ggml-org#12858) metal : optimize MoE for large batches (ggml-org#13388) CUDA: FA support for Deepseek (Ampere or newer) (ggml-org#13306) llama : do not crash if there is no CPU backend (ggml-org#13395) CUDA: fix crash on large batch size for MoE models (ggml-org#13384) imatrix : Add --parse-special for enabling parsing of special tokens in imatrix calculation (ggml-org#13389) llama-run: add support for downloading models from ModelScope (ggml-org#13370) mtmd : fix batch_view for m-rope (ggml-org#13397) llama : one-off chat template fix for Mistral-Small-2503 (ggml-org#13398) rpc : add rpc_msg_set_tensor_hash_req (ggml-org#13353) vulkan: Allow up to 4096 elements for mul_mat_id row_ids (ggml-org#13326) server : (webui) rename has_multimodal --> modalities (ggml-org#13393) ci : limit write permission to only the release step + fixes (ggml-org#13392) mtmd : Expose helper_decode_image_chunk (ggml-org#13366) server : (webui) fix a very small misalignment (ggml-org#13387) server : (webui) revamp the input area, plus many small UI improvements (ggml-org#13365) convert : support rope_scaling type and rope_type (ggml-org#13349) mtmd : fix the calculation of n_tokens for smolvlm (ggml-org#13381) context : allow cache-less context for embeddings (ggml-org#13108) ...
Nexesenex added a commit to Nexesenex/croco.cpp that referenced this pull request
Nexesenex added a commit to Nexesenex/croco.cpp that referenced this pull request
Update fattn-mma-f16.cuh
Seunghhon pushed a commit to Seunghhon/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
ljubomirj pushed a commit to ljubomirj/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
my-other-github-account pushed a commit to my-other-github-account/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
phibya pushed a commit to ziee-ai/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
fewtarius pushed a commit to fewtarius/CachyLLama that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
AlexiAlp pushed a commit to minghaop/llama.cpp that referenced this pull request
CUDA: FA support for Deepseek (Ampere or newer)
do loop unrolling via C++ template
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.Learn more about bidirectional Unicode characters
[ Show hidden characters]({{ revealButtonHref }})