Implement multimodal models (LLaVA) by monatis · Pull Request #3436 · ggml-org/llama.cpp (original) (raw)

yup.. can confirm following #2948 doesn't yield valid llava-v1.6-mistral-7b-GGUF... any suggestions?

$ python llama.cpp/convert.py llava-hf \
>   --outfile llava-v1.6-mistral-7b-GGUF.gguf \
>   --outtype f32
Loading model file llava-hf\model-00001-of-00004.safetensors
Loading model file llava-hf\model-00001-of-00004.safetensors
Loading model file llava-hf\model-00002-of-00004.safetensors
Loading model file llava-hf\model-00003-of-00004.safetensors
Loading model file llava-hf\model-00004-of-00004.safetensors
params = Params(n_vocab=32000, n_embd=4096, n_layer=32, n_ctx=32768, n_ff=14336, n_head=32, n_head_kv=8, n_experts=None, n_experts_used=None, f_norm_eps=1e-05, rope_scaling_type=None, f_rope_freq_base=1000000.0, f_rope_scale=None, n_orig_ctx=None, rope_finetuned=None, ftype=<GGMLFileType.AllF32: 0>, path_model=WindowsPath('llava-hf'))Found vocab files: {'tokenizer.model': WindowsPath('llava-hf/tokenizer.model'), 'vocab.json': None, 'tokenizer.json': WindowsPath('llava-hf/tokenizer.json')}
Loading vocab file 'llava-hf\tokenizer.model', type 'spm'
Vocab info: <SentencePieceVocab with 32000 base tokens and 0 added tokens>
Special vocab info: <SpecialVocab with 0 merges, special tokens {'bos': 1, 'eos': 2, 'unk': 0, 'pad': 0}, add special tokens {'bos': True, 'eos': False}>
Permuting layer 0
Permuting layer 1
Permuting layer 2
Permuting layer 3
Permuting layer 4
Permuting layer 5
Permuting layer 6
Permuting layer 7
Permuting layer 8
Permuting layer 9
Permuting layer 10
Permuting layer 11
Permuting layer 12
Permuting layer 13
Permuting layer 14
Permuting layer 15
Permuting layer 16
Permuting layer 17
Permuting layer 18
Permuting layer 19
Permuting layer 20
Permuting layer 21
Permuting layer 22
Permuting layer 23
Permuting layer 24
Permuting layer 25
Permuting layer 26
Permuting layer 27
Permuting layer 28
Permuting layer 29
Permuting layer 30
Permuting layer 31
model.embed_tokens.weight                        -> token_embd.weight                        | BF16   | [32000, 4096]
Traceback (most recent call last):
  File "F:\SANDBOX\convert_llava\llama.cpp\convert.py", line 1474, in <module>
    main()
  File "F:\SANDBOX\convert_llava\llama.cpp\convert.py", line 1460, in main
    model   = convert_model_names(model, params)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\SANDBOX\convert_llava\llama.cpp\convert.py", line 1198, in convert_model_names
    raise Exception(f"Unexpected tensor name: {name}")
Exception: Unexpected tensor name: model.image_newline
(llama-new)