GitHub - mlc-ai/binary-mlc-llm-libs (original) (raw)

Skip to content

Navigation Menu

Sign in

Appearance settings

• • GitHub Copilot Write better code with AI
  • GitHub Models New Manage and compare prompts
  • GitHub Advanced Security Find and fix vulnerabilities
  • Actions Automate any workflow
  • Codespaces Instant dev environments
  • Issues Plan and track work
  • Code Review Manage code changes
  • Discussions Collaborate outside of code
  • Code Search Find more, search less
    Explore
  • Why GitHub
  • All features
  • Documentation
  • GitHub Skills
  • Blog
• By company size
  • Enterprises
  • Small and medium teams
  • Startups
  • Nonprofits
    By use case
  • DevSecOps
  • DevOps
  • CI/CD
  • View all use cases
    By industry
  • Healthcare
  • Financial services
  • Manufacturing
  • Government
  • View all industries
    View all solutions
• Topics
  • AI
  • DevOps
  • Security
  • Software Development
  • View all
    Explore
  • Learning Pathways
  • Events & Webinars
  • Ebooks & Whitepapers
  • Customer Stories
  • Partners
  • Executive Insights
• • GitHub Sponsors Fund open source developers
  • The ReadME Project GitHub community articles
    Repositories
  • Topics
  • Trending
  • Collections
• • Enterprise platform AI-powered developer platform
    Available add-ons
  • GitHub Advanced Security Enterprise-grade security features
  • Copilot for business Enterprise-grade AI features
  • Premium Support Enterprise-grade 24/7 support
• Pricing

Provide feedback

We read every piece of feedback, and take your input very seriously.

Include my email address so I can be contacted

Saved searches

Use saved searches to filter your results more quickly

Sign in

Sign up

Appearance settings

mlc-ai / binary-mlc-llm-libs Public

• Notifications You must be signed in to change notification settings
• Fork54
• Star 244

244 stars 54 forks Branches Tags Activity

Star

Notifications You must be signed in to change notification settings

• Code
• Issues 23
• Pull requests
• Actions
• Projects
• Security
• Insights

Additional navigation options

BranchesTags

Folders and files

Repository files navigation

• README

binary-mlc-llm-libs

Model libraries are stored in the format:

{model_name}/{model_name}-{quantization}-{metadata}-{platform}.{suffix}

Metadata:

• ctx: context window size
• sw: sliding window size
• cs: prefill chunk size

For default configurations of metadata, we do not include that in the file name. We also do not include prefill chunk size if it is the same as the context window size or sliding window size (the default choice).

Default Metadata

| | Context Window Size | Sliding Window Size | Prefill Chunk Size | | | --------------------------- | ------------------- | ------------------ | ---- | | Llama-3-8b-Instruct | 8192 | N/A | 1024 | | Llama-3-70b-Instruct | 8192 | N/A | 1024 | | Llama-2-7b-chat-hf | 4096 | N/A | 4096 | | Llama-2-13b-chat-hf | 4096 | N/A | 4096 | | Llama-2-70b-chat-hf | 4096 | N/A | 4096 | | Mistral-7B-Instruct-v0.2 | N/A | 4096 | 4096 | | RedPajama-INCITE-Chat-3B-v1 | 2048 | N/A | 2048 | | phi-2 | 2048 | N/A | 2048 | | phi-1_5 | 2048 | N/A | 2048 | | gpt2 | 1024 | N/A | 1024 | | gpt2-medium | 1024 | N/A | 1024 |

About

No description, website, or topics provided.

Resources

Readme

Activity

Custom properties

Stars

244 stars

Watchers

15 watching

Forks

54 forks

Report repository

Releases9

Android-09262024 Latest Sep 26, 2024

+ 8 releases

Packages

No packages published

Contributors20

+ 6 contributors