Create Customization Target — NVIDIA NeMo Microservices (original) (raw)

Prerequisites#

Before you can create a customization target, make sure that you have:

• Access to the NeMo Customizer service
• The address of the base model you want to customize (model_uri)
  • If using a HuggingFace-style (hf://) model_uri, you must provide a HuggingFace endpoint (such as https://huggingface.co); if not specified, the Data Store’s HF-compatible endpoint is used (http://<datastore-host>/v1/hf)
• Review the customization target value reference guide to obtain the required values for the model you want to use.

Options#

You can create a customization target in the following ways.

API#

1. Submit a POST request to /v1/customization/targets.
   cURL
   curl -X POST \
   "${CUSTOMIZER_SERVICE_URL}/v1/customization/targets" \
   -H 'accept: application/json' \
   -H 'Content-Type: application/json' \
   -d '{
   "name": "llama-3.1-8b-instruct@2.0",
   "namespace": "meta",
   "description": "Customization target for Meta Llama 3.1 8B",
   "enabled": true,
   "base_model": "meta/llama-3.1-8b",
   "model_path": "llama-3_1-8b-instruct_2_0",
   "model_uri": "ngc://nvidia/nemo/llama-3_1-8b:2.0",
   "hf_endpoint": "",
   "tokenizer": {},
   "num_parameters": 123456789,
   "precision": "bf16-mixed",
   "project": "urn:your-project-id"
   }' | jq
2. Review the response.
   {
   "id": "cust-target-abc123",
   "name": "llama-3.1-8b-instruct@2.0",
   "namespace": "meta",
   "description": "Customization target for Meta Llama 3.1 8B",
   "enabled": true,
   "base_model": "meta/llama-3.1-8b",
   "model_path": "llama-3_1-8b-instruct_2_0",
   "model_uri": "ngc://nvidia/nemo/llama-3_1-8b:2.0",
   "hf_endpoint": "",
   "tokenizer": {},
   "num_parameters": 123456789,
   "precision": "bf16-mixed",
   "status": "created",
   "ownership": {},
   "custom_fields": {},
   "created_at": "2024-05-08T12:00:00Z",
   "updated_at": "2024-05-08T12:00:00Z"
   }
   Tip
   Model files are large and can take time to download from the model_uri. You can check the status of the download by getting target details and reviewing the status.
   Note
   If you’re re-creating a target with the same name after deleting it, ensure that the model download job for the deleted target doesn’t already exist. The model download job has a TTL of 10 minutes and is automatically removed after this period expires.