Azure Machine Learning : Llama2 Pay-as-You-Go

I look at a recently added preview feature: *"Deploy Models as a Service"* it is actually setting up a serverless, pay as you go Llama2 LLM api. I take a look at it and give it a try. *How to deploy Llama 2 family of large language models with Azure Machine Learning studio* https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-models-llama?view=azureml-api-2&source=docs#completions-api *Announcing Llama 2 Inference APIs and Hosted Fine-Tuning through Models-as-a-Service in Azure AI* https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/announcing-llama-2-inference-apis-and-hosted-fine-tuning-through/ba-p/3979227 ```python api_url = api_url+"/v1/completions" headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json", } def llama_paygo(prompt): payload = { "prompt": prompt, "temperature": 0.5, "max_tokens": 1024, "top_p": 0.1, } response = requests.post(api_url, json=payload, headers=headers) response_json = response.json() generated_text = response_json["choices"][0]["text"] formatted_text = generated_text.replace("\\n", "\n") print("formatted_text") print(formatted_text) ````

Azure Machine Learning : Llama2 Pay-as-You-Go

I look at a recently added preview feature: *"Deploy Models as a Service"* it is actually setting up a serverless, pay as you go Llama2 LLM api. I take a look at it and give it a try. *How to deploy Llama 2 family of large language models with Azure Machine Learning studio* https://learn.microsoft.com/en-us/azure/machine-learning/how-to-deploy-models-llama?view=azureml-api-2&source=docs#completions-api *Announcing Llama 2 Inference APIs and Hosted Fine-Tuning through Models-as-a-Service in Azure AI* https://techcommunity.microsoft.com/t5/ai-machine-learning-blog/announcing-llama-2-inference-apis-and-hosted-fine-tuning-through/ba-p/3979227 ```python api_url = api_url+"/v1/completions" headers = { "Authorization": f"Bearer {api_key}", "Content-Type": "application/json", } def llama_paygo(prompt): payload = { "prompt": prompt, "temperature": 0.5, "max_tokens": 1024, "top_p": 0.1, } response = requests.post(api_url, json=payload, headers=headers) response_json = response.json() generated_text = response_json["choices"][0]["text"] formatted_text = generated_text.replace("\\n", "\n") print("formatted_text") print(formatted_text) ````