Philipp Schmid • 6/20/2023

Securely deploy LLMs inside VPCs with Hugging Face and Amazon SageMaker

This tutorial demonstrates how to securely deploy open-source LLMs like HuggingFaceH4/starchat-beta inside an Amazon VPC using Amazon SageMaker and the Hugging Face LLM Inference Container. It covers setting up the environment, uploading models to S3 for offline access, deploying the model, running inference, and cleanup, specifically for scenarios requiring no internet access from the endpoint.

0 comments

#Hugging Face #Amazon Sagemaker #AWS Vpc