How To Run DeepSeek R1 on AWS Using Infrastructure as Code
This weekend, I changed my perspective on open source AI deployment. While scrolling through my social feeds, I noticed many posts about DeepSeek, a new open-source language model, causing a stir in the AI community. As someone who regularly deploys infrastructure for production environments, I was intrigued by DeepSeek’s promise of competitive performance at a fraction of the cost of major commercial models.
What caught my attention wasn’t just the benchmark numbers. However, DeepSeek’s 79.8% score on AIME 2024 mathematics tests is impressive, but rather the practical possibility of running these models on standard cloud infrastructure. I decided to put this to the test by deploying DeepSeek on AWS using Pulumi for infrastructure as code. Here’s what I learned from the experience.
Understanding DeepSeek’s Place in the AI Landscape
DeepSeek emerged from a Chinese AI startup founded in 2023. It brings something unique: High-performance language models released under the MIT license. While companies like OpenAI and Meta spend enormous resources on their models, DeepSeek achieves comparable results with significantly less investment.
Source: DeepSeek
In my testing, DeepSeek R1 demonstrated capabilities that make it particularly valuable for practical applications:
- Mathematics processing with 79.8% accuracy on AIME 2024 tests.
- Software engineering tasks with 49.2% accuracy on SWE-bench Verified.
- General knowledge handling with a 90.8% score on MMLU.
What makes this especially interesting for development teams is the availability of distilled versions with 1.5B to 70B parameters, allowing deployment on various hardware configurations, from local machines to cloud instances.
Deploying DeepSeek: A Practical Infrastructure Approach
After evaluating DeepSeek’s capabilities, I created a reproducible deployment process using Pulumi and AWS. The goal was to establish a GPU-powered environment that could efficiently handle the model while remaining cost-effective.
The deployment architecture I developed consists of three main components:
- A GPU-enabled EC2 instance (g4dn.xlarge) for model hosting.
- Ollama for model management and API compatibility.
- Open WebUI for interaction and testing.
Here’s the real-world deployment process I developed, focusing on maintainability and scalability:
Prerequisites
Before embarking on our self-hosted DeepSeek model journey, ensure you have:
- An AWS account;
- Pulumi CLI installed;
- AWS CLI installed;
- A basic understanding of Ollama, a tool that simplifies running large language models (LLMs) on your hardware.
Creating the Infrastructure
To get started, I created a new Pulumi project:
pulumi new aws-typescript
I chose TypeScript for this example, but you can select any language you prefer.
After setting up the project, I deleted the sample code and replaced it with the following configurations.
Create an Instance Role With S3 Access
To download the NVIDIA drivers, I needed to create an instance role with S3 access (AmazonS3ReadOnlyAccess is enough here)
import * as pulumi from "@pulumi/pulumi";
import * as aws from "@pulumi/aws";
import * as fs from "fs";
const role = new aws.iam.Role("deepSeekRole", {
name: "deepseek-role",
assumeRolePolicy: JSON.stringify({
Version: "2012-10-17",
Statement: [
{
Action: "sts:AssumeRole",
Effect: "Allow",
Principal: {
Service: "ec2.amazonaws.com",
},
},
],
}),
});
new aws.iam.RolePolicyAttachment("deepSeekS3Policy", {
policyArn: "arn:aws:iam::aws:policy/AmazonS3ReadOnlyAccess",
role: role.name,
});
const instanceProfile = new aws.iam.InstanceProfile("deepSeekProfile", {
name: "deepseek-profile",
role: role.name,
});
Create the Network
Next, I needed to create a VPC, subnet, Internet Gateway, and route table. This is all done with the following code snippet:
const vpc = new aws.ec2.Vpc("deepSeekVpc", {
cidrBlock: "10.0.0.0/16",
enableDnsHostnames: true,
enableDnsSupport: true,
});
const subnet = new aws.ec2.Subnet("deepSeekSubnet", {
vpcId: vpc.id,
cidrBlock: "10.0.48.0/20",
availabilityZone: pulumi.interpolate`${aws.getAvailabilityZones().then(it => it.names[0])}`,
mapPublicIpOnLaunch: true,
});
const internetGateway = new aws.ec2.InternetGateway("deepSeekInternetGateway", {
vpcId: vpc.id,
});
const routeTable = new aws.ec2.RouteTable("deepSeekRouteTable", {
vpcId: vpc.id,
routes: [
{
cidrBlock: "0.0.0.0/0",
gatewayId: internetGateway.id,
},
],
});
const routeTableAssociation = new aws.ec2.RouteTableAssociation("deepSeekRouteTableAssociation", {
subnetId: subnet.id,
routeTableId: routeTable.id,
});
const securityGroup = new aws.ec2.SecurityGroup("deepSeekSecurityGroup", {
vpcId: vpc.id,
egress: [
{
fromPort: 0,
toPort: 0,
protocol: "-1",
cidrBlocks: ["0.0.0.0/0"],
},
],
ingress: [
{
fromPort: 22,
toPort: 22,
protocol: "tcp",
cidrBlocks: ["0.0.0.0/0"],
},
{
fromPort: 3000,
toPort: 3000,
protocol: "tcp",
cidrBlocks: ["0.0.0.0/0"],
},
{
fromPort: 11434,
toPort: 11434,
protocol: "tcp",
cidrBlocks: ["0.0.0.0/0"],
},
],
});
Create the EC2 Instance
Finally, I can create the EC2 instance. For this, I need to create a SSH key pair and retrieve the Amazon Machine Images to use in our instances.
I also use a g4dn.xlarge, but you can change the instance type to any other instance type that supports GPU. You can find more information about the instance types.
If you need to create the key pair, run the following command:
openssl genrsa -out deepseek.pem 2048 openssl rsa -in deepseek.pem -pubout > deepseek.pub ssh-keygen -f mykey.pub -i -mPKCS8 > deepseek.pem
const keyPair = new aws.ec2.KeyPair("deepSeekKey", {
publicKey: pulumi.output(fs.readFileSync("deepseek.rsa", "utf-8")),
});
const deepSeekAmi = aws.ec2
.getAmi({
filters: [
{
name: "name",
values: ["amzn2-ami-hvm-2.0.*-x86_64-gp2"],
},
{
name: "architecture",
values: ["x86_64"],
},
],
owners: ["137112412989"], // Amazon
mostRecent: true,
})
.then(ami => ami.id);
const deepSeekInstance = new aws.ec2.Instance("deepSeekInstance", {
ami: deepSeekAmi,
instanceType: "g4dn.xlarge",
keyName: keyPair.keyName,
rootBlockDevice: {
volumeSize: 100,
volumeType: "gp3",
},
subnetId: subnet.id,
vpcSecurityGroupIds: [securityGroup.id],
iamInstanceProfile: instanceProfile.name,
userData: fs.readFileSync("cloud-init.yaml", "utf-8"),
tags: {
Name: "deepSeek-server",
},
});
export const amiId = deepSeekAmi;
export const instanceId = deepSeekInstance.id;
export const instancePublicIp = deepSeekInstance.publicIp;
Then, we configure the GPU instance with proper drivers and dependencies, install Ollama and run DeepSeek with this cloud config.
#cloud-config users: - default package_update: true packages: - apt-transport-https - ca-certificates - curl - openjdk-17-jre-headless - gcc runcmd: - yum install -y gcc kernel-devel-$(uname -r) - aws s3 cp --recursive s3://ec2-linux-nvidia-drivers/latest/ . - chmod +x NVIDIA-Linux-x86_64*.run - /bin/sh ./NVIDIA-Linux-x86_64*.run --tmpdir . --silent - touch /etc/modprobe.d/nvidia.conf - echo "options nvidia NVreg_EnableGpuFirmware=0" | sudo tee --append /etc/modprobe.d/nvidia.conf - yum install -y docker - usermod -a -G docker ec2-user - systemctl enable docker.service - systemctl start docker.service - curl -s -L https://nvidia.github.io/libnvidia-container/stable/rpm/nvidia-container-toolkit.repo | sudo tee /etc/yum.repos.d/nvidia-container-toolkit.repo - yum install -y nvidia-container-toolkit - nvidia-ctk runtime configure --runtime=docker - systemctl restart docker - docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama --restart always ollama/ollama - sleep 120 - docker exec ollama ollama run deepseek-r1:7b - docker exec ollama ollama run deepseek-r1:14b - docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
Deploying the Infrastructure
With all configurations in place, we can deploy the infrastructure using:
pulumi up
This command provides a preview of the changes, allowing you to confirm before proceeding. Once confirmed, Pulumi creates the resources, and after some time, the EC2 instance is ready with DeepSeek R1 running.
Access the Ollama Web UI
To retrieve the public IP address of our EC2 instance, I used the following command to instruct Pulumi to print out the configuration:
pulumi stack output instancePublicIp contains the code and configuration files <yout-ip>
I then opened web UI with this address: http://<ip>:3000/
Use DeepSeek R1
Head to the dropdown in the upper right corner and select the model you want to use.
I selected deepseek-r1:14b to test my model.
Finally, I used the central chat box to begin using the model. My example prompt is: What are Pulumi’s benefits?
Cleaning up
After you are done experimenting with DeepSeek, I clean up the resources by running the following command:
pulumi destroy
Looking Forward
DeepSeek represents a significant step forward in accessible AI deployment. The combination of MIT licensing and competitive performance could make it a viable option for production environments.
For teams considering DeepSeek deployment, I recommend:
- Starting with the 7B model for a balanced performance/resource ratio.
- Using infrastructure as code (like Pulumi) for reproducible deployments.
- Implementing proper monitoring and scaling policies.
- Testing thoroughly with production-like workloads before deployment.
My GitHub repository contains the code and configuration files from this deployment, allowing others to build upon this foundation for their AI infrastructure needs.
This experience has shown me that enterprise-grade AI deployment is increasingly accessible to smaller teams. As we continue to see advances in model efficiency and deployment tools, the barrier to entry for production AI will continue to lower, opening new possibilities for innovation across the industry.
If you’re interested in exploring AI models or need a robust setup for your projects, consider trying DeepSeek with Pulumi. Remember, while the setup is straightforward, securing your instance and understanding the model’s capabilities are crucial steps before going live.
