Skip to content

Commit c22541f

Browse files
mariia126mariia126
authored
Merge pull request #4 from resizes/sftpgo
feat: new SFTPGO blog post
2 parents 900d784 + 1825734 commit c22541f

File tree

4 files changed

+323
-0
lines changed

4 files changed

+323
-0
lines changed

‎blog/2025-02-17-sftp/URLimage.png

523 KB
Loading

‎blog/2025-02-17-sftp/index.md

Lines changed: 317 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,317 @@
1+
---
2+
slug: sftp-server-in-kubernetes-with-sftpgo
3+
title: SFTP server in Kubernetes with SFTPGo
4+
authors: maría
5+
tags: [DevOps, PlatformEngineering, SFTPGo, Kubernetes, DeveloperExperience, ExternalSecrets, Terraform, AWS]
6+
image: URLimage.png
7+
---
8+
9+
Have you ever needed to create an SFTP server? How do you do it?
10+
11+
For those who have created an SFTP server before, you probably know that it is not easy to create and maintain an SFTP server. There are many ways to do it, but in this case, we are going to install SFTPGo in our Kubernetes cluster.
12+
13+
SFTPGo is an open-source SFTP server that allows users to securely transfer files over SSH. It is written in Go (Golang) and is designed to be lightweight, easy to configure, and highly customizable. It supports multiple storage backends, including local filesystems, cloud storage (like S3, Google Cloud Storage, etc.), and more.
14+
15+
The deployment of SFTPGo on an EKS cluster begins with provisioning the required resources, so let's start by creating the necessary infrastructure with Terraform.
16+
17+
<!--truncate-->
18+
19+
# External Secret
20+
21+
For managing secrets in EKS, we are using [External Secrets Operator](https://external-secrets.io/). To store the secrets for our SFTP server, we will use AWS Secrets Manager. An external secret is a Kubernetes resource that allows you to manage secrets from an external secret manager, in this case, AWS Secrets Manager.
22+
23+
Here is the code to create the secret in AWS Secrets Manager:
24+
25+
```hcl
26+
resource "aws_secretsmanager_secret" "sftpgo" {
27+
name = "sftpgo"
28+
description = "Secrets for sftpgo in EKS production cluster"
29+
}
30+
```
31+
32+
Once created, we will need to enter the following secrets:
33+
34+
- `SFTPGO_DEFAULT_ADMIN_USERNAME`: example_user
35+
- `SFTPGO_DEFAULT_ADMIN_PASSWORD`: example_password
36+
- `SFTPGO_DATA_PROVIDER__DRIVER`: postgresql
37+
- `SFTPGO_DATA_PROVIDER__NAME`: sftpgo.db
38+
- `SFTPGO_DATA_PROVIDER__HOST`: sftpgo-postgresql.sftpgo.svc.cluster.local *(this might change, it depends on your needs!)*
39+
- `SFTPGO_DATA_PROVIDER__PORT`: 5432
40+
- `SFTPGO_DATA_PROVIDER__USERNAME`: sftpgo
41+
- `SFTPGO_DATA_PROVIDER__PASSWORD`: sftpgo_pg_pwd
42+
43+
As you can see, we are using a PostgreSQL database to store the users and the configuration for our SFTP server. And of course, we need to create the database. We will do this in the next section.
44+
45+
# SFTPGo resources
46+
47+
On the other hand, it is necessary to create any other resource related to this new SFTP (policies, IAM role, permissions, etc.).
48+
49+
We will create one role to access the **AWS Secrets Manager** and another to **access an S3 bucket**.
50+
51+
> The **s3 bucket** is used for storing the documents managed in the SFTP. We can use a single bucket for everything, but it is possible to use multiple buckets. Each user inside the SFTP can have access to a different bucket, or even a different folder inside the same bucket.
52+
53+
Our code would look something like this:
54+
55+
```hcl
56+
data "aws_iam_policy_document" "sftpgo" {
57+
statement {
58+
actions = [
59+
"secretsmanager:GetResourcePolicy",
60+
"secretsmanager:GetSecretValue",
61+
"secretsmanager:DescribeSecret",
62+
"secretsmanager:ListSecretVersionIds"
63+
]
64+
resources = [aws_secretsmanager_secret.sftpgo.arn]
65+
}
66+
}
67+
68+
resource "aws_iam_policy" "sftpgo" {
69+
name = "sftpgo"
70+
path = "/"
71+
description = "Policy to get sftpgo secrets"
72+
policy = data.aws_iam_policy_document.sftpgo.json
73+
}
74+
75+
resource "aws_iam_role" "sftpgo" {
76+
name = "external-secrets-sftpgo"
77+
78+
assume_role_policy = jsonencode({
79+
Version = "2012-10-17"
80+
Statement = [
81+
{
82+
Action = "sts:AssumeRole"
83+
Effect = "Allow"
84+
Principal = {
85+
AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/external-secrets"
86+
}
87+
}
88+
]
89+
})
90+
}
91+
92+
resource "aws_iam_policy_attachment" "sftpgo" {
93+
name = "sftpgo"
94+
roles = [aws_iam_role.sftpgo.name]
95+
policy_arn = aws_iam_policy.sftpgo.arn
96+
}
97+
98+
resource "aws_s3_bucket" "sftpgo" {
99+
bucket = "sftpgo" # TODO: change this to the name of the bucket you want to use
100+
}
101+
102+
data "aws_iam_policy_document" "s3_full_access" {
103+
statement {
104+
actions = [
105+
"s3:*",
106+
]
107+
resources = [
108+
aws_s3_bucket.sftpgo.arn,
109+
"${aws_s3_bucket.sftpgo.arn}/*",
110+
]
111+
}
112+
}
113+
114+
resource "aws_iam_policy" "s3_full_access" {
115+
name = "S3AccessPolicy-${aws_s3_bucket.sftpgo.bucket}"
116+
policy = data.aws_iam_policy_document.s3_full_access.json
117+
}
118+
119+
resource "aws_iam_role" "irsa_role" {
120+
name = "sftpgo"
121+
122+
assume_role_policy = jsonencode({
123+
Version = "2012-10-17"
124+
Statement = [
125+
{
126+
Effect = "Allow"
127+
Principal = {
128+
Federated = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:oidc-provider/${replace(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://", "")}"
129+
}
130+
Action = "sts:AssumeRoleWithWebIdentity"
131+
Condition = {
132+
StringEquals = {
133+
"${replace(data.aws_eks_cluster.cluster.identity[0].oidc[0].issuer, "https://", "")}:sub" = "system:serviceaccount:sftpgo:sftpgo"
134+
}
135+
}
136+
}
137+
]
138+
})
139+
}
140+
141+
resource "aws_iam_role_policy_attachment" "attach_s3_policy" {
142+
role = aws_iam_role.irsa_role.name
143+
policy_arn = aws_iam_policy.s3_full_access.arn
144+
}
145+
146+
resource "aws_s3_bucket_public_access_block" "sftpgo" {
147+
bucket = aws_s3_bucket.sftpgo.id
148+
149+
block_public_acls = true
150+
block_public_policy = true
151+
ignore_public_acls = true
152+
restrict_public_buckets = true
153+
}
154+
```
155+
156+
>**¿What´s the IRSA Role?**: This Terraform code creates an IAM role that can be assumed by a ServiceAccount in an EKS cluster via IRSA. The role has a policy attached to it that allows access to S3. This is useful for applications running on Kubernetes that need to access AWS resources, such as S3, securely and without needing to store credentials directly on the cluster.
157+
158+
To continue, we must also add a new data reference to our EKS cluster in our `data.tf`:
159+
160+
```hcl
161+
data "aws_eks_cluster" "cluster" {
162+
name = "example_name_cluster" # TODO: change this to the name of your cluster
163+
}
164+
```
165+
166+
---
167+
168+
# SFTPGo Helm Chart
169+
170+
When everything mentioned above has been created, we can continue creating the necessary chart to set up our SFTP server in Kubernetes.
171+
172+
Let's start by creating a new folder called `sftpgo`. Inside this folder, we will begin by creating the two main files: `values.yaml` and `Chart.yaml`.
173+
174+
### Chart.yaml
175+
176+
here will be listed the different dependencies that we will be using and their versions. In this case, **sftpgo** and **postgresql**.
177+
178+
>**PostgreSQL** is required to store the new users who will use this SFTP and the configuration for the SFTP server.
179+
180+
It should look something like this:
181+
182+
```yml
183+
apiVersion: v2
184+
name: sftpgo
185+
description: SFTPGo - Secure SFTP Server
186+
type: application
187+
version: 0.23.1
188+
appVersion: "2.5.4"
189+
dependencies:
190+
- name: sftpgo
191+
version: 0.23.1
192+
repository: "https://charts.sagikazarmark.dev"
193+
- name: postgresql
194+
version: 16.4
195+
repository: "https://charts.bitnami.com/bitnami"
196+
```
197+
198+
### Values.yaml
199+
200+
On the other hand, there is the `values.yaml`, where the ingress, the serviceAccount, variables (in this case, retrieved from a secret stored in AWS) etc. are collected.
201+
202+
It should looks like this:
203+
204+
```yml
205+
sftpgo:
206+
config:
207+
common:
208+
proxy_protocol: 1
209+
data_provider:
210+
create_default_admin: true
211+
212+
envFrom:
213+
- secretRef:
214+
name: sftpgo
215+
216+
serviceAccount:
217+
annotations:
218+
eks.amazonaws.com/role-arn: arn:aws:iam::ACCOUNT_ID:role/sftpgo # TODO: change this to the IRSA role
219+
220+
ui:
221+
ingress:
222+
enabled: true
223+
className: external
224+
annotations:
225+
kubernetes.io/external-dns.create: "true"
226+
hosts:
227+
- host: sftpgo.example.com # TODO: change this to the domain you want to use
228+
paths:
229+
- path: /
230+
pathType: ImplementationSpecific
231+
```
232+
233+
Once we have these two files, we will create the `Chart.lock` and the `charts` with the following command:
234+
235+
```sh
236+
helm dep up
237+
```
238+
239+
With this, a folder with the Helm Chart dependencies called `charts` and a `Chart.lock` file should have been created inside our `sftpgo` folder.
240+
241+
A folder called **templates** is also needed. There will be created both `externalsecret.yaml` and `secretstore.yaml`. This will allow us to manage the secrets we have previously stored in our **AWS Secret Manager**.
242+
243+
### secretstore.yaml
244+
245+
The secret store is necessary to be able to manage our secrets. Its code would look something like the following:
246+
247+
```yml
248+
apiVersion: external-secrets.io/v1beta1
249+
kind: SecretStore
250+
metadata:
251+
name: external-secrets-sftpgo
252+
spec:
253+
provider:
254+
aws:
255+
service: SecretsManager
256+
role: arn:aws:iam::ACCOUNT_ID:role/external-secrets-sftpgo # TODO: change this to the IRSA role
257+
region: us-east-1 # TODO: change this to the region of your cluster
258+
```
259+
260+
### externalsecret.yaml
261+
262+
Finally, the `externalsecret.yaml` file that contains all the secrets we mentioned earlier.
263+
264+
```yml
265+
apiVersion: external-secrets.io/v1beta1
266+
kind: ExternalSecret
267+
268+
metadata:
269+
name: sftpgo
270+
namespace: sftpgo
271+
272+
spec:
273+
refreshInterval: "10m"
274+
secretStoreRef:
275+
name: external-secrets-sftpgo
276+
kind: SecretStore
277+
278+
target:
279+
name: sftpgo
280+
281+
dataFrom:
282+
- extract:
283+
key: sftpgo
284+
```
285+
286+
# Let's test it!
287+
288+
With all our resources created, we can prove that everything is going well with several steps:
289+
290+
### 1. Enter the SFTPGo host
291+
We can enter the host that we created earlier in the `values.yaml` file (sftpgo.example.com) and see a login screen like the one in the following image. With what we have done, it should be possible to log in as the admin user using the credentials stored in our AWS Secret Manager.
292+
293+
![Sftpgo Login](sftpgo_login.png)
294+
295+
### 2. Send a file from our local to SFTPGo.
296+
>**Remember**: We are doing everything from a user login, so we first need to create a user in the SFTPGo UI.
297+
298+
To save a test file in SFTPGo, go to our terminal and log in with the following command:
299+
300+
```sh
301+
sftp [email protected]
302+
```
303+
304+
Later, after creating a test .txt file, we will save it in SFTPGo inside a folder called "example" *(It's can also be saved directly in the root directory; it's just a test to see the possibility of navigating and organizing files within SFTPGo)*.
305+
306+
```sh
307+
sftp> mkdir /example
308+
sftp> cd /home/example
309+
sftp> put test_file.txt
310+
```
311+
312+
If everything has gone well, it should be possible to see our file in the SFTPGo UI and if you go to the S3 bucket, you should also be able to see the file stored there.
313+
314+
# Resources
315+
316+
- [SFTPGo Docs](https://docs.sftpgo.com/latest/tutorials/postgresql-s3/)
317+
- [SFTPGo Helm Chart](https://artifacthub.io/packages/helm/sagikazarmark/sftpgo)

‎blog/2025-02-17-sftp/sftpgo_login.png

75.5 KB
Loading

‎blog/authors.yml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,3 +15,9 @@ guille:
1515
title: Platform Engineer
1616
url: https://guillermotti.com
1717
image_url: https://github.com/guillermotti.png
18+
19+
maria:
20+
name: María García
21+
title: Junior Platform Engineer
22+
url: https://github.com/mariia126
23+
image_url: https://github.com/mariia126.png

0 commit comments

Comments
 (0)