-
Notifications
You must be signed in to change notification settings - Fork 50
Documentation about k8s on a origin #10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Changes from all commits
73c1db4
b4c487c
63d50e4
f2b9415
b79ffb3
e75b4f9
a1443e0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
title: Running OSDF Origin in a Container | ||
DateReviewed: 2020-06-22 | ||
title: Running OSDF Origin in a Container (Docker and Kubernetes) | ||
DateReviewed: 2022-06-27 | ||
|
||
Running OSDF Origin in a Container | ||
======================================== | ||
|
@@ -25,13 +25,14 @@ Before Starting | |
|
||
Before starting the installation process, consider the following points: | ||
|
||
1. **Docker:** For the purpose of this guide, the host must have a running docker service and you must have the ability | ||
1. **Registration:** Before deploying an origin, you must | ||
[registered the service](install-origin.md#registering-the-origin) in the OSG Topology | ||
1. **Kubernetes or docker:** For the purpose of this guide, the host must have a running Kubernets or docker service and you must have the ability | ||
to start containers (i.e., belong to the `docker` Unix group). | ||
1. **Network ports:** The origin listens for incoming HTTP/S and XRootD connections on ports 1094 and 1095 (by | ||
default). | ||
default) and 9993 for the monitoring streams. | ||
1. **File Systems:** The origin needs a host partition to store user data. | ||
1. **Registration:** Before deploying an origin, you must | ||
[registered the service](install-origin.md#registering-the-origin) in the OSG Topology | ||
1. **Shoveler:** Consider the installation of the [Shoveler](https://github.com/opensciencegrid/xrootd-monitoring-shoveler) | ||
|
||
Configuring the Origin | ||
---------------------- | ||
|
@@ -45,10 +46,14 @@ replacing `<YOUR_RESOURCE_NAME>` with the resource name of your origin as | |
and `<FQDN>` with the public DNS name that should be used to contact your origin: | ||
|
||
```file | ||
XC_RESOURCENAME=YOUR_SITE_NAME | ||
XC_RESOURCENAME=<YOUR_SITE_NAME> | ||
ORIGIN_FQDN=<FQDN> | ||
export XC_ORIGINEXPORT=<PATH_TO_STORAGE> | ||
export XC_ROOTDIR=<PATH_TO_BE_ADDED_BY_XROOTD> | ||
``` | ||
|
||
Before starting the service, it could be necessary to run this script before starting the service `/usr/libexec/xcache/authfile-update --origin` (this is on the image crontab) | ||
|
||
Populating Origin Data | ||
---------------------- | ||
|
||
|
@@ -79,6 +84,46 @@ user@host $ docker run --rm --publish 1094:1094 \ | |
Replacing `<HOST PARTITION>` with the host directory containing data that your origin should serve. | ||
See [this section](#populating-origin-data) for details. | ||
|
||
|
||
For Kubernetes (with examples): | ||
|
||
```file | ||
apiVersion: apps/v1 | ||
kind: Deployment | ||
metadata: | ||
name: osdftest | ||
spec: | ||
replicas: 1 | ||
spec: | ||
containers: | ||
image: opensciencegrid/stash-origin:3.6-release | ||
command: [ "sleep" ] | ||
args: [ "infinity" ] | ||
ports: | ||
- containerPort: 1094 | ||
hostPort: 1094 | ||
protocol: TCP | ||
- containerPort: 1095 | ||
hostPort: 1095 | ||
protocol: TCP | ||
resources: | ||
limits: | ||
cpu: "2" | ||
memory: 1Gi | ||
requests: | ||
cpu: "2" | ||
memory: 1Gi | ||
- env: | ||
- name: XC_RESOURCENAME | ||
value: "osdforigintest" | ||
- name: ORIGIN_FQDN | ||
value: "osdftest.t2.ucsd.edu" | ||
- name: XC_ORIGINEXPORT | ||
value: /osdftest/ | ||
- name: XC_ROOTDIR | ||
value: / | ||
``` | ||
|
||
!!!warning | ||
A container deployed this way will serve the entire contents of `<HOST PARTITION>`. | ||
|
||
|
@@ -121,7 +166,9 @@ root@host $ systemctl start docker.stash-origin | |
You must [register](install-origin.md#registering-the-origin) the origin before considering it a | ||
production service. | ||
|
||
### Running on origin kubenetes with systemd | ||
|
||
TODO | ||
Comment on lines
+169
to
+171
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't think we need to give systemd-related Kubernetes instructions – once the deployment is in k8s, users shouldn't have to worry about maintaining a persistent service via systemd! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. On the OSDF caches, we can use supervisorctl restart stash-cache and it is useful during the administration process. We are using the opensciencegrid/stash-cache:3.6-testing-20220414-1721 image There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, that's within the container using supervisord. Systemd would only really be used on bare metal to manage services, which is unnecessary with k8s There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
I don't think this is necessary: the container should start supervisord and thus the relevant services. The operator should ideally not have to interact with supervisord at all! |
||
|
||
Validating Origin | ||
----------------- | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
command
in a deployment refers to theENTRYPOINT
so this will result in a container that starts up and does nothing.