Terraform for k8s-s390x-conformance job on ibmcloud

infra/ibmcloud/terraform/k8s-s390x-conformance/README.md

@@ -0,0 +1,59 @@
+# _TF: IBM K8s s390x Conformance_
+These define Terraform resources for setting up infrastructure for the Kubernetes on s390x conformance job.
+
+---
+## Initial Setup
+
+### Supporting infrastructure
+
+#### Deploy k8s-infra-setup resources
+
+- this covers things like Resource Group, s390x Virtual Server Workspace, Virtual Private Cloud, IBM Cloud Secret Manager Secrets, Transit Gateway, etc.
+- Once the deployment successfully completes, the `service_instance_id` and `secrets_manager_id` will be generated and should be used in the subsequent steps.
+
+---
+#### Deploy k8s-s390x-conformance resources
+
+**1. Navigate to the correct directory**
+<br> You need to be in the `k8s-s390x-conformance` directory to run the automation.
+
+**2. Check the `versions.tf` file**
+<br> Set `secret_key` and `access_key` in `versions.tf` to configure the remote S3 backend (IBM Cloud COS).
+
+**3. Initialize Terraform**
+<br> Execute the following command to initialize Terraform in your project directory. This command will download the necessary provider plugins and prepare the working environment.
+```
+terraform init -reconfigure
+```
+
+**4. Check the `variables.tf` file**
+<br> Open the `variables.tf` file to review all the available variables. This file lists all customizable inputs for your Terraform configuration.
+
+`ibmcloud_api_key`, `service_instance_id`, `secrets_manager_id` are the only required variables that you must set in order to proceed. You can set this key either by adding it to your `var.tfvars` file or by exporting it as an environment variable.
+
+**Option 1:** Set in `var.tfvars` file
+Create `var.tfvars` file and set the following variables in `var.tfvars` file:
+```
+ibmcloud_api_key    = "<YOUR_API_KEY>"
+secrets_manager_id  = "<SECRETS_MANAGER_ID>"
+```
+
+**Option 2:** Export as an environment variable
+Alternatively, you can export above as an environment variable before running Terraform:
+```
+export TF_VAR_ibmcloud_api_key="<YOUR_API_KEY>"
+export TF_VAR_secrets_manager_id="<SECRETS_MANAGER_ID>"
+```
+
+**5. Run Terraform Apply**
+<br> After setting the necessary variables (particularly the API_KEY), execute the following command to apply the Terraform configuration and provision the infrastructure:
+```
+terraform apply -var-file var.tfvars
+```
+Terraform will display a plan of the actions it will take, and you'll be prompted to confirm the execution. Type `yes` to proceed.
+
+**6. Get Output Information**
+<br> Once the infrastructure has been provisioned, use the terraform output command to list details about the provisioned resources.
+```
+terraform output
+```

infra/ibmcloud/terraform/k8s-s390x-conformance/main.tf

@@ -0,0 +1,52 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+module "resource_group" {
+  source = "./modules/resource_group"
+}
+module "iam_custom_role" {
+  source = "./modules/iam/custom_role"
+}
+
+module "service_ids" {
+  depends_on        = [module.iam_custom_role]
+  source            = "./modules/iam/service_ids"
+  resource_group_id = module.resource_group.conformance_resource_group_id
+}
+
+module "iam_access_groups" {
+  depends_on        = [module.iam_custom_role]
+  source            = "./modules/iam/access_groups"
+  resource_group_id = module.resource_group.conformance_resource_group_id
+}
+
+module "secrets_manager" {
+  source                            = "./modules/secrets_manager"
+  janitor_access_group_id           = module.iam_access_groups.janitor_access_group_id
+  vpc_build_cluster_access_group_id = module.iam_access_groups.vpc_build_cluster_access_group_id
+  secret_rotator_access_group_id    = module.iam_access_groups.secret_rotator_access_group_id
+  apikey                            = module.service_ids.sm_read_apikey
+  resource_group_id                 = module.resource_group.conformance_resource_group_id
+  secrets_manager_id                = var.secrets_manager_id
+}
+module "vpc" {
+  providers = {
+    ibm = ibm.vpc
+  }
+  source            = "./modules/vpc"
+  zone              = var.zone
+  resource_group_id = module.resource_group.conformance_resource_group_id
+}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/access_groups/access_groups.tf

@@ -0,0 +1,60 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+resource "ibm_iam_access_group" "vpc_build_cluster" {
+  name        = "vpc-build-cluster-access"
+  description = "Access group with the necessary permissions for provisioning VPC-based build clusters."
+}
+
+resource "ibm_iam_access_group_policy" "vpc_build_cluster" {
+  access_group_id = ibm_iam_access_group.vpc_build_cluster.id
+  roles           = ["VPCBuildClusterRole"]
+
+  resources {
+    service           = "is"
+    resource_group_id = var.resource_group_id
+  }
+}
+
+resource "ibm_iam_access_group" "janitor" {
+  name        = "janitor-access"
+  description = "Access group with the necessary permissions for the Boskos Janitor."
+}
+
+resource "ibm_iam_access_group_policy" "janitor_vpc" {
+  access_group_id = ibm_iam_access_group.janitor.id
+  roles           = ["JanitorVPCRole"]
+
+  resources {
+    service           = "is"
+    resource_group_id = var.resource_group_id
+  }
+}
+
+resource "ibm_iam_access_group" "secret_rotator" {
+  name        = "secret-rotator"
+  description = "Access group with the necessary permissions for secret-manager(rotator)."
+}
+
+resource "ibm_iam_access_group_policy" "secret_rotator" {
+  access_group_id = ibm_iam_access_group.secret_rotator.id
+  roles           = ["SecretRotator"]
+
+  resources {
+    service           = "secrets-manager"
+    resource_group_id = var.resource_group_id
+  }
+}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/access_groups/outputs.tf

@@ -0,0 +1,28 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+output "vpc_build_cluster_access_group_id" {
+  value = ibm_iam_access_group.vpc_build_cluster.id
+}
+
+output "janitor_access_group_id" {
+  value = ibm_iam_access_group.janitor.id
+}
+
+output "secret_rotator_access_group_id" {
+  value = ibm_iam_access_group.secret_rotator.id
+}
+

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/access_groups/variables.tf

@@ -0,0 +1,17 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+variable "resource_group_id" {}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/access_groups/versions.tf

@@ -0,0 +1,23 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+terraform {
+  required_providers {
+    ibm = {
+      source = "IBM-Cloud/ibm"
+    }
+  }
+}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/custom_role/iam_custom_role.tf

@@ -0,0 +1,68 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+resource "ibm_iam_custom_role" "vpc_build_cluster" {
+  name         = "VPCBuildClusterRole"
+  display_name = "VPCBuildClusterRole"
+  service      = "is"
+  actions = [
+    "is.vpc.vpc.read",
+    "is.vpc.vpc.create",
+    "is.vpc.vpc.update",
+    "is.vpc.vpc.list",
+    "is.vpc.vpc.delete",
+  ]
+}
+
+resource "ibm_iam_custom_role" "sm" {
+  name         = "SMRead"
+  display_name = "SMRead"
+  service      = "secrets-manager"
+  actions = [
+    "secrets-manager.secrets.list",
+    "secrets-manager.secret.read",
+  ]
+}
+
+resource "ibm_iam_custom_role" "janitor_vpc" {
+  name         = "JanitorVPCRole"
+  display_name = "JanitorVPCRole"
+  service      = "is"
+  actions = [
+    "is.instance.instance.delete",
+    "is.subnet.subnet.delete",
+    "is.security-group.security-group.delete",
+    "is.floating-ip.floating-ip.delete",
+    "is.vpc.vpc.read",
+    "is.subnet.subnet.read",
+    "is.security-group.security-group.read",
+    "is.instance.instance.read",
+    "resource-controller.instance.retrieve",
+    "resource-controller.group.retrieve"
+  ]
+}
+
+resource "ibm_iam_custom_role" "secret_rotator" {
+  name         = "SecretRotator"
+  display_name = "SecretRotator"
+  service      = "secrets-manager"
+  actions = [
+    "secrets-manager.secret-version.read",
+    "secrets-manager.secret-version.create",
+    "secrets-manager.secret.read",
+    "secrets-manager.secret.rotate",
+  ]
+}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/custom_role/versions.tf

@@ -0,0 +1,23 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+terraform {
+  required_providers {
+    ibm = {
+      source = "IBM-Cloud/ibm"
+    }
+  }
+}

infra/ibmcloud/terraform/k8s-s390x-conformance/modules/iam/service_ids/outputs.tf

@@ -0,0 +1,20 @@
+/*
+Copyright 2025 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+output "sm_read_apikey" {
+  value     = ibm_iam_service_api_key.service_id_apikey.apikey
+  sensitive = true
+}