Adding workspace infra state migration to Terraform templates for AWS, GCP, and Azure for the blogathon

2025-04-managing-databricks-accounts-and-workspaces-with-code/LICENSE.txt

@@ -0,0 +1,22 @@
+DB license
+Definitions. 
+
+Agreement: The agreement between Databricks, Inc., and you governing the use of the Databricks Services, as that term is defined in the Master Cloud Services Agreement (MCSA) located at www.databricks.com/legal/mcsa.
+
+Licensed Materials: The source code, object code, data, and/or other works to which this license applies.
+
+Scope of Use. You may not use the Licensed Materials except in connection with your use of the Databricks Services pursuant to the Agreement. Your use of the Licensed Materials must comply at all times with any restrictions applicable to the Databricks Services, generally, and must be used in accordance with any applicable documentation. You may view, use, copy, modify, publish, and/or distribute the Licensed Materials solely for the purposes of using the Licensed Materials within or connecting to the Databricks Services. If you do not agree to these terms, you may not view, use, copy, modify, publish, and/or distribute the Licensed Materials.
+
+Redistribution. You may redistribute and sublicense the Licensed Materials so long as all use is in compliance with these terms. In addition:
+
+You must give any other recipients a copy of this License;
+You must cause any modified files to carry prominent notices stating that you changed the files;
+You must retain, in any derivative works that you distribute, all copyright, patent, trademark, and attribution notices, excluding those notices that do not pertain to any part of the derivative works; and
+If a "NOTICE" text file is provided as part of its distribution, then any derivative works that you distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the derivative works.
+You may add your own copyright statement to your modifications and may provide additional license terms and conditions for use, reproduction, or distribution of your modifications, or for any such derivative works as a whole, provided your use, reproduction, and distribution of the Licensed Materials otherwise complies with the conditions stated in this License.
+
+Termination. This license terminates automatically upon your breach of these terms or upon the termination of your Agreement. Additionally, Databricks may terminate this license at any time on notice. Upon termination, you must permanently delete the Licensed Materials and all copies thereof.
+
+DISCLAIMER; LIMITATION OF LIABILITY.
+
+THE LICENSED MATERIALS ARE PROVIDED “AS-IS” AND WITH ALL FAULTS. DATABRICKS, ON BEHALF OF ITSELF AND ITS LICENSORS, SPECIFICALLY DISCLAIMS ALL WARRANTIES RELATING TO THE LICENSED MATERIALS, EXPRESS AND IMPLIED, INCLUDING, WITHOUT LIMITATION, IMPLIED WARRANTIES, CONDITIONS AND OTHER TERMS OF MERCHANTABILITY, SATISFACTORY QUALITY OR FITNESS FOR A PARTICULAR PURPOSE, AND NON-INFRINGEMENT. DATABRICKS AND ITS LICENSORS TOTAL AGGREGATE LIABILITY RELATING TO OR ARISING OUT OF YOUR USE OF OR DATABRICKS’ PROVISIONING OF THE LICENSED MATERIALS SHALL BE LIMITED TO ONE THOUSAND ($1,000) DOLLARS.  IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE LICENSED MATERIALS OR THE USE OR OTHER DEALINGS IN THE LICENSED MATERIALS.

2025-04-managing-databricks-accounts-and-workspaces-with-code/NOTICE.txt

@@ -0,0 +1,17 @@
+Copyright (2025) Databricks, Inc.
+
+This Software includes software developed at Databricks (https://www.databricks.com/) and its use is subject to the included LICENSE file.
+
+________________
+This Software contains code from the following open source projects, licensed under the Apache 2.0 license (https://www.apache.org/licenses/LICENSE-2.0):
+
+databricks/terraform-provider-databricks - https://github.com/databricks/terraform-provider-databricks
+Copyright 2022 Databricks, Inc.  All rights reserved.
+
+_____________
+This Software contains code from the following open source projects, licensed under the MPL License Agreement:
+
+hashicorp/terraform-provider-azurerm - https://github.com/hashicorp/terraform-provider-azurerm
+Copyright © 2017- Hashicorp, Inc; All Rights Reserved
+License - https://github.com/hashicorp/terraform-provider-azurerm/blob/main/LICENSE
+

2025-04-managing-databricks-accounts-and-workspaces-with-code/README.md

@@ -0,0 +1,15 @@
+# Terraform Workspace Infrastucture Migration Guide
+
+## Overview
+
+This project contain templates can be used to migrate the current infrastructure details of a Databricks workspace into a Terraform project.
+
+- [AWS](aws/README.md)
+- [Azure](azure/README.md)
+- [GCP](gcp/README.md)
+
+
+
+
+
+

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/README.md

@@ -0,0 +1,40 @@
+# Overview
+This code example is based on this [Databricks Terraform workspace module](https://github.com/databricks/terraform-databricks-examples/blob/main/modules/aws-databricks-workspace/main.tf) and showcases the first step to migrating an existing workspace's infrastructure to Terraform.
+
+
+If your workspace has features not included in this template, such as Private Link or Network Connnectivity Configurations, use the [Deployment section in the Databricks Terraform Provider's documentation](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/mws_workspaces#argument-reference)for the resource syntax to include.
+
+# Prerequistes
+* Databricks Account Admin role and access to the account console.
+* (Optional but recommended) A Databricks service principal with account admin role for authentication 
+* [Terraform CLI installed on your machine](https://developer.hashicorp.com/terraform/install)
+
+
+# Instructions
+* Configure Databricks Authentication. If using a service principal, generate an OAuth Token and add to your local .databrickcfg file. See the [documentation for details and alternative authenication methods](https://docs.databricks.com/aws/en/dev-tools/auth/oauth-m2m?language=Terraform). 
+* Fill out the variables in the terraform.tfvars file with values as specified below.
+* Run `terraform init` in your terminal.
+* Run `terraform plan` in your terminal. You should receive a "Plan: (# of resources added to main.tf) to import, 0 to add, 0 to change, 0 to destroy." message.
+
+* Run `terraform apply` in your terminal to import the current state of your workspace configuration. 
+
+# Variables to inlcude in .tfvars file
+
+### From the Databricks account console
+* databricks_account_id (string) - see upper right corner and click on the circular avatar.
+* workspace_name (string) - from workspace details page.
+* region (string) - from the region section in the workspace details page.
+* network_name (string) - from network configuration page. Click on the link in the network section of the workspace details page.
+* network_id (string) - from network configuration page URL. Click on the link in the network section of the workspace details page. This is the uuid after network-configurations/
+* vpc_id (string) - from network configuration page. Click on the link in the network section of the workspace details page.
+* vpc_private_subnets list(string) - from network configuration page. Click on the link in the network section of the workspace details page.
+* security_group_ids list(string) - from network configuration page. Click on the link in the network section of the workspace details page.
+* cross_account_role_arn (string) - from credential configuration page. Click on the link in the credentials section of the workspace details page.
+* credentials_name (string) - from credential configuration page. Click on the link in the credentials section of the workspace details page.
+* credentials_id (string) - from credential configuration page URL. Click on the link in the credentials section of the workspace details page. This is the uuid after credential-configurations/
+* root_storage_bucket (string) - bucket name from storage configuration page. Click on the link in the storage section of the workspace details page. 
+* storage_configuration_name (string) - name from storage configuration page. Click on the link in the storage section of the workspace details page. 
+* storage_configuration_id (string) - from storage configuration page URL. Click on the link in the storage section of the workspace details page. This is the uuid after storage-configurations/
+
+### From the workspace
+* workspace_id (string) - 16 digit number from workspace URL after o=

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/import.tf

@@ -0,0 +1,19 @@
+import {
+  to = databricks_mws_credentials.this
+  id = "${var.databricks_account_id}/${var.credentials_id}"
+}
+
+import {
+  to = databricks_mws_networks.this
+  id = "${var.databricks_account_id}/${var.network_id}"
+}
+
+import {
+  to = databricks_mws_storage_configurations.this
+  id = "${var.databricks_account_id}/${var.storage_configuration_id}"
+}
+
+import {
+  to = databricks_mws_workspaces.this
+  id = "${var.databricks_account_id}/${var.workspace_id}"
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/main.tf

@@ -0,0 +1,29 @@
+resource "databricks_mws_credentials" "this" {
+  role_arn         = var.cross_account_role_arn
+  credentials_name = var.credentials_name
+}
+
+resource "databricks_mws_networks" "this" {
+  account_id         = var.databricks_account_id
+  network_name       = var.network_name
+  security_group_ids = var.security_group_ids
+  subnet_ids         = var.vpc_private_subnets
+  vpc_id             = var.vpc_id
+}
+
+resource "databricks_mws_storage_configurations" "this" {
+  account_id                 = var.databricks_account_id
+  bucket_name                = var.root_storage_bucket
+  storage_configuration_name = var.storage_configuration_name
+}
+
+resource "databricks_mws_workspaces" "this" {
+  account_id     = var.databricks_account_id
+  aws_region     = var.region
+  workspace_name = var.workspace_name
+
+  credentials_id           = databricks_mws_credentials.this.credentials_id
+  storage_configuration_id = databricks_mws_storage_configurations.this.storage_configuration_id
+  network_id               = databricks_mws_networks.this.network_id
+
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/terraform.tfvars

@@ -0,0 +1,17 @@
+databricks_account_id = ""
+workspace_name = ""
+workspace_id = ""
+region = ""
+network_name = ""
+network_id = ""
+vpc_id = ""
+vpc_private_subnets = [ "", "" ]
+security_group_ids = [ "" ]
+cross_account_role_arn = ""
+credentials_name = ""
+credentials_id = ""
+
+
+root_storage_bucket = ""
+storage_configuration_name = ""
+storage_configuration_id = ""

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/variables.tf

@@ -0,0 +1,74 @@
+variable "workspace_name" {
+  type        = string
+  description = "(Required) Workspace name"
+}
+
+variable "workspace_id" {
+  type        = string
+  description = "(Required) Workspace ID"
+}
+
+variable "region" {
+  type        = string
+  description = "(Required) AWS region where the assets will be deployed"
+}
+
+variable "vpc_id" {
+  type        = string
+  description = "(Required) AWS VPC ID"
+}
+
+variable "security_group_ids" {
+  type        = list(string)
+  description = "(Required) List of VPC network security group IDs from Databricks Account Console"
+}
+
+variable "vpc_private_subnets" {
+  type        = list(string)
+  description = "(Required) AWS VPC Subnet IDs from Databricks account console"
+}
+
+variable "databricks_account_id" {
+  type        = string
+  description = "(Required) Databricks Account ID"
+}
+
+variable "cross_account_role_arn" {
+  type        = string
+  description = "(Required) AWS cross account role ARN from Databricks account console"
+}
+
+variable "root_storage_bucket" {
+  type        = string
+  description = "(Required) AWS root storage bucket"
+}
+
+variable "credentials_name" {
+  type        = string
+  description = "(Required) Credentials name from Databricks account console"
+}
+
+variable "credentials_id" {
+  type        = string
+  description = "(Required) Credentials ID from Databricks account console"
+}
+
+variable "network_name" {
+  type        = string
+  description = "(Required) Network name from Databricks account console"
+}
+
+variable "network_id" {
+  type        = string
+  description = "(Required) Network configuration ID from Databricks account console"
+}
+
+variable "storage_configuration_name" {
+  type        = string
+  description = "(Required) Storage name from Databricks account console"
+}
+
+variable "storage_configuration_id" {
+  type        = string
+  description = "(Required) Storage configuration name from Databricks account console"
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/aws/versions.tf

@@ -0,0 +1,14 @@
+terraform {
+  required_providers {
+    databricks = {
+      source  = "databricks/databricks"
+      version = ">=1.13.0"
+    }
+
+  }
+}
+
+provider "databricks" {
+  host     = "https://accounts.cloud.databricks.com"
+  account_id = var.databricks_account_id
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/azure/README.md

@@ -0,0 +1,39 @@
+# Overview
+This code example is based on this [Databricks Terraform workspace module](https://github.com/databricks/terraform-databricks-examples/tree/main/modules/adb-lakehouse) and showcases the first step to migrating an existing workspace's infrastructure to Terraform.
+
+
+If your workspace has features not included in this template, such as Private Link or Customer-Managed Keys, use the [Deployment section in the Databricks Terraform Provider's documentation](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs/resources/databricks_workspace)for the resource syntax to include.
+
+# Prerequistes
+* Permissions to view the details of the Databricks workspace in the Azure Portal.
+* Workspace admin role.
+* (Optional but recommended) A Databricks service principal with workspace admin role for authentication.
+* [Terraform CLI installed on your machine](https://developer.hashicorp.com/terraform/install).
+
+
+# Instructions
+* Configure [Azure authentication](https://registry.terraform.io/providers/hashicorp/azurerm/latest/docs#authenticating-to-azure).
+* Review the main.tf file and include any additional configuration.
+* Fill out the variables in the terraform.tfvars file with values as specified below. Include new values and variables if added in the previous step.
+* Run `terraform init` in your terminal.
+* Run `terraform plan` in your terminal. You should receive a "Plan: (# of resources added to main.tf) to import, 0 to add, 0 to change, 0 to destroy." message.
+* Run `terraform apply` in your terminal to import the current state of your workspace configuration. 
+
+
+
+# Variables to inlcude in .tfvars file
+
+### From the Azure portal
+* workspace_name (string) - from workspace overview
+* workspace_id (string) - resource ID from workspace overview JSON view in the upper right corner. 
+* resource_group_name (string) -  from workspace overview
+* location (string) -  from workspace overview
+* managed_resource_group_name (string) -  from workspace overview
+* sku_type (string) - pricing tier from workspace overview (premium or standard)
+* virtual_network_id (string) - resource ID from virtual network JSON view in the upper right corner. Click on the virtual network link in workspace overview page to find the virtual network attached to the workspace.
+* private_subnet_name (string) - from workspace overview page. Click on see more if not visible
+* public_subnet_name (string) - from workspace overview page. Click on see more if not visible
+* public_subnet_id (string) - subnet ID from virtual network subnet setting details page. Click on the subnet link in the workspace overview page and then click on the subnet name to access the subnet setting details page.
+* private_subnet_id (string) - subnet ID from virtual network subnet setting details page. Click on the subnet link in the workspace overview page and then click on the subnet name to access the subnet setting details page.
+* nat_gateway_id (string) - resource ID from NAT gateway JSON view in the upper right corner. Find the NAT gateway on the subnet settings page in the security section.
+

2025-04-managing-databricks-accounts-and-workspaces-with-code/azure/import.tf

@@ -0,0 +1,14 @@
+import {
+  to = azurerm_databricks_workspace.this
+  id = var.workspace_id
+}
+
+import {
+  to = azurerm_subnet_nat_gateway_association.private
+  id = var.private_subnet_id
+}
+
+import {
+  to = azurerm_subnet_nat_gateway_association.public
+  id = var.public_subnet_id
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/azure/main.tf

@@ -0,0 +1,25 @@
+resource "azurerm_databricks_workspace" "this" {
+  name                        = var.workspace_name
+  resource_group_name         = var.resource_group_name
+  managed_resource_group_name = var.managed_resource_group_name
+  location                    = var.location
+  sku                         = var.sku_type
+
+  custom_parameters {
+
+    no_public_ip                                         = true
+    virtual_network_id                                   = var.virtual_network_id
+    private_subnet_name                                  = var.private_subnet_name
+    public_subnet_name                                   = var.public_subnet_name
+  }
+}
+
+resource "azurerm_subnet_nat_gateway_association" "private" {
+  subnet_id      = var.private_subnet_id
+  nat_gateway_id = var.nat_gateway_id
+}
+
+resource "azurerm_subnet_nat_gateway_association" "public" {
+  subnet_id      = var.public_subnet_id
+  nat_gateway_id = var.nat_gateway_id
+}

2025-04-managing-databricks-accounts-and-workspaces-with-code/azure/terraform.tfvars

@@ -0,0 +1,13 @@
+workspace_name = ""
+workspace_id = ""
+resource_group_name = ""
+managed_resource_group_name = ""
+location = ""                 
+sku_type = ""   
+virtual_network_id = ""
+private_subnet_name = ""
+private_subnet_id = ""
+public_subnet_name  = ""    
+public_subnet_id = ""
+nat_gateway_id  = "" 
+subscription_id = ""