Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Users/curiekim/add default storage firewall templates #13901

Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
---
description: This template allows you to create an Default Storage Firewall enabled Azure Databricks workspace with Privateendpoint, all three forms of CMK, and User-Assigned Access Connector.
page_type: sample
products:
- azure
- azure-resource-manager
urlFragment: databricks-all-in-one-template-for-privateendpoint-cmk-all-forms-default-storage-firewall-default-storage-firewall
languages:
- bicep
- json
---
# Deploy an Azure Databricks Workspace with user assigned identity, acess connector, private endpoint, managed svc & CMK & DBFS encryption

[![Deploy To Azure](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/deploytoazure.svg?sanitize=true)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.databricks%2Fdatabricks-all-in-one-template-for-privateendpoint-cmk-all-forms-default-storage-firewall%2Fazuredeploy.json)

[![Visualize](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/visualizebutton.svg?sanitize=true)](http://armviz.io/#/?load=https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.databricks%2Fdatabricks-all-in-one-template-for-privateendpoint-cmk-all-forms-default-storage-firewall%2Fazuredeploy.json)


This template allows you to create a Azure Databricks workspace with privateendpoint, all three forms of customer managed keys (CMK), and user-assigned accessconnector. For more information, see the [Azure Databricks Documentation](https://docs.microsoft.com/azure/azure-databricks/).

`Tags: Microsoft.Databricks/workspaces, Microsoft.Resources/deployments, Microsoft.KeyVault/vaults/accessPolicies`

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"location": {
"value": "GET-PREREQ-outLocation"
},
"workspaceName": {
"value": "default"
},
"disablePublicIp": {
"value": true
},
"managedIdentityName": {
"value": "default"
},
"accessConnectorName": {
"value": "default"
},
"PrivateEndpointSubnetName": {
"value": "default"
},
"msCmkKeyVaultResourceGroup": {
"value": "GET-PREREQ-outKeyVaultResourceGroupName"
},
"msCmkKeyVaultName": {
"value": "GET-PREREQ-outKeyVaultName"
},
"msCmkkeyName": {
"value": "GET-PREREQ-outKeyName"
},
"msCmkkeyVersion": {
"value": "GET-PREREQ-outKeyVersion"
},
"azureDatabricksAppObjectId": {
"value": "GET-PREREQ-outObjectID"
},
"dbfsCmkResourceGroup": {
"value": "GET-PREREQ-outKeyVaultResourceGroupName"
},
"dbfsCmkKeyVaultName": {
"value": "GET-PREREQ-outKeyVaultName"
},
"dbfsCmkKeyName": {
"value": "GET-PREREQ-outKeyName"
},
"dbfsCmkKeyVersion": {
"value": "GET-PREREQ-outKeyVersion"
},
"diskCmkKeyVaultUrl": {
"value": "GET-PREREQ-outKeyVaultUri"
},
"diskCmkKeyVaultResourceGroup": {
"value": "GET-PREREQ-outKeyVaultResourceGroupName"
},
"diskCmkKeyVaultName": {
"value": "GET-PREREQ-outKeyVaultName"
},
"diskCmkKeyName": {
"value": "GET-PREREQ-outKeyName"
},
"diskCmkKeyVersion": {
"value": "GET-PREREQ-outKeyVersion"
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
{
"$schema": "https://aka.ms/azure-quickstart-templates-metadata-schema#",
"type": "QuickStart",
"itemDisplayName": "Deploy a default storage firewall enabled Azure Databricks Workspace with user-assigned access connector",
"description": "This template allows you to create an Default Storage Firewall enabled Azure Databricks workspace with Privateendpoint, all three forms of CMK, and User-Assigned Access Connector.",
"summary": "Deploy an Azure Databricks Workspace with UserAssignedIdentity, AccessConnector, PrivateEndpoint, managed services, CMK and DBFS.",
"tags": {
"resource": "Azure Databricks",
"level": "Intermediate"
},
"environments": [
"AzureCloud"
]
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
"contentVersion": "1.0.0.0",
"parameters": {
"vaultName": {
"type": "string",
"defaultValue": "[concat('keyvault-', uniqueString(resourceGroup().id))]",
"metadata": {
"description": "Specifies the name of the KeyVault, this value must be globally unique."
}
},
"location": {
"type": "string",
"defaultValue": "[resourceGroup().location]",
"metadata": {
"description": "Specifies the Azure location where the key vault should be created."
}
},
"keyName": {
"type": "string",
"defaultValue": "[concat('keyvault8-', uniqueString(resourceGroup().id))]",
"metadata": {
"description": "Name of the encryption key."
}
},
"keyVaultResourceGroupName": {
"type": "string",
"defaultValue": "[resourceGroup().name]",
"metadata": {
"description": "The resource group name of the key vault used for customer-managed key for managed services"
}
},
"ObjectID": {
"type": "string",
"defaultValue": "fd477f4b-567b-4181-9978-7f37f838c1a6",
"metadata": {
"description": "The object ID of the AzureDatabricks enterprise application."
}
}
},

"resources": [
{
"name": "vault-prereq",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/modules/Microsoft.KeyVault/vaults/1.0/azuredeploy.json",
"contentVersion": "1.0.0.0"
},
"parameters": {
"vaultName": {
"value": "[parameters('vaultName')]"
},
"location": {
"value": "[parameters('location')]"
},
"enablePurgeProtection": {
"value": true
}
}
}
},
{
"name": "key-prereq",
"type": "Microsoft.Resources/deployments",
"apiVersion": "2021-04-01",
"dependsOn": [
"vault-prereq"
],
"properties": {
"mode": "Incremental",
"templateLink": {
"uri": "https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/modules/Microsoft.KeyVault/vaults/keys/0.9/azuredeploy.json",
"contentVersion": "1.0.0.0"
},
"parameters": {
"vaultName": {
"value": "[parameters('vaultName')]"
},
"keyName": {
"value": "[parameters('keyName')]"
},

"kty": {
"value": "RSA"
},
"key_ops": {
"value": [
"encrypt",
"decrypt",
"sign",
"verify",
"wrapKey",
"unwrapKey"
]
}
}
}
}
],
"outputs": {
"outKeyVaultName": {
"type": "string",
"value": "[parameters('vaultName')]"
},
"outKeyVaultUri": {
"type": "string",
"value": "[split(reference('key-prereq').outputs.key.value.properties.keyUri, '/keys/')[0]]"
},
"outKeyName": {
"type": "string",
"value": "[parameters('keyName')]"
},
"outKeyVersion": {
"type": "string",
"value": "[split(reference('key-prereq').outputs.key.value.properties.keyUriWithVersion, '/')[5]]"
},
"outLocation": {
"type": "string",
"value": "[parameters('location')]"
},
"outKeyVaultResourceGroupName": {
"type": "string",
"value": "[parameters('keyVaultResourceGroupName')]"
},
"outObjectID": {
"type": "string",
"value": "[parameters('ObjectID')]"
}
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
{
"$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentParameters.json#",
"contentVersion": "1.0.0.0",
"parameters": { }
}

Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
---
description: This template allows you to create a network security group, a virtual network, private endpoint, and a default storage firewall enabled Azure Databricks workspace with the virtual network and the system-assigned access connector.
page_type: sample
products:
- azure
- azure-resource-manager
urlFragment: databricks-all-in-one-template-for-vnet-injection-privateendpoint-default-storage-firewall
languages:
- bicep
- json
---
# Azure Databricks All-in-one Templat VNetInjection-Pvtendpt-Default-Storage-Firewall

[![Deploy To Azure](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/deploytoazure.svg?sanitize=true)](https://portal.azure.com/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.databricks%2Fdatabricks-all-in-one-template-for-vnet-injection-privateendpoint-default-storage-firewall%2Fazuredeploy.json)
[![Deploy To Azure US Gov](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/deploytoazuregov.svg?sanitize=true)](https://portal.azure.us/#create/Microsoft.Template/uri/https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.databricks%2Fdatabricks-all-in-one-template-for-vnet-injection-privateendpoint-default-storage-firewall%2Fazuredeploy.json)
[![Visualize](https://raw.githubusercontent.com/Azure/azure-quickstart-templates/master/1-CONTRIBUTION-GUIDE/images/visualizebutton.svg?sanitize=true)](http://armviz.io/#/?load=https%3A%2F%2Fraw.githubusercontent.com%2FAzure%2Fazure-quickstart-templates%2Fmaster%2Fquickstarts%2Fmicrosoft.databricks%2Fdatabricks-all-in-one-template-for-vnet-injection-privateendpoint-default-storage-firewall%2Fazuredeploy.json)

This template allows you to create a network security group, a virtual network, private endpoint, and a default storage firewall enabled Azure Databricks workspace with the virtual network and the system-assigned access connector.
For more information, see the [Azure Databricks Documentation](https://docs.microsoft.com/azure/azure-databricks/).

### What is Azure Databricks?

Azure Databricks is an Apache Spark-based analytics platform optimized for the Microsoft Azure cloud services platform. Designed with the founders of Apache Spark, Databricks is integrated with Azure to provide one-click setup, streamlined workflows, and an interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

Azure Databricks is a fast, easy, and collaborative Apache Spark-based analytics service. For a big data pipeline, the data (raw or structured) is ingested into Azure through Azure Data Factory in batches, or streamed near real-time using Kafka, Event Hub, or IoT Hub. This data lands in a data lake for long term persisted storage, in Azure Blob Storage or Azure Data Lake Storage. As part of your analytics workflow, use Azure Databricks to read data from multiple data sources such as Azure Blob Storage, Azure Data Lake Storage, Azure Cosmos DB, or Azure SQL Data Warehouse and turn it into breakthrough insights using Spark.

This template allows you to install the following options

+ Databricks 14 day trial
+ Databricks Standard
+ Databricks Premium

### Databricks Resources

[Getting Started with Databricks](https://docs.microsoft.com/azure/databricks/getting-started/index)
[Databricks Admin Guide](https://docs.azuredatabricks.net/administration-guide/index.html)

### Microsoft Learn Modules

[Databricks Microsoft Learn Modules](https://docs.microsoft.com/learn/browse/?term=Databricks)

`Tags: Microsoft.Network/networkSecurityGroups, Microsoft.Network/virtualNetworks, Microsoft.Databricks/workspaces, Microsoft.Network/privateEndpoints, Microsoft.Network/privateDnsZones, Microsoft.Network/privateDnsZones/virtualNetworkLinks, Microsoft.Network/privateEndpoints/privateDnsZoneGroups`