Exam DP203 Course Tybul 09 Azure Data Factory

https://www.youtube.com/watch?v=nnAgdqrZZGM

Note that Azure SQL Database is not in the exam.

"SQL Databases" service in Azure is the implementation of a SQL Server in the cloud. In a server instance you can create databases. An example of PAAS - Platform as a service.

"Azure SQL Database" is the implementation of a SQL server single database in the cloud. You don't get the rest of the SQL Server there. An example of SAAS - Software as a service.

In the "Compute and storage" section when you are creating the SQL databases server, click to change the setting to "Basic" to generate a lower spec machine that is cheaper.

When you have created the SQL Server resource, you need to go into Security - Networking, and change "Public Network Access" from "Disabled" to "Selected Networks". Then further down click "Allow Azure Services and Resources" access to it.

In Data Factory Studio, which can be opened from within the Azure data factory resource in the portal, under the Manage blade, you can create a linked server to connect to a data source. Here you choose between data store and compute.

11 - Dynamic Azure Data Factory

Parameters in dataset names

12 - Integration Runtimes part 1

You have a set of virtual machines within Azure that are used to run Azure Data Factory jobs for all Azure tenants.

In order to prevent these machines from being able to access your own Azure resources, you need to create an Azure Managed VNET Integration Runtime. This I think defines a set of these machines for your own use. These machines are then bound to a new virtual network, and you give only this virtual network access to your Azure resources.

You create the integration runtime. And then you create a linked service that uses it to connect to the source database. As part of this you must also create a managed endpoint.

Then you go into the Azure SQL source resource, and in Networking there is tab "Private Access" (as opposed to "public access") and the new endpoint above can be seen. If you want this endpoint to be able to connect to the Azure SQL server, click "Approve".

Azure Managed VNET integration runtimes cost $1/hour, as opposed to the built-in integration runtimes that cost $0.005/hour.

Note that Azure VNET integration runtimes are a little slower than the built-in one.

13 - Integration Runtimes part 2

Azure self-hosted integration runtime. This is an integration runtime that is run in Azure, but within a virtual machine. So you have to create a virtual machine resource within the portal, and run it. This is a windows machine, and costs around $1/hour to run.

You RDP onto the virtual machine, and download the installer for the above runtime, and install it.

When it is installed, run it.

It will ask for the connection string to connect to the Azure SQL Database.