Exam DP203 Course Tybul: Difference between revisions

From MillerSql.com
NeilM (talk | contribs)
NeilM (talk | contribs)
No edit summary
 
(19 intermediate revisions by the same user not shown)
Line 4: Line 4:


== 03 Storage Account Overview ==
== 03 Storage Account Overview ==
https://www.youtube.com/watch?v=ClqZ1j_7NY4
[[Exam_DP203_Course_Tybul_03_Storage_Account_Overview]]


=== Tables ===
== 04 Storage Account Redundancy ==
Configured in portal - storage account.
[[Exam_DP203_Course_Tybul_04_Storage_Account_Redundancy]]


Used by NoSQL. Table is schemaless.
== 05 Storage Account Access Tiers ==
[[Exam_DP203_Course_Tybul_05_Storage_Account_Access_Tiers]]


It is disabled by default, and if you try to create a table, it fails with a permissions error.
== 06 Common File Types csv xml json ==
[[Exam_DP203_Course_Tybul_06_Common_file_types_csv_xml_json]]


https://learn.microsoft.com/en-us/azure/storage/tables/assign-azure-role-data-access?tabs=powershell
== 07 Common File Types parquet ==
[[Exam_DP203_Course_Tybul_07_Common_file_types_parquet]]


says: When you create an Azure Storage account, you are not automatically assigned permissions to access data via Microsoft Entra ID. You must explicitly assign yourself an Azure role for Azure Storage. You can assign it at the level of your subscription, resource group, storage account, or table.
[[Exam_DP203_file_formats_Parquet]]


Storage Table Data Contributor
== 07 Common File Types delta lake ==
[[Exam_DP203_Course_Tybul_07_Common_file_types_delta_lake]]


Edit: I have created a new account (Sandbox).
== 08 Common File Types Data lake structure raw layer ==
[[Exam_DP203_Course_Tybul_08_Data_lake_structure_Raw_layer]]


In a Storage account, go to '''Storage Browser - Tables''' to add a table (you can go to Data Storage - Tables, but there you can't edit the table contents)
== 09 Common File Types Azure Data Factory ==
[[Exam_DP203_Course_Tybul_09_Azure_Data_Factory]]


Click '''Add Entity''' to add a new row to the table. Define columns and datatypes on the fly - it is schemaless so these can be changed, including on the same column in different rows.
Note this also contains videos 10 - 14


=== Queues ===
== 15 Questions about new data source ==
Add a queue. Add a message to the queue. Can set to expire the messages after 7 days. You can "dequeue" the messages.
[[Exam_DP203_Course_Tybul_15_Questions_about_new_data_source]]


You can put anything in the messages, such as JSON code.
== 32 - A closer look at Databricks notebooks ==


=== File shares ===
[[Exam_DP203_Course_Tybul_32_A_closer_look_at_Databricks_notebooks]]
Create file shares. Note this doesn't seem to be allowed in the Sandbox.


Lift and shift from On-Premise file shares, to file shares in the cloud.
== 39 - Azure Synapse Analytics - Spark Pools ==


Me: Not sure whether file shares are available to on-premise or only to other Azure services.
[[Exam_DP203_Course_Tybul_39_Azure_Synapse_Analytics__Spark_Pools]]
 
File sync. A use case of this is where you have two on-premise servers A and B in different countries, where you want to be able to copy a large file from one to the other. This is unreliable over sFTP. So as an alternative, create two Azure file shares in Azure in those locations, and copy the file from on premise to those shares (which is a short distance). Then use the file sync functionality in Azure to copy the files across the longer distance between the two shares.

Latest revision as of 20:37, 31 December 2024

Piotr Tybul YouTube course on DP 203:

https://www.youtube.com/playlist?list=PLuQSde7Xvu7DCRenR1otgxAplTtnzKO9e

03 Storage Account Overview

Exam_DP203_Course_Tybul_03_Storage_Account_Overview

04 Storage Account Redundancy

Exam_DP203_Course_Tybul_04_Storage_Account_Redundancy

05 Storage Account Access Tiers

Exam_DP203_Course_Tybul_05_Storage_Account_Access_Tiers

06 Common File Types csv xml json

Exam_DP203_Course_Tybul_06_Common_file_types_csv_xml_json

07 Common File Types parquet

Exam_DP203_Course_Tybul_07_Common_file_types_parquet

Exam_DP203_file_formats_Parquet

07 Common File Types delta lake

Exam_DP203_Course_Tybul_07_Common_file_types_delta_lake

08 Common File Types Data lake structure raw layer

Exam_DP203_Course_Tybul_08_Data_lake_structure_Raw_layer

09 Common File Types Azure Data Factory

Exam_DP203_Course_Tybul_09_Azure_Data_Factory

Note this also contains videos 10 - 14

15 Questions about new data source

Exam_DP203_Course_Tybul_15_Questions_about_new_data_source

32 - A closer look at Databricks notebooks

Exam_DP203_Course_Tybul_32_A_closer_look_at_Databricks_notebooks

39 - Azure Synapse Analytics - Spark Pools

Exam_DP203_Course_Tybul_39_Azure_Synapse_Analytics__Spark_Pools