Skills Spark SQL: Difference between revisions
Created page with "Databricks data migration Python" |
|||
| (25 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
== Spark SQL == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
Spark SQL | |||
input_file_name() | |||
Refresh Table | |||
current_timestamp() | |||
ifnull(a, b) | |||
reverse(a) | |||
Datatypes: timestamp, string, int, bigint, date | |||
== Databricks == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
data migration | data migration | ||
Python | Databricks Compute (clusters) | ||
Widgets (variables) | |||
Backup tables (job) | |||
Databricks notebooks | |||
SQL MERGE | |||
== Python == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Streamsets == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
== Azure == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Azure Blob Storage == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Azure Delta Lake == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
Latest revision as of 18:46, 2 October 2024
Spark SQL
Used at:...
Used at: Ovo Energy SSE Energy smart meter
Spark SQL
input_file_name()
Refresh Table
current_timestamp()
ifnull(a, b)
reverse(a)
Datatypes: timestamp, string, int, bigint, date
Databricks
Used at:...
Used at: Ovo Energy SSE Energy smart meter
data migration
Databricks Compute (clusters)
Widgets (variables)
Backup tables (job)
Databricks notebooks
SQL MERGE
Python
Used at:...
Used at: Ovo Energy
Streamsets
Used at:...
Used at: Ovo Energy SSE Energy smart meter
Azure
Used at:...
Used at: Ovo Energy
Azure Blob Storage
Used at:...
Used at: Ovo Energy
Azure Delta Lake
Used at:...
Used at: Ovo Energy SSE Energy smart meter