Skills Spark SQL: Difference between revisions
No edit summary |
|||
| (22 intermediate revisions by the same user not shown) | |||
| Line 5: | Line 5: | ||
''Used at: '' | ''Used at: '' | ||
''[[Role_Ovo_Energy|Ovo Energy]]'' | ''[[Role_Ovo_Energy|Ovo Energy]]'' | ||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | </div> | ||
</div> | </div> | ||
Spark SQL | Spark SQL | ||
input_file_name() | |||
Refresh Table | |||
current_timestamp() | |||
ifnull(a, b) | |||
reverse(a) | |||
Datatypes: timestamp, string, int, bigint, date | |||
== Databricks == | == Databricks == | ||
| Line 15: | Line 28: | ||
''Used at: '' | ''Used at: '' | ||
''[[Role_Ovo_Energy|Ovo Energy]]'' | ''[[Role_Ovo_Energy|Ovo Energy]]'' | ||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | </div> | ||
</div> | </div> | ||
data migration | data migration | ||
Databricks Compute (clusters) | |||
Widgets (variables) | |||
Backup tables (job) | |||
Databricks notebooks | |||
SQL MERGE | |||
== Python == | == Python == | ||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Streamsets == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
== Azure == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Azure Blob Storage == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
</div> | |||
</div> | |||
== Azure Delta Lake == | |||
<div class="mw-collapsible mw-collapsed" style="width:400px; overflow:auto;"> | |||
<div>''Used at:...''</div> | |||
<div class="mw-collapsible-content"> | |||
''Used at: '' | |||
''[[Role_Ovo_Energy|Ovo Energy]]'' | |||
''[[Role_SSE_Energy_smart_meter|SSE Energy smart meter]]'' | |||
</div> | |||
</div> | |||
Latest revision as of 18:46, 2 October 2024
Spark SQL
Used at:...
Used at: Ovo Energy SSE Energy smart meter
Spark SQL
input_file_name()
Refresh Table
current_timestamp()
ifnull(a, b)
reverse(a)
Datatypes: timestamp, string, int, bigint, date
Databricks
Used at:...
Used at: Ovo Energy SSE Energy smart meter
data migration
Databricks Compute (clusters)
Widgets (variables)
Backup tables (job)
Databricks notebooks
SQL MERGE
Python
Used at:...
Used at: Ovo Energy
Streamsets
Used at:...
Used at: Ovo Energy SSE Energy smart meter
Azure
Used at:...
Used at: Ovo Energy
Azure Blob Storage
Used at:...
Used at: Ovo Energy
Azure Delta Lake
Used at:...
Used at: Ovo Energy SSE Energy smart meter