Skills Spark SQL

Spark SQL

Used at:...

  Used at: 
  Ovo Energy
  SSE Energy smart meter

Spark SQL

input_file_name()

Refresh Table

current_timestamp()

ifnull(a, b)

reverse(a)

Datatypes: timestamp, string, int, bigint, date

Databricks

Used at:...

  Used at: 
  Ovo Energy
  SSE Energy smart meter

data migration

Databricks Compute (clusters)

Widgets (variables)

Backup tables (job)

Databricks notebooks

SQL MERGE

Python

Used at:...

  Used at: 
  Ovo Energy

Streamsets

Used at:...

  Used at: 
  Ovo Energy
  SSE Energy smart meter

Azure

Used at:...

  Used at: 
  Ovo Energy

Azure Blob Storage

Used at:...

  Used at: 
  Ovo Energy

Azure Delta Lake

Used at:...

  Used at: 
  Ovo Energy
  SSE Energy smart meter