Exam 203 back-end services Spark

Languages supported in Spark include Python, Scala, Java, SQL, and C#.

To run Spark code it is necessary to first create a Spark pool. Then in Synapse Studio Develop tab, create a new Notebook, and select the Spark pool created to it. Then paste the following code into it and run:

%%pyspark
df = spark.read.load('abfss://files@datalakexxxxxxx.dfs.core.windows.net/product_data/products.csv', format='csv'
## If header exists uncomment line below
##, header=True
)
display(df.limit(10))

Note the first time it runs it will take several minutes to complete because it takes the time to start up the Spark pool.