Exam DP203 Azure Stream Analytics

Azure Stream Analytics.

Unbounded - no limit to the amount of data.

Temporal - each piece of data has a datetimestamp.

Aggregation over these temporal windows.

Real time or near-real time analysis.

Create a Stream Analytics job. Optionally create a Stream Analytics cluster.

Stream Analytics jobs are resources that should be created in resource groups.

reference inputs for static data

SELECT observation_time, weather_station, temperature
INTO cold-temps
FROM weather-events TIMESTAMP BY observation_time
WHERE temperature < 0

A field named EventQueuedUtcTime is automatically created to define the time when the event is received into the event queue.

EventProcessedUtcTime is the time it is processed.

Window functions

Window functions provide aggregation of streaming data over time periods. These are implemented by select statements on the source data with the following GROUP BY clauses (can also use HAVING statements):

Tumbling: Fixed time periods. GROUP BY TumblingWindow(minute, 1)

Hopping: Same as Tumbling, but where the time periods overlap. GROUP BY HoppingWindow(second, 60, 30)

Sliding: Periods start/end only when there is an event. There can be more than one event per period. GROUP BY SlidingWindow(minute, 1)

Session: Window opened by receipt of an event, and closes x seconds later, unless there is another event in this period, and so on until y seconds after the start. GROUP BY SessionWindow(second, 20, 60)

Snapshot: A period for every event. GROUP BY System.Timestamp()

Ingest streaming data

Create a Power BI Dashboard.

SELECT
    EventEnqueuedUtcTime AS ReadingTime,
    SensorID,
    ReadingValue
INTO
    [synapse-output]
FROM
    [streaming-input] TIMESTAMP BY EventEnqueuedUtcTime

Output to Power BI

There is a Power BI output type.

Use window functions to ensure data is sent to Power BI no more frequently than every second