Great news from Axel Thevenot – you can now orchestrate your queries directly in the BigQuery console, without diving into Dataform.
It’s a great alternative to Scheduled Queries.
Essentially, Workflows is a tool for automating and managing data processing tasks. You can create sequences of ETL tasks that automatically execute in a specified order.
Workflows are powered by Dataform and use SQL queries and Notebooks as their assets.
Each workflow is created right in the console and can include one or more elements. You can also define the execution order of the blocks. For instance, to train a machine learning model, a SQL query can first prepare the data, and then a notebook can use that data to train the model.
Workflows also allow you to schedule tasks. For example, you can set up a daily SQL query to update a table with the latest data, which powers a dashboard.
In short, Workflows make it easy to manage complex processes by combining multiple code elements into a single workflow and controlling when they run.
If you work with GA4 to BigQuery exports, be sure to check out my SQL cheat sheet.
