4.0 Messages FAQ

VSCode

There are a few places where Malloy 4.0 will require existing code to change. In the transitional period, both the 4.0 and Pre-4.0 syntax is accepted, and a warning will be generated for Pre-4.0 constructs, to give users guidance in changing their code. Once 4.0 fully releases, these will be errors and not warnings.

This is a list of all the 4.0 compatibility messages, with examples code showing how to transform existing code for Malloy 4.0.

Extend a source with extend

Creating a new source by adding new propertied to an existing source is called extension in Malloy. There have been three different ways to express extension of a source, now only the form source extend { extensions } is accepted.

 source: flights is table('duckdb:flights.parquet') {   primary_key: id }

 source: flights is duckdb.table('flights.parquet') extend {   primary_key: id }

Refine a query with +

Creating a new query by adding properties to an existing query is called refinement in Malloy. There have been three different ways to express refinements, now only the form query + { refinements } is accepted.

query: flights -> by_carrier { limit: 5 } query: flights -> by_carrier refine { limit: 5 }

Sources contain views

The object inside a source which can be used to build a query is now called a view:, it used to be called a query:

 source: flights is table('duckdb:flights.parquet') {   query: by_carrier is -> {     group_by: carrier; aggregate: flight_count is count()   } }  

 source: flights is table('duckdb:flights.parquet') extend {   view: by_carrier is {     group_by: carrier; aggregate: flight_count is count()   } }  

select: instead of project:

The project: keyword has been renamed to select:.

New syntax for count(*) and count(distinct)

The SQL syntax for distinct counts count(distinct expression) is deprecated in favor of count(expression).

query: people -> {     aggregate: distinct_first_names is count(distinct first_name) }

run: people -> {     aggregate: distinct_first_names is count(first_name) }

The SQL syntax for counting the records of a table count(*) is deprecated in favor of count().

{? } deprecated

The old filter shortcut syntax {? condition } is deprecated. Use the more explicit { where: condition } syntax instead.

Nesting Malloy in SQL with %{}% is now %{}

Query interpolation in SQL strings should use } to end the interpolation rather than }%.

 sql: sql_source is {    select: """SELECT * FROM (%{ flights -> by_carrier }%)"""   connection: "bigquery" }

 source: sql_source is    bigquery.sql("""SELECT * FROM (%{ flights -> by_carrier })""")

-> no longer begins a named query or view

It used to be necessary to use an arrow (->) to base a new source or query off of an existing query. This is no longer necessary.

A leading -> operator no longer signifies the beginning of a query pipeline.

SQL and tables are accessed through the connection

When referencing a SQL table, the connection name should now be specified connection_name.table('table_path') instead of table('connection_name:table_path').

Prior to Malloy 4.0, SQL blocks were declared using sql: sql_block is ... and used with from_sql(sql_block) .... Now, SQL blocks are created with the connection_name.sql(...) syntax and can be used as sources directly.

 sql: sql_block is {    select: """SELECT * FROM (%{ flights -> by_carrier }%)"""   connection: "bigquery" }   source: sql_source is from_sql(sql_block)

 source: sql_source is    bigquery.sql("""SELECT * FROM (%{ flights -> by_carrier })""")

Prior to Malloy 4.0, SQL blocks were declared using sql: sql_block is .... Now, SQL blocks are created with the connection_name.sql(...) syntax.

 sql: sql_block is {    select: """SELECT * FROM (%{ flights -> by_carrier }%)"""   connection: "bigquery" }

 source: sql_source is    bigquery.sql("""SELECT * FROM (%{ flights -> by_carrier })""")

Queries can be used as a source without from()

Prior to Malloy 4.0, the from() function was required to convert a query into a source. Now a query can be used directly anywhere where a source is needed.

 source: source_from_query is from(flights -> by_carrier) {   measure: carrier_count is count() }

 source: source_from_query is flights -> by_carrier extend {   measure: carrier_count is count() }

Extensions to view objects move to an extend: {} section

When the ability to extend the source of a query operation was first introduced, the extensions were written alongside the query. Now those extensions should be written in an extend: block inside the query.

On the fly joins in queries should now be specified in an extend: block.

 run: flights -> {   join_one: carriers with carrier   group_by: carriers.nickname }

 run: flights -> {   extend: { join_one: carriers with carrier }   group_by: carriers.nickname }

The declare: keyword has been deprecated, and should be replaced with dimension: or measure:. If the declare: is used inside a query, it also needs to be put into an extend: block.

 source: flights is table('duckdb:flights.parquet') {   declare: distance_km is distance * 1.6    declare: flight_count is count() }

 source: flights is duckdb.table('flights.parquet') {   dimension: distance_km is distance * 1.6    measure: flight_count is count() }

 run: flights -> {   declare: distance_km is distance * 1.6    declare: flight_count is count()   group_by: distance_km   aggregate: flight_count }

 run: flights -> {   extend: {     dimension: distance_km is distance * 1.6     measure: flight_count is count()   }   group_by: distance_km   aggregate: flight_count }

top: ... by removed

Previously, you could specify an ordering field in the top: specification. Now you should use order_by: explicitly.

query: flights -> {    group_by: carrier   aggregate: flight_count   top: 5 by flight_count }

query: flights -> {    group_by: carrier   aggregate: flight_count   limit: 5   order_by: flight_count }

Write nameless queries with run:

query: should now only be used to declare a query with a name. To run a query without naming it, use run:.