CM Syntax - Introduction

Updated: Jul 27

Hey! How's it going?!

I'm glad to see you're still interested and want to learn more. In this post, we will uncover one of - to me at least - the most interesting aspects of CM: Syntax Extensions.

If you are familiar with languages such as Lisp (Lisp's Macros are especially fun and powerful when compared to the others in this list), Python, C++, and Boo (or to a lesser degree, the .Net platform) you might already have leveraged the preprocessor's capabilities beyond the "standard" language specification into the realm of meta-programming.

The key aspect here is to be able to expand the SLE for whatever platform you are working on into something that fits your requirements in a more efficient and/or friendly manner.

You can potentially use this capability to:

  • Create a DSL to better map the code to the business jargon of your project;

  • Add new features to the underlying GPL you are working with;

  • Create templates to reduce code redundancy;

Hello Syntax

Without further ado, I think it's time for us to get to work and create our first CM Syntax Extension (which from now on I'll refer to simply as CSE).

There are several different "types" of syntax structures in CM. You can find them all just by searching for “extends Syntax” inside the base CM repository.

Core CM's CSE Structures
Core CM's CSE Structures

This list contains all of Core's standard CSE-related structures. One could argue that even though the topic itself is somewhat dense, at least we don't have a whole lot of different classes to learn, right?

In fact, we are not even going to explore all of these just yet. For the purpose of this post, we are only going to look into a few of those and try to get a really solid understanding of what really is this CSE thing and how this infrastructure works so we can leverage this knowledge into the next steps of our journey towards CM Closures.

For our very first syntax extension, we are going to work with Statements.

...a syntactic unit of an imperative programming language that expresses some action to be carried out... A statement may have internal components (e.g., expressions). - Wikipedia

I think that based on the above description extracted from Wikipedia you can tell that I kinda lied a little bit when I said that we'd be looking first into Statements. That's because in order to understand and utilize statements you first need to create the expressions that will be part of it. Worry not though, CSE got you covered!

CSE's Expressions

I don't really think I could do a better job explaining these concepts with my own words so, let's start by taking a look at what Wikipedia also has to say about Expressions:

...a syntactic entity in a programming language that may be evaluated to determine its value... a combination of one or more constants, variables, functions, and operators ... to produce ("to return", in a stateful environment) another value.

In short: Expressions are instructions that produce - in CET's case also return - a value inside your program.


Luckily for us, Core also brings some CSE expression as part of its source code and you can - and should - look what's under the hood by searching "public expr" in your base directory.

We are not going to explore any existing syntax tough. I believe we are capable of starting off with something simple but complete enough to provide you with sufficient understanding to navigate your way across the several examples you will find in Core, all by yourself.

Time 2 Code

Our first Expression syntax will be, as you probably guessed, a Hello World. We will create a syntax that when used will pln and return the message "Hello World from syntax".

Here's the code we will be using:

Go ahead, run this and you will see the following output:

Hello World from syntax
Hello World from syntax

Looking at the code I gave you do you see what we did there (in terms of basic syntax expansion)?

Let's explore the highlights of our little experiment:

  • We've expanded the semantic behind the exclamation point and made it part (a required part, by the way) of our newly created expression;

  • We've successfully wrapped a set of instructions inside a construct that CM preprocessor will evaluate during compile time;

Under the Hood

Let's expand on the previous highlighted items and add some more information on how this magic works behind the scenes while introducing some points you should keep in mind while working with CSE.

Syntax resolution

The easiest way to explain how this works is if you picture the usage of CSE's as if it were using a small Bot that goes around your CM code finding all the places from which you called it and replacing the code in there with the code you've placed inside its own definition file.

Our little Bot friend is smart enough to preserve the context in which you have called it while also being versatile allowing parameterization in a way that it can replace the placeholder call with the exact expressions, statements, and/or members (we will cover those in the future) you've asked it to put in there.

To better illustrate this Bot abstraction let's take a look at what actually is compiled when we run the helloWorld! extension above:

Wait, what?!

That's right, no sign of the original syntax declaration. It's gone. Our Bot friend is not part of the final code, it is only used to generate the code that will be used as the final code.

CSE Source Code Update

Understanding that the syntax code doesn't actually exist anywhere on the final code is an important part of a happy life while coding for it. It could get really frustrating very fast if you are trying to do something using Syntax and the compiler is yelling at you or, as it happens more often than it should, your code just won't update at all.

Say you have the following files:



For simplicity's sake, let's assume that both these files are located under the same directory and share the same file and package references.

Here is the source code for both files:

All pretty standard at this point, right? It is, and in fact, it will actually work exactly as you would expect based on what we've seen so far.

Go ahead and compile those files and run the code in You will see something like this:

Hello World from syntax
Hello World from syntax