Language Oriented Programming or LOP is a software development approach which employs domain specific languages, or DSLs for short. A domain specific language, in contrast to general purpose languages, is a language which is designed to deal with one specific kind of tasks or with several related task kinds within a specific domain. For example, SQL is a DSL in a domain of database queries. LOP solves your main problem by dividing it into several subproblems in different domains, then each subproblem is solved using an appropriate DSL. In rather big projects, LOP can reduce singificantly the amount of work which has to be done.
In this article we will speak about JetBrains MPS, being developed by JetBrains, Inc., which is a so-called "language workbench", i.e. a tool that helps you to use LOP in practice. You can read more about language workbenches on Martin Fowler's site and about LOP in Sergey Dmitriev's article.
A lot of people who heard about MPS don't realize how it's possible to take advantage of it. The current state of MPS is far from the point where you can throw away your Java or C# IDE and start writing everything in MPS with LOP, however it's possible to use LOP and simplify some tasks significantly today. Actually, you probably use in your current project some kind of poor man's LOP, techniques which make it possible to create small sublanguages in the program written on a language like Java, C#, or Ruby (Martin Fowler called such sublanguages Internal DSLs). This article explains the pattern that can employ MPS in its current state, describes prevalent poor man's LOP tricks and presents several examples of it.
Let's consider some typical component of a program. We can divide it in two parts: the framework and its usages. Frameworks are usually reusable, well developed and tested; they contain domain model's description in some programming language. Usages, in respect to the framework, are rather trivial code that follows conventions created by the framework. This code often looks like code in a mini-language embedded in a program on a basic language. Of course, one framework might use another framework for its implementation, but the code concerning the first framework in the second one is rather trivial anyway.
We can simplify this with framework-usage pattern. First, we wrap the framework with a language, and use this language where we used to use framework. Than we write a generator for the framework which converts constructs of the language into framework's method calls. This simplifies both the framework and the usages. We simplify framework, because we don't have to design convenient API with a lot of defaults anymore, all of this is moved to the language, your framework might be as simple as a bunch of static methods. Usages, because we have full power of language designers, we can design new constructs and optimizations which can't be expressed in mainstream language.
In the next sections we present examples of such framework-usage separation, describe some tricks that can improve this code and make it more convenient to use, present the framework-usage pattern improved with LOP and finally describe the pattern in detail.
We are all familar with web applications. Today they are so widespread that almost every person uses them not to mention software developers. Enormous number of frameworks that simplifies web development this pattern was developed. Most of them contains poor's man LOP tricks:
Often in Java-like language frameworks we can find language-like tricks that make code which uses this language-pretending framework more concise and convenient to use. These tricks include convenience methods, reflection and XML metadata. We describe them in detail in the next section. Martin Fowler in his article about language workbenches calls them Internal DSLs.
If we use standard OOP language's constructs in a slightly unusual way, we can benefit from it.
Consider an XML manipulation library. If we want to create an XML element we could write this:
Element record = new Element("record");
Element name = new Element("name");
name.setText(name);
Element address = new Element("address");
address.setText(address);
record.addContent(name);
record.addContent(address);
But if we make all modifiers returning this, we can make this code more concise and convenient:
Element record =
new Element("record")
.addContent(
new Element("name")
.setText(name))
.addContent(
new Element("address")
.setText(address));
This piece of code is more readable than that we have written above because you can see a created XML element's
structure right from the code. This can be improved even more if we were able to embed XML into code written
in basic language (here, into Java code).
In some scripting languages like Python, if you want to create a list, you can write this : [1, 2, 3]. Unfortunately, Java doesn't have such a syntax. We have to write something like this:
List l = new ArrayList();
l.add(1);
l.add(2);
l.add(3);
But with vararg method we can define this:
static List list(Object... os) {
List result = new ArrayList();
for (Object o : os) {
result.add(o);
}
return result;
}
Now, if we want to create a list filled with given elements we can write this:
list(1, 2, 3);
Of course, it isn't as short as in Python, but it's much better than our first Java version.
You can use static methods to create a sublanguage for some domain. This trick is widespread in java.lang package. If you want to convert a array to a list you can write this:
Arrays.asList(array);
Or if you want to make some list unmodifiable you can write this:
Collections.unmodifiableList(list);
When you work with abstract syntax trees you can create one helper class with static methods and than
create ASTs in a very convenient form:
import static com.company.htmlast.HtmlAstUtil.*;
return
html(
head(title(pageTitle)),
body(....));
HtmlAstUtil will look like this:
class HtmlAstUtil {
public static HtmlAst html(HtmlAst... children) {
...
}
}
This example looks much like the example above where XML elements were created. The difference is
that in the example above the instance methods of the created elements were used to construct their subtrees,
while in the current example they're static methods from a helper class which does all the job.
Reflection is a very powerfull thing, especially in dynamically typed language like Ruby, Python or Smalltalk. For example, in the ActiveRecord framework from Ruby On Rails, you can create a persistence mapping by writing this code:
class User < ActiveRecord::Base
end
When you use this class as in code below, a method call will be intercepted by ActiveRecord and it will try to
find a field named "email" in the database. You don't need to explicitly declare email field in your code:
u = User.find(id);
email = u.email;
Of course it's impossible to do so in a statically typed language like Java.
Unfortunately, this approach has one disadvantage: it's very hard to create an IDE for dynamically typed language, and this task becomes almost impossible when we have a lot of internal DSLs. It's very hard for IDE developers to make code completion and refactorings available for dynamic languages. In the case when code resembles Java code, when we have some DSL implemented with interception, the task becomes even harder: we have to write an IDE extension for every DSL.
Everyone who's familar with the old version of JUnit knows how it works with tests. Instead of creating a class for every test, it uses a convention that in any class that extends junit.framework.TestCase any method which name starts with "test" is treated as a test.
public class TestSomething extends TestCase {
public void testA() {
...
}
public void testB() {
...
}
}
In JDK 5.0 the source metadata were introduced. You can mark methods, classes, fields and parameters with annotations: a special kind of sign to attach metadata. In the latest JUnit version you can mark any method in a class with @Test annotation, and it will be treated as a test:
public class TestSomething {
@Test
public void a() {
...
}
@Test
public void b() {
...
}
}
XML has a very rich framework for XML-based language creation. It has several kinds of schema, several kinds of query languages. Most of these languages are based on XML itself, So it's no surprise that many existing frameworks use XML to present their data. But XML is rather verbose and quite hard to read and edit (a good editor, like the one that is built into IntelliJ IDEA simplifies this task), so it's much better to use a tool like MPS to create langugages than use naked XML.
As we can see, developers have invented a lot of tricks to create sublanguages inside their basic language, but most of them are just a poor man's LOP. So let's go on and see how similar tasks can be solved with MPS.
Consider the following framework: JDOM library. It contains some methods for reading XML files and writing into XML. Such methods like getChild, getAttribute and so on. We could imagine a DSL for more convenient work with such methods. And actually, we have written a small language for XML reading, which is generated into JDOM method calls.
Language allows writing such constructions as following: d.hospital.divison[8].name, which means, if d is a JDOM Document, "read an attribute 'name' of 8th element named 'division' of element 'hospital' of document d". To write such constructions, you obviously must have a kind of schema or description of structure of your document, to let MPS know which elements may contain which children and attributes. So, first we'll consider document structure description.
Below it's shown a structure description for an element hospital.
element hospital tag name: <default> properties: name : stringType children: division cardinality= 0..n mainDoctor cardinality= 0..1As you can see, hospital contains an element representing its main doctor, and some elements representing hospital's divisions. Also an element for hospital may contain an attribute called "name". The optional "tag name" is filled if you want different names for your element in DSL code and for your element's XML tag. By default, tag name is equal to the name of an element, but you can change it by filling this field in.
Now when we have structure description, we can write something like
document hospitalDocument d = new Document ( ) ;
System . out . println ( d . hospital . division [ 3 + 4 ] . name ) ;
for ( element ward el : d . hospital . division . ward ) {
System . out . println ( el . name ) ;
}
The generated Java code will look like below:
Document d = new Document();
System.out.println(((List<Element>)d.getRootElement().getChildren("division")).get(3 + 4).getAttributeValue("name"));
for(Element el : ((List<Element>)((List<Element>)d.getRootElement().getChildren("division")).get(1).getChildren("ward"))) {
System.out.println(el.getAttributeValue("name"));
}
We just have considered a small but illustrative example of usage of DSL-framework pattern. We've created a small convenient DSL in MPS to read XML files with JDOM as a framework. DSL provides a domain-specific, clear interface to JDOM. Code in our DSL is generated to framework calls.
First let's consider the framework-usage pattern for a basic language and then for a LOP tool like MPS.
In Java-like languages this pattern seems obvious, so it's hardly can be named a pattern, but because it becomes really useful in LOP we describe it.
Contains the most generic code. Because we use generative approach here it contains considerably fewer convenience methods or doesn't contain them at all.
Task-specific code uses framework to simplify programming task. It uses framework's language-like constructs described above to make code closer to the domain, but usually most of these tricks aren't used by framework authors. Look, for example, at the Java standard library which can be improved in many ways.
This structure is outlined on the image below.
In the mixed case, a new level is added: the MPS level. Respectively, two more components are added: The Language and The Language Usages. The former Task-Specific Code (belongs to Java level) becomes generated Task-Specific Code.
The Language presents concepts implemented in the framework in a more convenient form. The Language is created using MPS and belongs to the MPS layer. The Language contains a lot of constructs that play the same role that convenience methods would play if we didn't use LOP.
The Language Usages is code or you can say a model written in our Language (at the MPS layer). It represents the solution of your task, so we can say that The Language Usages is a task-specific code written in the domain-specific language.
This modified structure is outlined on the image below. New boxes added (compared to the previous presentation) are marked with blue.
Let's consider advantages and disadvantages of the approach described.
Advantages:
We have reviewed several DSL-like approaches, which help developers create usable API for their frameworks. With MPS, however, you are enabled to express your API not with such DSL-like tricks, but directly, with real DSLs. Hence, even if MPS in many cases yet can not be used for writing a whole application in LOP manner, you can start benefit from it using it with the 'framework-usages' pattern.
We would like to thank the following people for their reviews, comments, suggestions and help: Igor Alshannikov, Sergey Dmitriev, Vadim Gurov, Dmitri Kirillov, Maxim Mazin, Timur Zambalayev.