Language-independent programming (Part 1; the basic motivation)

I have been gradually coming to believe that one of the biggest problems with programming is that code lives in text files that are written in a particular programming language, and that programmers directly manipulate these text files.

The improved alternative that I would like to propose is for code to live in abstract trees with reverse-compilation hints that allow it to reverse-compile differently depending on context.

Let me give you a little bit of motivation for this idea.

Let's say I'm building a complicated object for an end-user to manipulate and interact with via a computer, and I want be able to improve my end users experience of interacting with this object over time. For example, let's say I've created a computer game. If I were to write a computer game, I wouldn't give my users tools to directly modify the files that their computer stores to keep track of their progress and let them quit the game and reload it later. Instead, I would give the user the ability to play the game, and I would write utilities to allow the user to save the game... i.e. to abstract from the game state sufficient information that the program I wrote could recreate the game-state that the user had reached the next time the user loads the game.

This process of saving an abstraction of what the user was interacting with and then converting that abstraction back into game state later allows the game creators a lot more freedom to update the game. Let's say a particular gun is overpowered and the game creators decide that they need to nurf that gun to make the game more fair. Since the saved game-state only has a reference to that gun rather than a full description of that gun, the programmers only need to update the game, and they don't have to update the saved file at all.

Now, switch over to programming languages. Let's say I wrote a programming language in which I had statements like 'print' or 'exec' that were a little bit different from functions, mostly with regard to their syntax but that otherwise behaved a lot like functions. Let's say that after twenty years of developing this language, I decided that having these things be statements instead of functions was a bad idea for several reasons, both in terms of the way the language internally distinguishes between functions and statements (statements cannot be overridden with an alternative implementation but functions can), and how the syntax works. It's a bit clunky to have these particular concepts be key words or to have them have different syntax rules from functions.

Now, I want to release an update to the programming language, but I really can't do it because doing so would involve requiring people who have been using the language for the past twenty years to change a whole bunch of files that use syntax that is no longer consistent with what the update would require, so instead I create a backwards-incompatible update and cease to actively update the old programming language and cause there to be a great schism between code compatible with the old way of doing things and code compatible with the new way of doing things.

It's a step forward, but it isn't a very tidy one.

It would be a lot nicer if instead of having to update these files, we could just have them reverse-compile to the appropriate syntax when we decided that the change needed to be made. (And forward compile to behaving like functions instead of behaving like statements. To satisfy both constraints we would need something that is not truly compiled, but capable of compiling in either direction.)

Now, let's consider a second point: programmer preference.

Let's say I really don't like programs to have more than four spaces worth of indentation per line, and in fact I prefer two. My coworker would rather have eight spaces worth of indentation, and he would rather have that spaces-worth of indentation be encoded in tabs than in spaces (we're working in a language that doesn't really care which one we use). So we pick one of our preferences and stick to it, but both of us are working on other projects that we code according to our own preferences, so we occasionally get confused and write code that mixes the two.

Again, it would be a lot nicer if somehow we had the code stored in a way that would cause it to be written back to us in a manner consistent with our preferences instead of a way that forced us to look at the same file of text even though we would both rather that it be formatted in different ways from each other.

For an even stronger disagreement of preferences, we use different editors. In my co-workers editor having 100 characters to a line makes a ton of sense, but in mine, it's best to keep it to 80. Again, the same thing.

We can go even further. After we start working on this project, I realize that having
3+5 * 4
represent
32
makes a whole lot more sense to me than having it represent
23
and that furthermore, I find this representation to be significantly more readable to me than
(3+5) * 4
or
(3 + 5) * 4
My co-worker is a purist who wants to keep the traditional interpretation of parentheses and thinks that that having whitespace sensitivity for that sort of thing in a programming language is a BAD idea.

So I install a module that lets me represent the code the way I want it represented, and he leaves it set up the default way.

And all of this is just the beginning of what you could begin to do differently (and better) if you found a way to store code in something abstract with guidance for how to be reverse-compile back to what the programmer wanted.

If you were going to build this system you would start by doing something that doesn't do too much in the way of producing abstractions, but you could systematically build out and maintain this sort of system much more easily than you can systematically build out and maintain the sorts of things that people currently write.