Practical Programming Language 01 - introduction

Hello and welcome to this new Youtube channel about the Practical Programming Language. My name is Shachar Shemesh (that’s actually a very reasonable name in Hebrew) and we are here today to start a new channel that follows the development of a new programming language which I decided to call Practical, and I’ll talk a bit later about why. We’re going to be talking about four main topics. The first one is a little about me. Who I am, professional background, where I come from. The second one is we talk about is why start a new programming language, especially in this day and age. Why the existing programming languages aren’t good enough, and I’ll talk mostly about four languages that are kind of trying to cover the same areas as practical which are, obviously, C++, D, Rust and Zig.

We’ll talk about what drove me to start working on Practical 01:25 - and we’ll talk about kind of the manifest, the Practical manifest, of what I’m envisioning Practical to provide when it’s mature enough for you to use. So let’s start with the first one. I’ve been a programmer now for almost 40 years, of which about 30 years I’ve spent as a professional programmer, i.e. programming for money. I did originally started on the Apple II computer, then I moved to the Amiga computer, may it rest in peace, and then I reluctantly moved to the PC world, but switched my main focus fairly quickly to Linux and writing open source. I have several open source projects that I wrote from scratch including rsyncrypto, fakeroot-ng and sshpass, which is a project that I really spend very little time on and is, by far, my most used project of all, despite the fact that its own documentation says do not use this program. Professionally I worked two years at Checkpoint as security team team leader.

That meant doing Incident response whenever someone found a 02:59 - security problem in one of Checkpoint’s products, and doing development of feature that are more security oriented. I’ve worked, I’ve been a freelance for my own, my own company called Lingnu open source consulting for a little over eight years. I’ve spent a little over a year at Akamai. They’re a huge company which almost everyone uses but few have heard of because, they route about a third of the internet traffic and what they do is they are a CDN, that’s content distribution network and what happens, if you connect, if you’re trying to download an update for your operating system from Microsoft or from Apple, then what happens is that you get a URL that says “Microsoft” or says “Apple”, but actually you’re downloading the content from the Akamai network, which allows them to bring the content closer to you, thus save on overall bandwidth. So about a third of the Internet traffic goes through the Akamai network and for four years I worked at a company called Weka IO which develops software- defined storage program it’s distributed software that together clusters the CPUs into forming or exporting a coherent single file system and I mentioned that because it’s an interesting company but also, but also because, they were developing in D so for a little over four years almost four and a half I was developing in D as my main daytime job development platform. And when we talk about Practical that’s a very important fact because a lot of the reasons for choosing to work with practical, to work on practical, came from this experience from working with D, and understand, on the one, hand understanding where C++ doesn’t provide everything you’d want it to, the things that, opening my eyes, or expanding the horizon to things beyond what C++ provides on the one hand, and understanding that there are deficiencies in the current language of current leading languages for system’s programming, for low-level development of high performance, low latency, applications.

Which obviously leads me to asking where practical is with a 06:03 - map of languages. How it relates to other programming languages. I think the main sources of inspiration for practical are, obviously, C++ which you cannot ignore and then, of course, D and Rust. I’ll discuss why none of those are good enough on their own. I’ll also explain where Practical wants to expand beyond what they provide. So let’s start with C++. C++, for the record, I love C++. I think one of the main things that are missing, the languages that currently tried to replace, or fit into the same category, as where C++ is, of object-oriented polymorphism on the one hand, but system programming on the other, one of the problems is that I feel like that the people writing those languages do not love C++, do not get C++.

Therefore they’re trying to, for me, 07:27 - their aim is a little off. But C++, C++ is a language that gives you a lot of power and one thing that I’m particularly fond of in C++ is that, when you create a library in C++, create a container or template or in any form of… you write code that is meant to serve other programmers when they write their own code, C++ gives you a lot of power to encode the semantics of your library into the interface in such a way, that the compiler then enforces that semantics. Which means that, when someone makes a mistake, that mistake of semantically using your library in a semantically wrong way, that mistake translates into a compilation error. And that’s something that I really really love about C++ and something that I wouldn’t want to see diminished.

And when 08:37 - I was working on D I often felt that to be the case. I often felt that D, while really introducing a lot of things that C++ either didn’t have or didn’t have easily, they kind of lost track of that. So D did not put a strong enough emphasis, for my liking, on allowing me as a library developer to encode the semantics of what I’m writing into such a way that anyone using the library will be forced to follow that semantics. So that was one problem I had with D. The other is that the different components didn’t seem to fit well together. It’s like cog wheels that are incorrect, imprecisely manufactured, that when they rotate they make me sound, this churning sound.

You got a feeling that the different language features were not 09:47 - orthogonal so if you could use one feature to go from here to here and another feature to go from here to here, you couldn’t shift it and just go here using those augmentation of those two features. They had interactions that prevented you from using them in such a way. And I felt the language was not focused enough for me to enjoy using. Having said that, it has a lot of features that are really really really nice. The compile time execution that allowed me allowing you to write code that goes, that gets executed during the compilation of the program rather than when it’s run, and the introspection: the capability of that compile time executed code to ask questions about types, about the program, about functions, and then modify itself based on the answers, that was something that was really really strong with D, and something that I realized that when, I was leaving Weka, that I’m going to miss.

And that brought in a very long-standing dream of mine, or fantasy 11:14 - if you want, of writing a programming language and sort of put content into it. So now I wasn’t just “hey I’m gonna write a programming language”. I said “hey, I know what that programming language needs to have for me to want to use it”, and hopefully for you. And that sort of focused my plans or turn them from some fantasy to concrete “hey I’ve got something to say”. Now, with that in mind, I actually think D’s runtime execution is not where I want runtime execution to be, because I think the main deficiency I had with it… there were two, actually.

One was that quite often and despite having introspection and 12:12 - compile-time execution, you found yourself using tricks that are best described as C++ template programming, i.e. you were using template instantiations with recursive instantiation of specialization in order to get the semantics you want. You were not writing the code directly in order to do that. And I found out that, at least as far as the planning goes, I think I can do better. The other area I thought these runtime execution wasn’t good enough was around algorithmic containers.

12:54 - I think I happened on it when I was trying to use a hash table for data that is completely known at compile time. So I wanted a hash table that was filled during compile time, and then during runtime there are only lookups. And what I realized was that I have to write a new hash table in order to do that. I cannot write a single code that will match both compile time known data and dynamic data the changes at runtime. I can’t write a single container that would do both because either I know the type at compile time or I do not know the type.

Either I know the content at compile time or I do not know 13:52 - the content at compile time. And that made me realize that these compile time execution improvement over C++, though it and undoubtedly is, was still leaving areas where we can do better. And that sort of was the focus of Practical when I started out with it. And one of the questions I get most often was why not Rust. Either why not just go with Rust, or why not use Rust and expand on it.

I’ll preface by saying, I do not know Rust as thoroughly 14:47 - and as well as I know C++ and D. So if you’re a Rust enthusiast and feel strongly about something I say, please feel free to say something in the comments and I’ll do my best to look at it. From the research I’ve done, Rust is chock-full with good ideas. When I started toying with the idea of a programming language I kept saying “well in Practical… I will have variables will be const by default”.

And the answer I would, almost 15:31 - without fail, get to such statements was “that’s the way it’s in Rust”. And when I look at Rust I see a lot of very good ideas, really good ideas. My problem with Rust is that, instead of taking those ideas and augmenting the language with them, Rust sort of says, you know, “that’s the path, you should follow it”. And this sort of dogmatic approach, in my opinion, rarely survives well when it comes in contact with everyday programming. The Rust answer to that dilemma, which does come up, is to sort of double down.

To say “Oh 16:27 - so we’ll do more runtime libraries and more twists and turns in order to keep the core dogma in place”. And my approach to programming is completely different I would like to work with a program, and C++, to large extent, does that, that doesn’t say “that’s the way you have to do it”. It says “well, you know, if that’s the way you want to do it, we’re giving you tools to make it easy. But if you want to stray off the path, we’re not forcing you to stay there”. And I think that’s a much more correct approach to programming, to programming languages, and it actually focuses on why I called the language “Practical”.

17:24 - It actually gives, in the title of the language, my philosophy of how a programming language should be. I think programming language is a means to an end and I think that it should give you the tools to be effective, but should not force you to use those tools. And whenever I say that to a fan of Rust, the answer I invariably get is use “unsafe”. And the problem here, I think, runs deeper than encapsulating a part of the code with unsafe, because we’ve seen cases where writers of very prominent libraries in Rust being criticized for using unsafe. So the community sort of doesn’t really see it as a solution.

And I think the Rust 18:30 - approach is too dogmatic to be, I’m sorry about the pun, practical. One last language that must be mentioned at this point is Zig. The reason Zig must be mentioned is because I started working on Practical, I started trying to fill in what I want Practical to do, and then, when I actually showed it to people, they said “you know there’s a language that cehcks quite a lot of those boxes”. And that’s Zig. You remember my compile time known data hash table example from before? Well the Zig videos actually have an example of that and calculating what is called perfect hash for it. A perfect hash is a hash where you play with the parameters so that there are no collisions, and then you can write the code assuming there are no collision, and the code gets simpler for that. And that’s not the only example.

I 19:45 - think the most prominent one was, I figured out that I want “type” as a first-class citizen, so you can have variable of type “type”, and then say “a=int”, then give manipulations and that. Zig does that as well. It checked so many of my big ticket items for Practical that I actually stopped and said: “Does it make sense to develop Practical? Maybe it makes more sense to see whether I can fit some of my small ticket items and bigger ticket items missing from Zig into the existing language?” Sadly, the answer is: no it does not. First of all, Zig takes a very different approach to the small ticket items then what I took with Practical, and I think the small ticket, I won’t fit it into this video, but the small ticket items are important. In fact, arguably they are more important than the big ticket items. Once I started working on Practical, I realized that I have a lot of improvements on the “small tickets”, things like how type promotion works, how overload resolution works, that improve dramatically over the way C++ does it, D does it, I’m not familiar enough with Rust to give you the specifics there.

21:25 - It’s a dramatic change of the way you as a programmer probably intuitively know how these things work, and as a result I think even if I never get to the big-ticket compile-time execution items, which I hope I will get, but I think, having Practical, even if Practical itself doesn’t become very popular, if people start looking at those and saying: “know what? That’s a good idea. Let’s introduce it into another programming language” and that programming language becomes popular then I’ve done my share. Because I think and, I will do a more complete video about it later on, but I think the way C++, and essentially C, handles integer types is fundamentally broken. And the way C does integer is also the way that C++ does integer manipulation, and also the way that D does integer manipulation, and a bunch of other languages that sort of said: “Well, you know, we’ll just do it the same as them because we want code to transfer well, because for various reasons.” And those are important, and they’re completely broken.

So, with that out 22:57 - of the way, what do I plan for Practical? I think the most important aspect you should take away is that Practical aims to be a practical programming language. Do not aim to have completely solved all of the world problems in one particular domain. I want a language that will… it should offer you good defaults, and those defaults are probably not going to be significantly different than what you know if you’re programming in Rust. Necause, like I said, I think Rust did a lot of very good decisions at its core. But it won’t force you to follow that path.

The idea, I think, is to have 23:44 - a sort of beaten path that should be easy, and straying from the beaten path should not be impossible, and, in fact, should be should grow linearly the further away from the beaten path you stray. So if you want to do everything on the beaten path, but you know just a small thing different, then that should be fairly easy. And if you say “no I’m going different route completely” then, yes, I’m gonna make you work hard for that. At the end of the day, the main focus I have on Practical is avoiding the everyday frustrations. The idea is to prevent you from creating small annoying bugs accidentally. That’s what I’m planning, in a sense.

I’m going to release 24:48 - future videos detailing the progress. Basically, at this point in time, when I’m filming this I’m not sure when you’ll be watching this, but when I’m recording this, I have a compiler. It does some very basic programming stuff, so you can define functions, you keep that if you define a few operators and a variables to to get them, you can pass variables around, some basic casts, the type system I mentioned earlier is actually implemented. I was wondering, and I think what I’m going to do is that the next few videos we’ll sort of backtrack over progress I’ve already done, and when I catch up to where the master of Practical development is, I’m guessing the videos will start tracking my current progress. I wish I could tell you that we’re gonna have regular videos but, that’s probably not going to happen because Practical isn’t my day job, it’s a hobby project I’m doing, progress will happen and these videos will happen when I get the time to do them.

What I will try to do is that 26:16 - every video will have some substance to it. It won’t have updates just because I promised you a video on Thursday and it’s Thursday so I’ll put up a video. I will try to have every video have some meaningful subject to it. That’s it! Thank you for watching. I would love you to subscribe, to like, to comment if you have any. I’ll try not to beg for comments each video because I know it’s the currency on YouTube, but I find it really annoying so just remember that I would really, I am not begging for comment or likes and comments every time, but I really really really would love to get them.

Thank you 27:04 - for watching! I will see you in the next video. .