Why I created Kit

The Kit programming language is a cross-platform sytems programming language that compiles to C. It was designed primarily for the requirements of game development, but isn’t limited to that domain. It emphasizes developer productivity and provides zero-cost, high-level abstractions that simultaneously optimize for performance and utility.

Background

I spend a lot of my free time contributing to open source projects. Until last year, much of that time was spent contributing to the Haxe language ecosystem, including the language and compiler itself and projects like HaxePunk, a cross-platform game engine.

Haxe is a well designed language with some fantastic innovations, but I gradually became disenchanted with it for a few reasons. For one, I found myself struggling to get good performance and fighting with the garbage collector to avoid dropping frames. Also, the community is small and projects are unstable; dependencies of HaxePunk added breaking changes much faster than I could build anything with it, so I was constantly trying to catch up.

Most importantly, I had ideas of what I wanted Haxe to be, but after engaging in the feature proposal process a few times, I found that they didn’t fit in with the creator’s vision. Sometimes this was subjective. Other times it was due to the large number of targets and use cases Haxe must support. The variety of targets makes Haxe something of a Swiss army knife; you can use it to do several things pretty well, but it’s initially unclear what it’s really great at or intended for.

This is ultimately what caused me to part ways with Haxe, and imagine the language I wished it was – and increasingly, what I would create myself if I wasn’t constrained by the historical design decisions Haxe had made.

In June 2018, our son Miles was born and I took 12 weeks off from work to spend with my family. During this time I found myself keeping odd hours, and often had blocks of a few hours during the day (or night) where the baby was sleeping, I couldn’t sleep, and I was too tired to do much other than code. Strangely enough, these blocks of time were perfect for development, and I realized that if I applied myself, I could build something more ambitious in those 12 weeks than my regular after-work hours allowed. Suddenly, "Kit" (which was originally called Otter, then Pika – I was clearly angling for a mammal mascot of some kind) seemed a lot more feasible and development began in earnest.

Motivation

How is Kit good for game development, specifically?

  • It’s a low-level language with no garbage collector by default.
  • Kit makes it dead simple to leverage libraries like SDL and OpenGL.
  • Code written in Kit is portable to game consoles, etc. by leveraging those platforms’ existing C compilers.
  • Specific idioms (implicits, traits) make custom memory management strategies that are common in game development more ergonomic in Kit.

Kit borrows concepts from several other languages, primarily Rust and Haxe (as well as similar languages like Scala and Kotlin.) A natural question is, why not use one of these other languages instead? Ultimately, none of them gives the precise combination of features and capabilities that Kit does, so it inhabits a unique part of the language design space. The goal is to provide a language that rivals higher level languages in ergonomics and abstractions, but which provides more control over performance and, unlike Haxe, focuses on one great compile target over many good ones.

Ecosystem

While writing C can leave something to be desired compared to modern alternatives, compiling to C and leveraging its ecosystem is fantastic. There’s a C compiler for just about every platform and architecture that you could ever want to target today, and interoperability with C gives you access to a vast set of mature libraries, from SDL to libpng to libcurl.

Working in a language with a small community can be somewhat painful by comparison; often the selection of libraries is limited, and interoperability requires bindings which must be manually created and maintained. I wasn’t about to solve the problem just by introducing yet another variation, so I knew Kit had to have fantastic interoperability with C; this led to several design decisions:

  • Kit compiles to standard C, and leaves the problem of portability to the C compiler (which is good at it.)
  • Kit’s type system is built directly on top of C’s – so there isn’t a dichotomy of "C types" vs. "my language’s types."
  • C libraries can be used directly in Kit code, without bindings. Include a C header in your file and any names declared in that header are now accessible from Kit code, completely type safe.

Abstract Types

Something I’ve borrowed from Haxe and extended is the concept of "abstracts." An "abstract type" is a type which has semantic meaning at compile time, but doesn’t have a unique runtime type. Its identity is erased during compilation. This makes it possible to decouple the semantic intent of a value from its underlying storage, and makes code self-documenting and refactoring easier.

A great example is Color – in game development, a 32 bit unsigned integer can be used to represent an ARGB color, but not every unsigned int is intended to be used as a color. By defining a Color abstract, Kit will see these as separate types:

abstract Color: Uint32 {
    public function getRed(): Uint8 {
        return (this & 0xff0000) >> 16;
    }
    // ...
}

function main() {
    var c = 0x80ffe6 as Color;
    var r = c.getRed();
}

Developers can give Color "methods" as if it were an object in an object-oriented language. But when Kit generates C from this abstract, its identity as a Color is erased; method calls become regular function calls, and Color values become unsigned ints. This provides modern ergonomics without runtime overhead.

Term Rewriting

I made my first foray into language design in college with a toy language called Scotch, an interpreted functional language featuring term rewriting. Term rewriting basically allows developers to write expression-level substitution rules, replacing pieces of the program’s AST with different expressions based on the rules that are in scope.

With Kit I’m revisiting the concept of term rewriting, but now in a procedural systems language. I find it’s an elegant and powerful way to subsume multiple important language features that would otherwise have to be bolted on, including:

// operator overloading
({$a: MyStruct} + {$b: MyStruct}) => $a.value + $b.value;
// custom type conversions
({$a: MyType} as CString) => $a.toString();
// inlined properties and functions
($this.index) => 5;
// overloaded functions
(myFunc({$i: Int})) => myFuncInt($i);
(myFunc(${f: Float})) => myFuncFloat($f);

Even for loops are implemented entirely by rewrite rules in the standard library; with the exception for loops over numeric ranges, the compiler doesn’t have any special knowledge of what it should to do with them. In Kit, term rewriting rules can also be associated with a type itself and come into scope whenever an expression contains a value of that type. This makes creating types with custom semantics very ergonomic.

Metaprogramming

Boilerplate is a side project killer, but writing a macro to automate boilerplate feels great.

Kit has several mechanisms for evaluating code at compile time, which I’ll dive into in more depth in the future. One mechanism is procedural macros, which can execute arbitrary code, to generate new code that is then compiled into the program:

// this macro generates a function with a given name
macro generateFunction(name: CString) {
    printf("function %s() { puts('hi'); }
}

// call the macro, with a value known at compile time
generateFunction("helloWorld");

function main() {
    // when `main` is typed, the function will exist
    helloWorld();
}

Conclusion

Kit stands on the shoulders of some fantastic languages and combines some of my favorite features from each. While the compiler is still pre-alpha, the language is usable now and already feels great to work with. The challenges of working with a less mature language are mitigated by the mature C ecosystem Kit has adopted.

I woke up one morning to find that Kit had been shared on HN and it’s been great to see what the community has done so far – users are integrating Kit with their favorite game engines and running it on PC, 3DS, Arduino or GameCube.

There’s a lot more that I want to add to Kit, I love working with it so far, and I’m excited about what I think it can become.

One thought on “Why I created Kit

Leave a Reply to haxerefugee Cancel reply