Mike's corner of the web.

Nope: a statically-typed subset of Python that compiles to JS and C#

Sunday 22 March 2015 15:55

Nope is a statically-typed subset of Python that uses comments as type annotations. For instance, here's the definition of a Fibonacci function:

#:: int -> int
def fib(n):
    seq = [0, 1]
    for i in range(2, n + 1):
        seq.append(seq[i - 1] + seq[i - 2])
    
    return seq[n]

print(fib(10))

And here's a generic identity function:

#:: T => T -> T
def identity(value):
    return value

Since the types are just comments, any Nope program is directly executable as a Python program without any extra dependencies.

Having written your program with type annotations, you can now compile it to some horrible-looking JavaScript or C# and run it:

$ python3 fib.py
55
$ nope compile fib.py --backend=dotnet --output-dir=/tmp/fib
$ /tmp/fib/fib.exe
55
$ nope compile fib.py --backend=node --output-dir=/tmp/fib
$ node /tmp/fib/fib.js
55

Why?

A little while ago, I wrote mammoth.js, a library for converting Word documents to HTML. Since some people found it useful, I ported it to Python. Both implementations are extremely similar, and don't heavily exploit the dynamic features of either language. Therefore, a third port, even to a statically-typed language such as C# or Java, is likely to end up looking extremely similar.

This led to the question: if I annotated the Python implementation with appropriate types, could I use it to generate vaguely sensible C# or Java code? Nope is an experiment to find out the answer.

Should I use it?

This project is primarily an experiment and a bit of fun. Feel free to have a play around, but I'd strongly suggest not using it for anything remotely practical. Not that you'd be able to anyway: many essential features are still missing, while existing features are likely to change. The type-checking is sufficiently advanced to allow partial type-checking of the Python port of Mammoth.

I discuss a few alternatives a little later on, and explain how Nope differs. In particular, there are a number of existing type checkers for Python, the most prominent being mypy.

Examples

Simple functions

Functions must have a preceding comment as a type annotation.

#:: int -> int
def triple(value):
    return value * 3

Functions with optional and named arguments

Optional arguments are indicated with a leading question mark, and must have a default value of None. For instance:

#:: name: str, ?salutation: str -> str
def greeting(name, salutation=None):
    if salutation is None:
        salutation = "Hello"
    
    print(greeting + " " + name)

print(greeting("Alice"))
print(greeting("Bob", salutation="Hi"))

Note that the type of salutation changes as it is reassigned in a branch of the if-else statement. It is initially of type str | none, which is the union of the formal argument type and none (since it's optional). After the if-else statement, it is of the narrower type str, which allows it to be safely used in the string concatenation.

Variables

Most of the time, Nope can infer a suitable type for variables. However, there are occasions where an explicit type is required, such as when inferring the type of empty lists. In these cases, an explicit type can be specified in a similar manner to functions:

#:: list[int]
x = []

Classes

When annotating the self argument of a method, you can use the explicit name of the class:

class Greeter(object):
    #:: Greeter, str -> str
    def hello(self, name):
        return "Hello " + name

For convenience, Nope also introduces Self into the scope of the class, which can be used instead of referring to the containing class directly:

class Greeter(object):
    #:: Self, str -> str
    def hello(self, name):
        return "Hello " + name

As with local variables, instance variables assigned in __init__ can have type annotations, but will often be fine using the inferred type:

class Greeter(object):
    #:: Self, str -> none
    def __init__(self, salutation):
        self._salutation = salutation

    #:: Self, str -> str
    def hello(self, name):
        return self._salutation + " " + name

Generic classes are also supported, although the exact syntax might change:

#:generic T
class Result(object):
    #:: Self, T, list[str] -> none
    def __init__(self, value, messages):
        self.value = value
        self.messages = messages

Code transformations

To preserve some of the advantages of working in a dynamic langauge, Nope supports code transformations: given the AST of a module, a transformation returns an AST that should be used for type-checking and code generation. So long as the transformation and the original runtime behaviour are consistent, this allows you to use code such as collections.namedtuple:

import collections

User = collections.namedtuple("User", [
    #:: str
    "username",
    #:: str
    "password",
])

The current implementation is a bit of a hack, but the ultimate goal is to let a user specify transformations to apply to their code. Ideally, this would allow Python libraries such as SQLAlchemy to be supported in a type-safe manner.

And the rest

Nope also supports while and for loops, try statements, raise statements, destructuring assignment and plenty more. However, I've left them out of this post since they look the same as they do in Python. The only difference is that Nope will detect when inappropriate types are used, such as when trying to raise a value that isn't an exception.

I've started using Nope on a branch of Mammoth. Only some modules are currently being type-checked by Mammoth, such as html_generation.

If you're feeling brave, Nope has a set of execution tests that check and compile sample programs. It's the not the greatest codebase in the world with many unhanded or improperly handled cases, but feel free to read the tests if you want to dive in and see exactly what Nope supports. At the moment, Nope compiles to Python (which means just copying the files verbatim), JavaScript and C# with varying degrees of feature-completeness. The C# implementation in particular has huge scope for optimisation (since it currently relies heavily on (ab)using dynamic), but should already be fast enough for many uses.

Type system

Nope ends up with a few different kinds of type in its type system. It would be nice to be able combine some of these, but for the time being I've preferred to avoid introducing extra complexity into existing types. At the moment, Nope supports:

• Ordinary classes, made by using class in the usual way. Since inheritance is not yet supported, a type T is a subclass of itself and no other ordinary classes.
• Function types, such as:
  • int, str -> str (Requires two positional arguments of type int and str respectively, and returns a str.)
  • int, x: int, ?y: str -> int (Has two required arguments, the second of which can be passed by the name x. Has an optional third argument called y.)
• Structural types. For instance, we might define a structural type HasLength with the attribute __len__ of type -> int (takes no arguments, returns an integer). Any type with an appropriately typed attribute __len__ would be a subtype of HasLength. Currently not definable by users, but should be.
• Generic types. Nope currently supports generic functions, generic classes and generic structural types. For instance, the generic structural type Iterable has a single formal type parameter, T, and an attribute __iter__ of type -> Iterator[T], where Iterator is also a generic structural type.
• Type unions. For instance, a variable of type int | str could hold an int or a str at runtime.

What about IronPython or Jython?

Both IronPython and Jython aim to be Python implementations, rather than implementing a restricted subset. This allows them to run plenty of Python code with little to no modification.

Nope differs in that it aims to allow the generation of code without any additional runtime dependencies. Since the code is statically typed, it should also allow better integration with the platform, such as auto-complete or Intellisense in IDEs such as Visual Studio, Eclipse and IntelliJ.

What about mypy?

mypy is an excellent project that can type check Python code. If you want a project that you can practically use, then mypy is by far more appropriate. There are other type checkers that also use Python annotations to specify types, such as obiwan.

At this point, Nope differs in two main regards. Firstly, Nope's scope is slightly different in that I'm aiming to compile to other languages.

The second main difference is that Nope aims to have zero runtime dependencies. Since mypy uses Python annotations to add type information, programs written using mypy have mypy as a runtime dependency. This also allows some meta-programming with somewhat consistent type annotations, as shown in the collections.namedtuple from earlier:

import collections

User = collections.namedtuple("User", [
    #:: str
    "username",
    #:: str
    "password",
])

What next?

I'm only working on Nope in my spare time, so progress is a little slow. The aim is to get a C# version of Mammoth working by using Nope in the next few months. Although there might not be feature parity to begin with, I'd be delighted if I got far enough to get the core program working.

If, in a surprising turn of events, this turns out to work, I'd be interested to add effects to the type system. Koka is the most notable example I know of such a system, although I'll happily admit to being some unknowledgeable in this area. Thoughts welcome!

Topics: Software design, Language design