Nim for the discerning Rubyist

​So... you've been learning Elixir. Or maybe Rust. You might have even fallen for the charms of Go. All great languages in their own right that will greatly enrich your programming arsenal. In this blog post, however, I'd like to talk about a path less travelled. What if I told you that you could use a language as performant as Elixir but without having to learn a new programming paradigm? Or as powerful as Rust but without the constant memory-management overhead. Or as highly-concurrent as Go but with great generics and meta-programming capabilities? 

And what if I told you that this language was almost as elegant and expressive as Ruby itself? Well, I am telling you. The language is called Nim and it's been around for a few years now, though only recently has it started gaining momentum and mind-share. It's a statically-typed and compiled language that I like to describe as the outcome of a wild night's union between Modula-3, Python and C++, with Lisp on one side giving directions (I'll give you a moment to visualise this).

I'm not going to give you a tutorial on the language, you can do this yourselves by reading the documentation page, or getting the excellent Nim in Action book. Instead, I'll demonstrate what we can do with Nim. But first, let's see how Nim compares to Ruby.

Like Ruby, Nim gives us:

1. Many ways to do the same thing. Implicit, explicit and inline function return values. Functional and procedural iterators. Uniform Function Call. Etc, etc.
2. Lots of syntactic sugar. See also #1.
3. A getting-out-of-your-way-and-let-you-code attitude.
4. Blocks, a.k.a anonymous functions we can pass around.
5. Statements as expressions, i.e. statements are evaluated and can be assigned to.
6. Powerful meta-programming ability.
7. Garbage collection. Though the GC can be adjusted or even turned off!
8. Outward simplicity with a lot of internal complexity.
9. A nice package manager (called nimble).
10. Joy of programming ;)

Unlike Ruby, Nim:

1. Is statically and strongly typed. But some nice type-inference and type-suffixing, make it a non-oppressive type system.
2. Is compiled. Nim compiles to C-family (C, C++, Objective C) source code, which is then linked natively to produce binaries.
3. Is truly portable (no runtime required). See #2.
4. Is highly performant. Again see #2.
5. Has a whitespace-significant (Python-esque) syntax. (I know, I don't like it either, but hey-ho)
6. Is paradigm-agnostic. Imperative in nature, it can support OO, functional and procedural programming styles. 
7. Has a flexible and safe concurrency model.
8. Has a powerful and fully-configurable GC that can be turned off (though in that case we have to manage our own memory)
9. Favours immutable variables and parameters (though mutable ones can be defined)
10. Can be compiled to JavaScript (a sub-set of Nim can).
11. Can seamlessly inter-operate with C-family languages

In short, Nim is a concise and expressive, super-flexible, highly-performant general purpose language. It can used for systems-level programming, with the GC configured for performance or fully turned-off. But thanks to its powerful meta-programming (templates and LISP-like macros) it can also be used for high-level application and even web development.

"If Nim is so great, then why isn't everyone using it?"

Nim is greatly under-rated. IMO, it's a more complete language with a wider range of use-cases than either Rust or Go. However, it doesn't have Google or Mozilla's backing and all the exposure that comes with it.  Also, it's still immature. Version 1.0 has not been reached yet and the language is still subject to changes and backwards incompatibilities. This has partly to do with the lack of big corporate sponsorship and it may change in the future, as the language gains popularity.

"I know Ruby, why should I care about Nim?"
​
From a practical perspective, you can do things with Nim that you can't do with Ruby. Such as processing a lot of data really fast. Or doing parallel computations. Or writing apps for memory/space constrained environments. I find Nim complements Ruby brilliantly and been looking to get both languages working together, each playing to their strengths. Let me show you how,

Imagine we have some processing-intensive task, like getting a high-numbered element from the Fibonacci series. What we'll do is this: we'll build a fibonacci function in Nim, wrap it in a dynamic library, and then call it from Ruby. 

 And that's the Nim code done! We created a recursive function that receives and returns an integer. The result variable is Nim's functions' built-in return value, that's why we don't have to explicitly declare it. We could as easily have used return, or just omitted explicit returns and let the last statement be returned by default (just like in Ruby).

The things in the angular brackets are called pragmas and are compiler directives. Here, we're telling the Nim compiler to export our function in a dynamic library using the C calling convention (cdecl, exportc).

​Now let's build our  library

We're telling Nim to build using the gcc compiler (I run Linux, but you can use whichever compiler you want), do a release build (no debugging info, better performance) and build a dynamic library (app:lib).

​Nim will oblige by building a libutils.so file. We could have changed that name if we'd wanted to. Now let's go build our Ruby client.

We'll be using the FFI library to interact with the dynamic library we created, so go ahead and install the FFI gem. Next, let's write some Ruby code:

So we're using FFI to import the dynamic library we created with Nim into our Ruby app space and mapping the Nim fibonnaci function so we can call it from our Ruby code. In addition, I've added a Ruby implementation of fibonacci and some benchmarks so we can compare the two. Let's run our Ruby code: 

 ○ ruby nim-client.rb 
                     user               system        total               real
ruby_fibo  6.960000   0.000000   6.960000 (  6.948251)
nim_fibo  0.420000   0.000000   0.420000 (  0.423239)

The Nim implementation of fibonacci hugely outperforms the Ruby version. But that's not the impressive part. The impressive part is that we can make it even faster! How? By leveraging Nim's parallel processing ability. Let' say  we needed to calculate the value of three fibonacci items.  The simple approach would be to do it sequentially, i.e. fibonacci(n) + ​fibonacci(n+1) + ​fibonacci(n+2). But with Nim, we can do it in parallel. Watch:

The points of interest here is that we are::

• Importing the threadpool module. Threadpool is a thread scheduler/manager that will create and schedule our threads 
• Creating a tuple (NumList) to hold our result values. Tuples are a  bit like Ruby's Structs or like database records. Very useful for returning multiple values from a function.
• Declaring the return value of ​our parallel_fibo function as a pointer to the tuple. This is because we need to define where in memory our results will live, so that we can access them from our Ruby client.
• ​Allocating memory for our results pointer, of adequate size to hold our tuple. This is not something we would usually do if we ran self-contained Nim code, but we need to since we are sharing memory with another process.
• Using spawn to run our fibonacci function in separate threads. The return value of spawn is of type FlowVar. This is similar to what some languages call futures or promises. It ensures that values are assigned to the variable asynchronously, when the thread is finished. We can access the spawn return values, when they are ready, with the ^ operator, (data.x = ^res). Note how we use multiple threads without having to fret over thread creation, management or synchronisation. Nim's threadpool does all that for us safely. Nim does the heavy lifting so we can concentrate on the application logic.
• Providing a mechanism to clean up after ourselves. We don't want to leave dangling pointers behind so we write a clear_up function to de-allocate the memory we created.

​
We can now build the Nim library

This is just like we did earlier, but this time we're also using the threads:on option, as we're using multi-threading.

Now for our Ruby code:

Just like before, we're attaching the two Nim functions we want to use (parallel_fibo, clear_up) to our FFI. But this time we'll leverage one of FFI's coolest features: ManagedStruct. It allow us to map a foreign data type, such as Nim's tuple, to a native Ruby object. Since our Nim parallel_fibo returns a tuple of three ints, we'll make sure our Ruby Parallels class also has three ints. The only tricky bit here is that we have to define the offsets of each type in our structure. Since the Nim compiler uses 8 bytes for an integer on a 64-bit system, we can can delimit the types in our ManagedStruct by 8 bytes each. 

The other clever thing about a ManagedStruct is that we can tell it how to clean up after itself by defining the release class method. In our case, cleaning up involves calling the Nim clear_up function which de-allocates the pointer memory. This will be invoked when the Ruby GC collects our ManagedStruct object, when we've finished using it.

Now we can call the parallelised fibonacci functions from our Ruby code. 

If you can, take a look at your CPU resources while this is happening.  I have a quad-core processor and this is what it looks like when parallel_fibo ​is called:

Three of my four CPUs just shoot up! Execution time is half of what it would be if I called fibonacci in Nim three times sequentially and about 100 times faster than calling Ruby's fibonacci three times!  In a nutshell, I can now run parallel tasks using my MRI Ruby. No fuss, no switching over to JVM, no writing C-code :)

I could go on about other great features of the language such as customising or tuning off the GC in order to work at system-level. Or writing powerful web-app DSLs using templates and macros. But I'm running out of space here, so I intend to write more posts on leveraging Nim and some of its features in the near future. Watch this space.

TL;DR
Nim is a flexible, fast and concise language that can be used alongside Ruby to provide integrated solutions to a wide-range of problems. In this post I demonstrated how we can delegate tasks that require high-performance or parallelisation from Ruby to Nim, with ease. The Nim code is easy to write and read and gives us a lot of power in an elegant and expressive manner.  

PS: the code used in this post can be found at https://gitlab.com/snippets/1668121-1668123
PPS: you may also be interested in watching a talk I gave on Nim, last year.