Getting started with Rust/WebAssembly

I’ve been using the Rust programming language at work quite a lot recently, and really like it.

In this week’s issue of This Week in Rust, I saw that WebAssembly was now supported natively in rustc (previously it was supported via Emscripten).

WebAssembly is a binary executable format designed to give (near-)native performance of code embedded in webpages.

Although I don’t do web development as work, I do sometimes do it for side projects, and it’s always fun to try something new!

Setup was easy (following these instructions over as Hello, Rust!) and I got a basic Javascript to Rust/WebAssembly function call working (following this tutorial).

Calling Javascript from Rust

So we have Javascript calling Rust.  Now let’s try getting Rust calling back into Javascript.

To this, first we have to tweak add.rs (now renamed to call-js.rs) to add an “extern” function, and call it from add_one rather than returning the value – not that the call to javascript_fn needs to be marked unsafe, because calling out of Rust always is.

extern "C" {
  pub fn javascript_fn(num: i32);
}

#[no_mangle]
pub fn add_one(x: i32) {
  unsafe {
    javascript_fn(x + 1);
  }
}

And then provide the Javascript function for Rust to call as an import.  The name of the function must match the name in the Rust code above.  It appears that “env” as the key is hard-coded from Rust – if anyone knows how to change this, please let me know!

var env = {
  javascript_fn: num => { alert(num); },
}

fetch('call-js.wasm')
.then(response => response.arrayBuffer())
.then(bytes => WebAssembly.instantiate(bytes, {env: env}))
.then(results => { results.instance.exports.add_one(41); });

This then gives the same behavior as the simple add example, just with a lot more code – great!

Passing Strings to Javascript

Of course, the next step is to pass a string.  This is a little more difficult, because strings aren’t a primitive type in WebAssembly, so there’s no convention for passing them.

To pass the string from Rust to Javascript, we can pass a pointer and a length, as follows:

extern "C" {
  pub fn javascript_fn(ptr: *const u8, len: u32);
}

#[no_mangle]
pub fn add_one(x: i32) {
  unsafe {
    let msg = format!("Hello world: {}", x + 1);
    javascript_fn(msg.as_ptr(), msg.len() as u32);
  }
}

Now we need to change the Javascript code to parse the pointer and length back into a string.  The “pointer” that’s passed to Javascript is actually just a number – an index into a special “memory” export (of type WebAssembly.Memory).  We can turn the relevant section of this memory into a Uint8Array, and then use the TextDecoder.decode() method to turn that into a string.

var mod;
var env = {
  javascript_fn: (ptr, len) => {
    var buf = new Uint8Array(mod.instance.exports.memory.buffer, ptr, len)
    var msg = new TextDecoder('utf8').decode(buf);
    alert(msg);
  }
}

fetch('pass_string.wasm')
.then(response => response.arrayBuffer())
.then(bytes => WebAssembly.instantiate(bytes, {env: env}))
.then(module => {
  mod = module;
  module.instance.exports.add_one(41);
});

Note that, in order to access the memory export, we also had to save a reference to the loaded module (in the “mod” variable).

If you run this, you now get an alert saying “Hello world: 42”.  The string is being constructed in Rust, and passed through to Javascript.

This is all a bit long-winded, but I haven’t found a better way – any observations on how to do it better would be appreciated!

Next Steps

Next step is to get Javascript to be able to pass strings into Rust… and then interwork between other Javascript and Rust objects.

RouteMaster: Recovering Blockchain Wallets via Physical Location

Last weekend, Yin Yee and I went to the Blockchain Hackathon at Huckletree.  The theme (in case you couldn’t guess) was blockchain, and there were 2 challenges each with 3 tracts:

  1. Global Citizenship
    1. Encrypted Key Re-connection
    2. Bio Identity ‘Access Recognition’
    3. Data Sovereignty
  2. Consuming Digital Objects
    1. Digital Ownership Tracking
    2. Consumption – Access Control
    3. Monitoring Digital States

I was interested in working on the first challenge, as how to define identity is an interesting question.  It also felt to me that, for the second challenge, any Digital Rights Management (DRM) system would have to be centralized, and this would weaken a key value of blockchain: decentralization.  Indeed, most of the projects for the second challenge focused on trying to make consumers more honest by improving transparency rather than by securing media.

My only prior experience with blockchain was at a Fintech hackathon in November 2015, where we built Credit Passport, a service to allow you to securely share your financial history with credit providers to help preserve your credit rating when moving between countries.  We used Colu (which at the time was aiming to be a blockchain-secured data store, much like IBM’s Hyperledger) as a store, and so we didn’t need to get too involved in the details of blockchain technology itself.

…which all meant this hackathon was a great opportunity to learn!

(Yin Yee had much more experience with blockchain, having designed and built a cryptocurrency for her masters’ thesis and worked for a blockchain startup.)

metroline_buses_lt16_28lk13_fjn29_26_lt26_28ltz_1026292c_19_september_2013
Routemaster buses, after which our team was named

On Saturday morning, Peter Jones pitched an idea around checking in at a series of real-world locations to take back control of a wallet for which you’d lost the key – the clever thing here was that we were bootstrapping blockchain security from existing real-world physical security – for example, it would be relatively hard for a hacker to both get into my home and into my office at work.  Hanson Aboagye joined Yin Yee, Peter and me to form the “RouteMaster” team.  We spent quite a while brainstorming and ramping up on Ethereum (a blockchain for smart contracts).  Eventually, at about 15:30, we settled on building

  • an Ethereum smart contract that could
    • store your Ether cryptocurrency and make transactions if requested to by its current “owner”
    • store a list of locations that its owner regularly visited
    • change owner if a new user logged into the same list of locations and the previous owner stopped logging in
  • an Android client that
    • stores Ethereum account details
    • allows the user to make payments
    • allows the user to check in at locations, detected via Bluetooth Low Energy (BLE) beacons
  • a Raspberry Pi 3-based beacon that
    • broadcast its identity, IP address and an ever-changing random token over BLE
    • accepted HTTP requests from the Android client to check in on the user’s behalf (checking the random token to ensure that they were local).

That was the idea.  We ran into a few problems along the way.

  • The Android Bluetooth Low Energy stack seems to be a little unreliable – even using well-rated apps from the Google Play store, some phones would sometimes stop detecting BLE beacons while others could. We tried power-cycling and clearing the cache but to no avail. In the end, this was the thing that stopped us demonstrating the full flow.
  • Ethereum nodes need to download quite a lot of data to synchronize the blockchain. This is understandable, but with everyone at the venue trying to use one blockchain or another and (presumably) using a lot of network bandwidth, the initial synchronization was quite slow.
  • This was our first time developing in Solidity – the language in which Ethereum smart contracts are written – and so we wrote a few bugs (and wrote contracts that used a lot of gas, i.e. Ethereum compute cycles). Thanks to Nick from Ethereum (on the Colour Card team) for answering our beginner questions!
  • We found a documentation bug that it took us a while to debug, which stopped us from unlocking multiple accounts for RPC API access. Once we raised it, though, it was fixed very quickly – thanks to bas-vk for the fast turn-around!

The presentations were 15 minutes long, which was much longer than I’m used to – 2-3 minutes is more common, and TechCrunch Disrupt London Hackathon in December last year was just 1 minute per team! However, many of the presentations included some pretty detailed explanations of the protocols and algorithms used, so there was plenty to cover.

We were the penultimate team to present.  Peter talked about the business case and then we dived into the demo.  Unfortunately, our demo script was pretty complicated, and we made a mistake so couldn’t show everything – at least our code didn’t let us down this time!

We were very surprised to win second prize given how strong the other teams were! Congratulations to all the other winners, thanks to the sponsors and organizers for making it happen and all the other teams for their energy, enthusiasm and knowledge – it was a great experience and I learned far more at this hackathon than many I’ve been to!

The source for both the Android client and the Raspberry Pi-based server are up at https://github.com/matt-williams/RouteMaster.

Entangled Dice with Resin.io

On the weekend of 11-12 March, Yin Yee and I attended the resin.io Spring Hackathon.

Unlike many hackathons, this one was purely technical – no need for hastily-constructed business cases for why your hack is going to become the next “unicorn” – which was very refreshing!

resin.io is a platform for deploying and managing software on internet of things devices, using Docker to making it easy to build, roll out and update firmware.

So what was the hack?

480px-rubik27s_cube-svgOne of my friends at work is a Rubix cube enthusiast.  Everyone is familiar with 3×3
cubes, but they also come in 4×4 and 5×5 variants (and probably more).  There are also extensions from regular 3D cubes into 4D “hypercubes”.  Obviously, we perceive the world in 3D, so we can’t perceive a Rubix hypercube, but a common approach is by analogy from 3D to 2D.  Imagine separating a 3D cube into its 6 2D (square) faces, and laying them out flat (in 2D).  You could separate a 4D hypercube into 8 3D (cube) “faces” and lay them out in 3D.

In other words, you could visualize a Rubix hypercube as 8 regular (3D) Rubix cubes.  Great!

…except the key property of Rubix cubes is that when you rotate one face, it affects all the others, so for a Rubix hypercube we’d need to link the 3D Rubix cubes together.  I described this as “entangling” them (with reference to quantum entanglement), but expected to implement this over Wifi.

Unfortunately, we felt this was a bit too complex for the 12 hours or so we had for the hackathon, so instead of “entangling” Rubix cubes, we instead did so with dice.  Whenever one of the dice is rolled, they both show the same new random value.  The idea was that you could use this to play dice games remotely.

(Those familiar with quantum entanglement might observe that our entangled dice should each show the opposite face to the other.  That’s a good point, but would make them significantly less useful!)

sense-hat-web
Sense HAT, with 8×8 LED display

We used a Raspberry Pi 3 as our controller, and a Sense HAT for accelerometer input (to
tell when the die was “rolled”) and 8×8 LED display (to show the number on the die).  Since the whole point of the hack was to have 2 dice with the same value on them, we had 2 sets of this hardware.

Originally, I was hoping to play with Rust as I like the language and there is some support for Rust on resin.io.  Unfortunately, there isn’t a Rust crate for the Sense HAT and we were quite tight for time, so we chose to use Python instead, using its standard sense-hat library for hardware control and flask as a web server.

resin.io provides function to expose (via its own VPN) a web server running on a device out to the public Internet.  This allowed one die to communicate with the other over HTTP to agree a new die value.  (There are obviously security risks here, but we didn’t have time to address these for the hack.)

We encoded the new die value in the URL, e.g. /roll/1 or /roll/5, and tried implementing this using flask’s variable rules.  Unfortunately, this didn’t work on the Raspberry Pi (although it worked fine on my development PC) – it seemed to be something specific to the Python build on the device.  In the end, we used a horrible hack of defining separate routes in flask for each possible die value, i.e. /roll/one, /roll/two, etc..

…and it worked!  We plugged the Raspberry Pis into USB batteries, taped it all together and had 2 portable, entangled dice.

aimmuwr

The code is up at https://github.com/matt-williams/quantum-dice.

All in all, this was a great event – thanks to the resin.io team for organizing and for following up on the hacks!  In particular, thanks to Gergely Imreh for reviewing my code and pointing out the get_all_by_application() API that allows resin.io devices to find each others’ identities – very useful for this project!