Using Scala Implicits to Implement a Messaging Protocol

By

We’re building some cool new stuff at Primal. Awesome ideas and talented people are a must have in order to build successful and exciting products, but we also need our tools to step up to the challenge. This is why we’ve started adopting Scala and Akka. In this post, I’m going to describe how we’re using Scala’s implicits in order to implement a very important part of our internal messaging fabric, without having to over-burden the business logic.

For some background, see Introducing Primal Assistants: A framework for software agents.

(For the impatient, you can find all of this working code in our Github repository.)

The Goal

Passing messages between Actors in Akka is very simple, and looks like this:

someActor ! SomeMessage("Hello there, Mr. Actor")

Simple, elegant, and already hides part of the pain from the developer. When you send a message from one Actor to another, the sender goes along with it. This becomes more obvious when you see the method definition of !:

def !(msg: Any)(implicit sender: ActorRef)

Akka ensures that the sender of the message gets wrapped up inside an envelope by putting the current Actor into the implicit scope with a member value:

implicit final val self = // this Actor's corresponding ActorRef

What we, at Primal, are looking for is to create our own envelope that carries a great deal more information than just the sender of the message. We want to create an envelope that has the following:

  • The class type of the sender
  • The unique identifier of the sender
  • The unique identifier of the intended recipient
  • The class type of the message being sent
  • The timestamp of the creation time of the message
  • A unique identifier for the current “unit of work” in which this message is participating
  • A sequence number that indicates where this message sits in the timeline of the unit of work
  • A version number for the protocol version of the envelope
  • The message itself

And the important part, is that we want all of this information to be supplied without burdening the code that sends the message. It should look like this:

someActor emit SomeMessage("Hello there, Mr. Actor")

The guy that actually sends the message should have to do very little in order to gain all of the functionality that we want. The other side of that coin is that the guy shouldn’t be able to screw up.

The How (in pictures)

There are actually two separate goals, and they’re in tension:

  1. Simplicity for the user of our library
  2. Richness of data in the envelope

This tension is resolved using implicits, OO inheritance principles, and functional delegation. First, let’s remember what Akka does for us automatically:

When sending a message from one Actor to another, Akka automatically packages up our message and the Actor reference of the guy doing the send into an envelope that Akka can pull apart later. We want to convert the message being sent into an envelope of our own before Akka gets a hold of it. Based on nothing more than a different method (i.e. emit vs. !) we should be able to provide all we need.

The How (in code)

Let’s start with the Envelope class:

 

Clearly there are some support types in there that need some definitions, so let’s define them:

 

Nice… our basic ADT is in place, and it’s now it’s time to see how it gets constructed. In order to make it happen all sexy-like, it’s going to be done implicitly, by converting from a Scala Any (which is the type that Akka uses for messages) into our Envelope.

 

Now, when we mix EnvelopeImplicits into the right right place, we’ll have an implicit conversion from a message to our envelope. But why on earth would Scala every invoke that conversion? The answer lies in another usage of implicits that provides some extra functionality to the ActorRef:

 

Now, were you to mix ActorRefImplicits into the right place you’d have an implicit class, which would provide you the appropriate emit and emitForward calls. The important bit to note is what type these new methods require… Envelope. When you do this…

someActor emit SomeMessage("Hello!")

… the compiler sees a problem. SomeMessage("Hello!") is not of type Envelope. Rather than throw an error and fail the compile, it searches the implicit scope for a conversion method that can translate something in SomeMessage‘s class hierarchy into an instance of Envelope. Assuming that is available, the conversion will happen and you’re good to go!

Now, let’s create a base Actor that handles the Envelopes for us, as well as puts the appropriate stuff into the implicit scope for subsequent message sending.

 

The job of the EnvelopingActor is to handle the complexity for its derivations. Four implicit values are managed:

  • “This” Actor’s component type
  • “This” Actor’s component Id
  • Any current workId that might be known
  • Any current message sequence number that might be known

With these values in the implicit scope, any message emit that the derivation might execute will ensure that enveloping happens correctly.

So, let’s see it in action…

 

And this prints out something like the following…

Envelope(
  ComponentType(MyActor),
  ComponentId(123456),
  ComponentId(/$a),
  MessageType(String),
  WorkId(6cbdf4ac-5843-4c6b-af30-67e7d28b7b78),
  MessageNum(1),
  EnvelopeVersion(1),
  1369255095475,
  Hello
)

There you have it! The main point here is the simplicity of the derivations of the EnvelopingActor. One of the most important aspects of Scala’s implicits is their ability to move complexity out of user code and into library code. I think the definition of MyActor embodies that whole idea:

 

Enjoy Scala’s implicits! We certainly are.

You can find all of this working code in our Github repository.

Getting Started

Developers, check out the data API section of our website for instructions to get started.

Want to use Primal directly on the Web? Take a tour to learn more.

If you’d like to stay in the loop on our progress, follow us on Twitter.

And if you’re interested in working with us directly, we’re hiring!