There is a young man, probably a Navy officer, who accompanies the President. This young man has a black attaché case which contains the codes that are needed to fire nuclear weapons. I could see the President at a staff meeting considering nuclear war as an abstract question. He might conclude: “On SIOP Plan One, the decision is affirmative, Communicate the Alpha line XYZ.” Such jargon holds what is involved at a distance.

My suggestion was quite simple: Put that needed code number in a little capsule, and then implant that capsule right next to the heart of a volunteer. The volunteer would carry with him a big, heavy butcher knife as he accompanied the President. If ever the President wanted to fire nuclear weapons, the only way he could do so would be for him first, with his own hands, to kill one human being. The President says, “George, I’m sorry but tens of millions must die.” He has to look at someone and realize what death is—what an innocent death is. Blood on the White House carpet. It’s reality brought home.

When I suggested this to friends in the Pentagon they said, “My God, that’s terrible. Having to kill someone would distort the President’s judgment. He might never push the button.“

- Roger Fisher, Bulletin of the Atomic Scientists, March 1981 issue

Single table inheritance (STI) in Rails allows you to store Ruby subclasses in the same database table.

Let’s get started with a brand-new Rails project to learn what that means. I’ll be using Rails 4.2.3 and SQLite as the database.

$ rails new sti-demo

First, an empty User model:

In app/models/user.rb:

class User < ActiveRecord::Base
end

Generate a migration for it. To implement STI, we add a column called type of type string to the class. Let’s also have a name column:

$ rails g migration create_users type:string name:string

class CreateUsers < ActiveRecord::Migration
  def change
    create_table :users do |t|
      t.string :type
      t.string :name
    end
  end
end

Migrate the SQLite database:

$ rake db:migrate

== 20150627182720 CreateUsers: migrating ======================================
-- create_table(:users)
   -> 0.0012s
== 20150627182720 CreateUsers: migrated (0.0012s) =============================

Fire up the Rails console:

$ rails c

Let’s create a new user:

>> User.create({name: "Mr. Bean"})

(0.1ms)  begin transaction
SQL (0.6ms)  INSERT INTO "users" ("name") VALUES (?)  [["name", "Mr. Bean"]]
(2.6ms)  commit transaction
=> #<User id: 1, type: nil, name: "Mr. Bean">

No problems so far.

Now, let’s say we want to differentiate users between regular users and power users. At this juncture, let’s pretend we don’t know about STI. Since we’ve already created the type column, let’s use that, so we can find different types of users by writing something like this:

>> User.where(type: "PowerUser")

Go ahead and create such a user:

>> User.create({name: "George", type: "PowerUser"})

ActiveRecord::SubclassNotFound: Invalid single-table inheritance type: PowerUser is not a subclass of User
    from /Users/siawyoung/.rvm/gems/ruby-2.2.0/gems/activerecord-4.2.1/lib/active_record/inheritance.rb:215:in `subclass_from_attributes'
    from /Users/siawyoung/.rvm/gems/ruby-2.2.0/gems/activerecord-4.2.1/lib/active_record/inheritance.rb:55:in `new'
    from /Users/siawyoung/.rvm/gems/ruby-2.2.0/gems/activerecord-4.2.1/lib/active_record/persistence.rb:33:in `create'
    from (irb):8

Uh oh. What just happened? Why is Active Record complaining about PowerUser not being a subclass of User?

It turns out that Active Record, upon, seeing a column named type, automatically assumes you’re implementing STI.

Into Rails

Down The Stack

Let’s take a dive into the Rails source code to find out how Active Record automatically assumes this behaviour, by finding out step by step what happens when we attempt to type User.create({name: "George", type: "PowerUser"}) to create a new power user.

The source code below is as of commit 943beb1ea23866d46922a5c850f79a0fb9531f9b on the 4-2-stable branch.

Sean Griffin explains briefly in his comments in Active Record’s inheritance module.

In activerecord/lib/active_record/inheritance.rb:

# Active Record allows inheritance by storing the name of the class in a column that by
# default is named "type" (can be changed by overwriting <tt>Base.inheritance_column</tt>).
# This means that an inheritance looking like this:
#
#   class Company < ActiveRecord::Base; end
#   class Firm < Company; end
#   class Client < Company; end
#   class PriorityClient < Client; end
#
# When you do <tt>Firm.create(name: "37signals")</tt>, this record will be saved in
# the companies table with type = "Firm". You can then fetch this row again using
# <tt>Company.where(name: '37signals').first</tt> and it will return a Firm object.

... (some comments about dirty checking) ...

# If you don't have a type column defined in your table, single-table inheritance won't
# be triggered. In that case, it'll work just like normal subclasses with no special magic
# for differentiating between them or reloading the right type with find.

create method in activerecord/lib/active_record/persistence.rb:

persistance.rb       create

def create(attributes = nil, &block)
  # attributes == {name: "George", type: "PowerUser"}
  if attributes.is_a?(Array)
    attributes.collect { |attr| create(attr, &block) }
  else
    object = new(attributes, &block)
    object.save
    object
  end
end

We start our investigation from Active Record’s persistance module, which contains the method definitions of some of the most-commonly used Active Record methods, such create, new, save, update and destroy (protip: use Command-R in Sublime Text to search by method definition).

We first check if the supplied attributes argument is an array. If so, we recursively call create for each member in the array. This means you can do something like:

User.create([[{name: "a"}, {name: "b"}], {name: "c"}]) # notice the nested array

and Active Record will create three users, and even return them to you in the same array structure you specified:

(0.1ms)  begin transaction
SQL (6.1ms)  INSERT INTO "users" ("name") VALUES (?)  [["name", "a"]]
(0.8ms)  commit transaction
(0.1ms)  begin transaction
SQL (0.3ms)  INSERT INTO "users" ("name") VALUES (?)  [["name", "b"]]
(1.0ms)  commit transaction
(0.1ms)  begin transaction
SQL (0.3ms)  INSERT INTO "users" ("name") VALUES (?)  [["name", "c"]]
(0.7ms)  commit transaction
=> [[#<User id: 7, type: nil, name: "a">, #<User id: 8, type: nil, name: "b">], #<User id: 9, type: nil, name: "c">]

If we supply a hash, then the new method is called:

new method in activerecord/lib/active_record/inheritance.rb:

persistance.rb       create
inheritance.rb       new

Notice the *args splat operator, which converts all but the last &block argument into an Array:

def new(*args, &block)
  if abstract_class? || self == Base
    raise NotImplementedError, "#{self} is an abstract class and cannot be instantiated."
  end

  # args == [{:name=>"George", :type=>"PowerUser"}]
  attrs = args.first
  if subclass_from_attributes?(attrs)
    subclass = subclass_from_attributes(attrs)
  end

  if subclass
    subclass.new(*args, &block)
  else
    super
  end
end

We take the first member of the args array and run the subclass_from_attributes? method on it, which is located in the same file, some ways down:

subclass_from_attributes? method in activerecord/lib/active_record/inheritance.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes?

def subclass_from_attributes?(attrs)
  # attrs              == {:name=>"George", :type=>"PowerUser"}
  # attribute_names    == ["id", "type", "name"]
  # inheritance_column == "type"
  attribute_names.include?(inheritance_column) && attrs.is_a?(Hash)
end

This method checks through all of the attributes in the model to see if any of their names match the specified inheritance column, which in this case is "type". Therefore, subclass_from_attributes? returns true.

Let’s see where attribute_names and inheritance_column come from.

attribute_names method in activerecord/lib/active_record/attribute_methods.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes?
attribute_methods.rb attribute_names

# Returns an array of column names as strings if it's not an abstract class and
# table exists. Otherwise it returns an empty array.
#
#   class Person < ActiveRecord::Base
#   end
#
#   Person.attribute_names
#   # => ["id", "created_at", "updated_at", "name", "age"]
def attribute_names
  @attribute_names ||= if !abstract_class? && table_exists?
      column_names
    else
      []
    end
end

where column_names is:

column_names method in activerecord/lib/active_record/model_schema.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes?
attribute_methods.rb attribute_names
model_schema.rb      column_names

def column_names
  @column_names ||= columns.map { |column| column.name }
end

I’m going to stop here because columns steps into whole new territory: namely, into the scary ActiveRecord::ConnectionAdapters module. What this means is that, its at this point where Rails actually connects to the database itself to get (and cache) information about its tables.

In fact, you can call it in the console and see for yourself:

>> User.columns
[#<ActiveRecord::ConnectionAdapters::Column:0x007fa29fa2b0e0
  @cast_type=
   #<ActiveRecord::Type::Integer:0x007fa298e34bc0
  ... (redacted) ...
  @name="id",
  @null=false,
  @sql_type="INTEGER">,
 #<ActiveRecord::ConnectionAdapters::Column:0x007fa29fa2afc8
  @cast_type=
   #<ActiveRecord::Type::String:0x007fa29fa2b680
  ... (redacted) ...
  @name="type",
  @null=true,
  @sql_type="varchar">,
 #<ActiveRecord::ConnectionAdapters::Column:0x007fa29fa2aeb0
  @cast_type=
   #<ActiveRecord::Type::String:0x007fa29fa2b680
   ... (redacted) ...
  @name="name",
  @null=true,
  @sql_type="varchar">]

And finally, the crux of the matter, which can be found in activerecord/lib/active_record/model_schema.rb:

inheritance_column method in activerecord/lib/active_record/model_schema.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes?
model_schema.rb      inheritance_column

# Defines the name of the table column which will store the class name on single-table
# inheritance situations.
#
# The default inheritance column name is +type+, which means it's a
# reserved word inside Active Record. To be able to use single-table
# inheritance with another column name, or to use the column +type+ in
# your own model for something else, you can set +inheritance_column+:
#
#     self.inheritance_column = 'zoink'
def inheritance_column
  (@inheritance_column ||= nil) || superclass.inheritance_column
end

# Sets the value of inheritance_column
def inheritance_column=(value)
  @inheritance_column = value.to_s
  @explicit_inheritance_column = true
end

As you can see, I’ve included both the getter and setter for inheritance_column. So this is the setter that allows you to do self.inheritance_column = :some_other_column_name to change the name of the inheritance column that Rails looks for, in case you don’t want to use "type".

Where in the code is "type" explicitly defined as a default though?

module ModelSchema
  extend ActiveSupport::Concern
  included do

    ... (redacted) ...

    self.inheritance_column = 'type'

  end

A-ha! It’s included as a concern in the included block.

Up The Stack

Let’s pick up where we left off, at new:

persistance.rb       create
inheritance.rb       new

if subclass_from_attributes?(attrs)
  subclass = subclass_from_attributes(attrs)
end

Since one of the attributes does include the specified inheritance column, subclass_from_attributes? returns true, and so Rails runs the subclass_from_attributes method, found in the same file:

subclass_from_attributes method in activerecord/lib/active_record/inheritance.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes

def subclass_from_attributes(attrs)
  # attrs == {:name=>"George", :type=>"PowerUser"}
  subclass_name = attrs.with_indifferent_access[inheritance_column]
  # subclass_name == "PowerUser"     

  if subclass_name.present?
    subclass = find_sti_class(subclass_name)

    if subclass.name != self.name
      unless descendants.include?(subclass)
        raise ActiveRecord::SubclassNotFound.new("Invalid single-table inheritance type: #{subclass.name} is not a subclass of #{name}")
      end

      subclass
    end
  end
end

This method takes the value in the attribute identified as the inheritance column, then attempts to find a subclass by that name, using the find_sti_class method.

with_indifferent_access allows you to access a hash with either a symbol or string representation of the key:

>> attrs
=> {:name=>"George", :type=>"PowerUser"}
>> attrs[:type]
=> "PowerUser"
>> attrs["type"]
=> nil
>> attrs.with_indifferent_access[:type]
=> "PowerUser"
>> attrs.with_indifferent_access["type"]
=> "PowerUser"

find_sti_class method in activerecord/lib/active_record/inheritance.rb:

persistance.rb       create
inheritance.rb       new
inheritance.rb       subclass_from_attributes
inheritance.rb       find_sti_class

def find_sti_class(type_name)
  # type_name == "PowerUser"
  if store_full_sti_class # defaults to true
    ActiveSupport::Dependencies.constantize(type_name)
  else
    compute_type(type_name)
  end
rescue NameError
  raise SubclassNotFound,
    "The single-table inheritance mechanism failed to locate the subclass: '#{type_name}'. " +
    "This error is raised because the column '#{inheritance_column}' is reserved for storing the class in case of inheritance. " +
    "Please rename this column if you didn't intend it to be used for storing the inheritance class " +
    "or overwrite #{name}.inheritance_column to use another column for that information."
end

store_full_sti_class defaults to true, unless it is explicitly set to false in the model.

Then, we use the constantize method provided courtesy of the Dependencies module in Active Support to find out if such a class has indeed been loaded by Rails during initialization:

>> ActiveSupport::Dependencies.constantize("User")
=> User(id: integer, type: string, name: string)
>> ActiveSupport::Dependencies.constantize("PowerUser")
*** NameError Exception: wrong constant name PowerUser
=> nil

And of course, constantize returns an exception because there’s no PowerUser or PowerUser class defined, let alone loaded. This NameError exception is rescued by find_sti_class, which raises the exception which we witnessed near the start of this post.

Patching Things Up

Let’s include a class definition for PowerUser:

class User < ActiveRecord::Base
end

class PowerUser < User
end

This time:

>> ActiveSupport::Dependencies.constantize("User")
=> User(id: integer, type: string, name: string)
>> ActiveSupport::Dependencies.constantize("PowerUser")
=> PowerUser(id: integer, type: string, name: string)

and then we go into the next if clause in subclass_from_attributes:

# subclass.name == "PowerUser"
# self.name     == "User"
if subclass.name != self.name
  unless descendants.include?(subclass)
    raise ActiveRecord::SubclassNotFound.new("Invalid single-table inheritance type: #{subclass.name} is not a subclass of #{name}")
  end

  subclass
end

where descendants is a method in Active Support’s DescendantsTracker module which is used to keep track of the Rails object hierarchy and descendants in memory. The module exposes two methods: descendants, and direct_descendants, both of which should be self-explanatory.

>> User.descendants
=> [PowerUser(id: integer, type: string, name: string)]
>> User.direct_descendants
=> [PowerUser(id: integer, type: string, name: string)]

Now, since subclass is now defined, new is once again called with the subclass, and the cycle repeats.

if subclass
  subclass.new(*args, &block)
else

And we’re finally done!

Aurora’s Github repo

TLDR: I learned Erlang and wrote a JSON API server called Aurora for a school project.

Introduction

I recently concluded a school project involving writing an Android app with “some elements of concurrency”. My group wrote a chat messaging app - a stripped-down clone of Whatsapp, but with some additional features such as message tagging, conversation notes (think Google Docs, but on a per-conversation basis), and event polling.

I volunteered to write the API server that would serve the Android front-end application. I chose Erlang because I heard it was cool and I wanted to learn functional programming.

It seems any blog post regarding projects written in Erlang should include performance benchmarks and an eventual (and completely obvious) conclusion as to why Erlang provides superior performance under heavily concurrent loads. Instead, I will write about my journey picking up Erlang and some other things I learned from writing Aurora.

Learning Erlang, and Why People Think That Learning Functional Programming is Difficult

Because it is, especially if you’re used to thinking imperatively. The soundbite I use most when talking to people is: “Imagine programming without loops”. Of course, its not as bad it sounds, and when you get the hang of it, for loops will seem completely stone-age. map, filter and fold will be your new best friends.

The difficulty is compounded by what I call the “I Think I Know What’s Going On Wait What” effect. Because functional programs (or at least, well-written ones), can be more terse and/or expressive than their imperative counterparts, it sometimes provides a false sense of understanding. When it comes time to write something from scratch, the programmer is completely paralyzed.

I was stuck in many ruts as I made my way through Fred Hebert’s excellent Erlang book, Learn You Some Erlang. The chapters on socket programming in Erlang (chapter 23), ETS (chapter 25), and Mnesia (chapter 29) were particularly illuminating.

The Importance of Ripping Other People’s Code Off

It wasn’t until I found some fairly basic code for a TCP server off a Github repo did I slowly begin to understand how to write my own. And even then, I experimented a lot by copying and pasting and changing each line, line by line, to get a sense of how everything came together.

Beyond the initial scaffolding stage, I continued to find code samples indispensable - often finding alternative ways to achieve the same functionality, corroborating and finding the most elegant/(insert some arbitarary measure of “best”) way. I think the relative paucity of Erlang code on the interwebs psychologically reinforces the preciousness of code samples.

Personal Notes on Erlang

Maps

I was fortunate enough to embark on this project just as Erlang 17 was released. It turns out that maps only found their way into the standard library from this release. Noting a conspicuous lack of knowledge about Erlang’s development history, I was nonetheless appalled to find that a key-value data structure took so long to be introduced into the standard library. I used maps heavily throughout my code.

Joe Armstrong, creator and still-current maintainer of Erlang, talks about maps in Erlang 17 here.

Atoms

I cannot overstate how much I enjoy first-class atom support in Erlang. For the uninitiated, they are conceptually similar to symbols in Ruby in that they are both immutable string-like¹ objects, and that multiple references to atoms and symbols actually refer to the same object in memory. This greatly improves performance, and also reduces programming errors to some degree. Atoms in Erlang are maintained in a centralized atom table, and their text representations are stored once for each unique representation. Notably, this atom table is not garbage collected, which means that dynamic creation of atoms is strongly discouraged. On the other hand, since symbols in Ruby are objects, I’m guessing they live on the heap along with other objects. I have no idea if symbols receive preferential treatment in, say, MRI Ruby’s incremental garbage collection system.

Some Other Details Regarding Aurora

As mentioned earlier, Aurora is an Erlang API server application that services a frontend Android chat messaging application. It uses JSON as the data protocol, transported over raw TCP. I don’t know why we didn’t go with HTTP, but it turned out fine, for the most part. I had to reimplement näively some features of HTTP I needed, such as status codes.

Written about a period of 2 months, Aurora’s source code is about 2100 lines long. About 1400 lines of that is contained in one monolithic file called controller.erl, which contains the controller/business logic, and also for legacy reasons, the Mnesia database’s REST-ful APIs and some convenience methods wrapped around this core set of APIs.

Code Smell, And What I Think I’d Have Done Differently But Am Actually Too Lazy To Revisit

Undoubtably, there is significant code smell in some portions of the code. Of particular badness are bloated functions. There are some functions which contain way too many nested syntactic constructs - case...ofs, ifs. For example, there’re infinitely many better ways to write the below function, which handles the asynchronous cast from the listener socket:

% responds to ROOM_INVITATION message
% invite a user into an existing chatroom
% only admins can invite
handle_cast({room_invitation, ParsedJson, FromSocket}, State) ->
    
    FromPhoneNumber = maps:get(from_phone_number, ParsedJson),
    ToPhoneNumber   = maps:get(to_phone_number, ParsedJson),
    ChatRoomID      = maps:get(chatroom_id, ParsedJson),

    User = find_user(ToPhoneNumber),
    Room = find_chatroom(ChatRoomID),

    if
        User == no_such_user ->
            messaging:send_status_queue(FromSocket, FromPhoneNumber, 5, <<"ROOM_INVITATION">>, <<"No such user">>);
        Room == no_such_room ->
            messaging:send_status_queue(FromSocket, FromPhoneNumber, 5, <<"ROOM_INVITATION">>, <<"No such room">>);
        true ->

            % if the user is already in the room, we cannot invite him, d'oh
            case check_if_user_in_room(Room, ToPhoneNumber) of

                false ->

                    % and of course, we can't invite users if the chatroom is a single chatroom
                    case maps:get(group, Room) of

                        true ->

                            % only admins can invite users
                            case check_if_user_admin(FromPhoneNumber, ChatRoomID) of

                                user_is_admin ->
                                    ChatRoomName = maps:get(chatroom_name, Room),
                                    Expiry       = maps:get(expiry, Room),
                                    UpdatedUsers = add_user_to_room(Room, ToPhoneNumber),
                                    send_chatroom_invitation(ChatRoomID, ChatRoomName, UpdatedUsers, maps:get(active_socket, User), maps:get(phone_number, User), Expiry, group),
                                    messaging:send_status_queue(FromSocket, FromPhoneNumber, 1, <<"ROOM_INVITATION">>, <<"User invited successfully">>);
                                user_not_admin ->
                                    messaging:send_status_queue(FromSocket, FromPhoneNumber, 8, <<"ROOM_INVITATION">>, <<"User is not admin of the room">>)
                            end;

                        false ->

                            messaging:send_status_queue(FromSocket, FromPhoneNumber, 5, <<"ROOM_INVITATION">>, <<"Cannot invite people into single chatroom">>)

                        end;

                true ->
                    messaging:send_status_queue(FromSocket, FromPhoneNumber, 5, <<"ROOM_INVITATION">>, <<"User is already in the room">>)
            end

    end,
    {noreply, State};

However, making your functions too piecewise can adversely affect readability. As with most things, there is a balance to be achieved here.

Build Tools

The standard for building Erlang projects is rebar, but I opted to keep it simple with a bare bones Emakefile because I was in way over my head at that time and didn’t have the necessary bandwidth to learn any other toolings.

Sublime Linter’s contrib-erlc plugin was a massive boon - its linting messages were informative and surprisingly accurate.

Mnesia

I used Mnesia, which was nice because its query language, QLC, is written in Erlang as well. Like Active Record for Rails, I was able to stick to using the same language/DSL throughout the application, which is nice. I discuss more about Mnesia, including its locking strategies for handling concurrency, in the documentation report.

Dependencies

Aurora has only one dependency, jsx (which conveniently shares the same name as the XML-like markup language popular in React), a JSON parsing library that converts JSON strings (represented as utf-8 binary) into Erlang terms and vice versa via its decode and encode functions. jsx is also nice enough to convert keys into atoms, and has map support via its return_maps option (godsend). Otherwise, output is represented as proplists.

Parting Notes

As part of the school project’s requirements, we were also required to write a documentation report. Aurora’s part is from Page 21 to 57 and goes into implementation-specific details and what functionalities it (and by extension, the Android chat messaging app itself) supports. Thanks to Glen and Francisco for teaming on this project.

Relevant Links

Github repo
Documentation Report
Resources I Used:
Learn You Some Erlang
Official Erlang docs on, for example: lists, Mnesia
Mnesia: A Distributed Robust DBMS for Telecommunications Applications (H. Mattsson, H. Nilsson, C. Wikström)
Other useful Github repos:
jsx (JSON parser for Erlang)
contrib-erlc (Erlang linter plugin for Sublime Text/Linter)

So familiar and overwhelmingly warm
This one, this form I hold now
Embracing you, this reality here
This one, this form now, so
Wide eyed, and hopeful
Wide eyed, and hopefully wild
We barely remember
What came before this precious moment
Choosing to be here right now
Hold on, stay inside
This body holding me, reminding me that
I am not alone in-
This body makes me feel eternal
All this pain is an illusion

- Parabol, Tool, Lateralus

Faint recollection of how Ænima used to be my display picture on MSN Messenger, almost a decade ago.

Love is a smoke raised with the fume of sighs;
Being purged, a fire sparkling in lovers’ eyes;
Being vexed, a sea nourished with loving tears.
What is it else? A madness most discreet,
A choking gall, and a preserving sweet.

- Romeo, Romeo & Juliet, Act 1, Scene 1