• How Ruby Classes Are Implemented

    Previous post: How Ruby Objects are Implemented

    Ruby classes are represented by a RClass structure. It’s a fairly big structure, so I’ve broken it down according to what a Ruby class actually does and what it contains.

    RClass

    A Ruby class is a Ruby object that also contains method definitions, attribute names, a superclass pointer, and a constants table.
    - Pat Shaughnessy

    class Fruit
      attr_accessor :colour
      attr_accessor :taste
    end
    

    A Ruby class contains method definitions.

    Method definitions are stored in a m_tbl method table. In Fruit, this will be a table that contains the keys colour, colour=, taste, and taste=, and whose values are pointers to the actual method definitions and their YARV instructions.

    A Ruby class contains instance-level attribute names.

    As mentioned in the previous post, a RObject contains a pointer iv_index_tbl which is a hash table that maps attribute names to their positions in the ivtpr array. In Fruit, this will be a hash table that contains the keys colour and taste, and whose values are their indices in the ivptr array.

    A Ruby class is also a Ruby object.

    p Fruit.class           # Class
    p Fruit.singleton_class # #<Class:Fruit>
    

    If you print out Fruit’s class, it shows Fruit as being an instance of the Class class. A Ruby class is also a Ruby object.

    As a Ruby object, a Ruby class will also have:

    • A klass class pointer that references the RClass of its class (and is stored, like any other RObject, in a RBasic structure)
    • Its own methods (called class methods - those that you define with def self.some_method). These methods are not stored in the class itself, but in its metaclass, or sometimes called the singleton class. This singleton class is what klass is set to, not Class.
    • A table of its own class-level instance variables called iv_tbl

    A Ruby class has a pointer to a superclass, which allows it implement inheritance.

    class Pear < Fruit
    end
    
    p Pear.class      # Class
    p Pear.superclass # Fruit
    

    A Ruby class will have a super pointer that references its superclass. In this case, Pear will have a klass pointer referencing Class, and a super pointer referencing Fruit. There is a dinstinction between the two - klass refers to the class from which the Pear class object is instantiated, whereas super refers to Pear’s logical superclass. The Pear class object is not instantiated from Fruit, but Class.

    A Ruby class contains constants.

    class Pear < Fruit
      SIZE = 6 # units in cm
    end
    

    A Ruby class can contain constants. Constants are stored in a const_tbl constants table.

    Ruby classes are also objects. When a class is created, Ruby actually creates two objects - the class object itself, as well as a metaclass object.

    Class Variables

    In Ruby, there are class variables, which are dintinct from class-level instance variables, as mentioned above. Class-level instance variables are defined with a single @ prepended to the variable name and are scoped to each individual class object. Class variables are defined with a double @@ prepended to the variable name and is visible globally across all instances of the class. For example:

    class Fruit
      @smell = nil
      @@edible = false
      def self.smell
        @smell
      end
      def self.edible
        @@edible
      end
    end
    
    class Pear < Fruit
      @smell = "Fragrant"
      @@edible = true
    end
    
    p Fruit.smell    # nil
    p Pear.smell     # "Fragrant"
    p Fruit.edible   # true
    p Pear.edible    # true
    

    Internally, both types of variables are stored together in the same iv_tbl tables of both Fruit and Pear. When setting a class-level instance variable, @smell is set to nil in Fruit’s iv_tbl and "Fragrant in Pear’s. However, when setting a class variable, Ruby checks the iv_tbl of that class and all its superclasses for the presence of the variable, and sets them all to the same value. When Pear redefines @@edible to true, that change is also reflected in Fruit’s iv_tbl.

  • How Ruby Objects Are Implemented

    Next post: How Ruby Classes are Implemented

    I’m currently reading Pat Shaughnessy’s excellent book Ruby Under a Microscope, and these are notes that I’ve summarized from the chapters I’m currently going through. This post, and the next, are notes from Chapter 6, Objects and Classes. It can be a bit confusing to describe the content purely in words, but the book itself contains many helpful diagrams, so pick it up if you’re interested!

    RObject

    Every Ruby object is the combination of a class pointer and an array of instance variables.
    - Pat Shaughnessy

    A user-defined Ruby object is represented by a structure called an RObject, and is referred to by a pointer called VALUE.

    Inside RObject, there is another structure called RBasic, which all Ruby values will have.

    Aside from the RBasic structure, RObject also contains numiv, a count of how many instance variables the object has, ivptr, a pointer to an array of values of the instance variables, and iv_index_tbl, which is a pointer to a hash table stored in the object’s associated RClass structure that maps the name/identity of each instance variable to its position in the ivtpr array.

    class Fruit
      attr_accessor :colour
      attr_accessor :taste
    end
    
    apple        = Fruit.new
    apple.colour = "red"
    apple.taste  = "sweet"
    
    orange        = Fruit.new
    orange.taste  = "sour"
    

    In this case, the RObject representing apple will have a numiv of 2, and its ivptr will be a pointer to an array containing the values red and sweet. The RObject representing orange will have a numiv of 1, and its ivtpr will be a pointer to an array containing just sour.

    Both apple and orange will have a RBasic structure whose klass pointer references the same Fruit RClass structure.

    Generic Objects (RString, RArray…)

    Generic objects such as strings and arrays are represented by more specialized versions of RObject, called RString, RArray, etc. Their internal representations are more optimized for the kind of values they store. An example of this optimization is the presence of ary, which is an array of a certain fixed size. This array will be used to store the values of the instance variables in the structure itself if they fit, instead of allocating memory for and referencing an external array.

    They also contain the RBasic structure.

    RBasic

    RBasic contains a few internally-used flags and a pointer to its associated class, called klass. Classes are represented by a RClass structure, which is discussed in the next post.

    Simple Values

    Simple values like (small) integers, nil, true and false do not have an associated RObject-like structure. Instead, their value is stored directly in VALUE itself. The identity of these values are indicated by different flags in VALUE (do not confuse the flags in VALUE with those in RBasic, they are different).

    For example, if the FIXNUM_FLAG is 1, then Ruby knows to intepret the rest of VALUE as an integer value instead of a pointer address to its associated RObject (or RString etc.) structure.

  • Logging and Notifications with Rogger, Graylog2 and Twilio

    Rogger 0.1.2 was released yesterday, so check it out if you need a way to log your Rails applications / Rake tasks on Graylog2!

    Anyway, this post is a showcase of how we’re using Rogger in production to log exceptions raised in Rake tasks and notify us by SMS using Twilio.

    We have a few Rake tasks that run daily. Some of them are pretty sizable, and we want to know if any of them fails and throws an exception. Using Rogger’s log_exceptions, we can wrap the task like so:

    namespace :important_tasks do
      task :first_task => :environment do
        Rogger.log_exceptions do
          some_function_that_might_fail_and_throw
        end
      end
    
      task :second_task => :environment do
        Rogger.log_exceptions do
          Rogger.info "Starting task important_tasks:second_task..."
          bad_function_that_might_fail
        end
      end
    end
    

    If the task does throw, Rogger will log it in Graylog2:

    log

    Notice the level. Level 4 corresponds to the :error level (level 7 for :debug), which is what log_exceptions sends exception messages with.

    Graylog2 Streams

    We then create a stream in Graylog2 with the following rules:

    source must match exactly MyImportantApp
    level must be smaller than 5
    

    The source name corresponds to the app_name setting as configured in rogger.yml, and defaults to <%= Rails.application.class.parent if defined?(Rails) %>.

    Twilio Notifications

    We install the official Graylog2 Twilio SMS plugin on all our Graylog2 nodes. The instructions are in the README but it’s really straightforward - just wget the .jar file into your Graylog2 plugins folder and restart the Graylog2 server process (./graylogctl restart).

    Once it’s successfully installed on your Graylog2 nodes, you can add the alert as a callback, and add your Twilio account details in:

    alert

    We can test if the alert is working by using “Send Test Alert”, and even do a full integration test by writing a sure-fail test (something like x = 1/0 will do nicely):

    messages

    Conclusion

    Now, everytime any of our aforementioned daily Rake tasks throw an exception, we will be notified by SMS - thankfully we set it up to run daily at a sane time (no, not in the middle of the night).

  • Rogger 0.1.2 Released

    Rogger v0.1.2 has been released - a patch that makes logging to Graylog2 even easier.

    For example, in a Rake task you can do something like:

    namespace :my_tasks do
      task :first_task => :environment do
        Rogger.debug "This is a debug message"
        Rogger.info  "This is a info message"
        Rogger.warn  "This is a warn message"
        Rogger.error "This is an error message"
        Rogger.fatal "This is a fatal message"
    
        Rogger.info do
          { short_message: "Don't forget to include a short message as required by GELF", custom_key: "This is a custom key-value pair that will be parsed by Graylog2", custom_key2: "This is a another one" }
        end
    
        Rogger.log_exceptions do
          x = 1/0 # will raise a ZeroDivisionError that will be logged
        end
    
        Rogger.log_exceptions! do
          x = 1/0 # will log the exception and rethrow the exception
        end
      end
    end
    

    Notably, this is also the first time ever that I’m using metaprogramming in actual code. The actual bit is below:

    ::GELF::Levels.constants.each do |const|
      module_eval <<-EOT, __FILE__, __LINE__ + 1
        def self.#{const.downcase}(msg = "", &block)
          @@logger.#{const.downcase}(msg, &block)
        end
      EOT
    end
    

    This bit defines the debug, info etc. module-level methods that you see above using module_eval, encapsulating @@logger singleton’s methods as module methods instead. These levels correspond to Rails log levels and are used elsewhere in gelf as well, so Rogger takes advantage of that.

    The user, then, does not have to deal with the actual GELF logger instance, as is already the case pre 0.1.2 (Rogger hooks onto the Rails logger by using ActiveSupport::Logger.broadcast).

  • Invalid Gemspec

    A minor, but frustrating bug. While working on Rogger, I came across the following problem while bundling my local Rogger into a test project:

    The gemspec at /Users/siawyoung/github/rogger/rogger.gemspec is not valid. The validation error was 'rogger-0.1.1 contains itself (rogger-0.1.1.gem),
    check your files list'
    

    I was quite confused. I checked the gem file but nothing inside seemed like a reference to rogger-0.1.1.gem.

    I tried building the gem:

    $ gem build rogger.gemspec
    WARNING:  See http://guides.rubygems.org/specification-reference/ for help
    ERROR:  While executing gem ... (Gem::InvalidSpecificationException)
        ["rogger-0.1.1.gem"] are not files
    

    The same error happened.

    It turns out this happens because the gemspec gets the list of files from the

    `git ls-files -z`.split("\x0")
    

    command. I happened to check the gem file rogger-0.1.1.gem in, which was why this error was occurrring. Removing it, then, fixed it:

    $ rm rogger-0.1.1.gem
    $ git commit -am "Remove rogger-0.1.1.gem"
    $ git push