Roll your own lazy loading collection

Today I realized (just again) how much I like the new AREL way of building queries. Especially the querie being actually executed at the moment you access the iterator (calling each for instance):

customers = Customer.where(:status => "approved").order("name DESC")
# No SQL fired so far

customers.each do |customer|
  puts "Name: #{customer.name}"
end
# Calling each, will fire the SQL - nice!

Ok, you all know that already, so why bothering you with that?!

Cause I realized (just again) how nice this works together with fragment caching in Rails and how I miss this feature when not working with ActiveRecord but doing some time consuming as a database query as well.

For today this other time consuming stuff was parsing a RSS feed downloaded from a Wordpress blog and display each retrieved article on each and every page. Of course this is something you wanna cache. And you want to cache both, getting and parsing the RSS and rendering each article.

In the old days I would implement cumbersome duplicate code in both the view and the controller to handle the caching correctly:

# app/controllers/application_controller.rb
class ApplicationController < ActionController::Base

  before_filter :fetch_blog_entries

  private
  
    def fetch_blog_entries
      unless fragment_exists?("blog_entries")
        @blog_entries = BlogEntry.find(:all)
      end
    end

end

And here the corresponding view code:

/ app/views/layouts/_blog_entries.html.haml
#blog-entries
  %h2
    Latest Blog Posts
  - cache "blog_entries", :expires_in => 1.hour do
    - @blog_entries.each do |blog_entry|
      #blog-entry
        %h3
          = link_to blog_entry.title, blog_entry.link

        = raw blog_entry.content
        %p{"id" => "creator-info"}
          written by #{blog_entry.creator}, #{time_ago_in_words(blog_entry.created_at)}

I'll admit in that simple case the duplication isn't a real pain, but hey, we all need some fun from time to time ;-) Due to this I went on to remove that nasty double cache check logic. And here we come back to where I started: I wanted something like the AREL behaviour for this one too! And since Ruby is some great language with a great support for closures this wouldn't be much of a problem, right? Right!

Here is the implementation of the BlogEntry#find-method before I added support to lazy load the RSS-feed:

# app/models/blog_entry.rb
class BlogEntry
  class <<self

    def find(*args)
      options = args.extract_options!

      feed = Nokogiri::XML(open(MyApp::Application.config.blog_feed_url))
      feed.xpath("//item").collect do |item|
        BlogEntry.new(extract_attributes_from_feed_item(item))
      end
    end

    def extract_attributes_from_feed_item(item)
      attributes = {}
      
      item.xpath("*/text()").each do |text|
        attribute_name = case text.parent.name
                         when "pubDate" : :created_at
                         when "encoded" : :content
                         else
                           text.parent.name.to_sym if accessible_attributes.include?(text.parent.name)
                         end
        attributes[attribute_name] = text.content if attribute_name.present?
      end

      return attributes
    end

  end
end

To get a lazy loaded collection of all the articles in the feed I just need a collection wrapper to store my closures:

# app/models/lazy_load_collection.rb
class LazyLoadCollection
  include Enumerable

  def initialize(lazy_collection, after_load_callback = nil)
    @lazy_collection     = lazy_collection
    @after_load_callback = after_load_callback.present? ? after_load_callback : lambda { |args| return args }
  end

  def each(&block)
    collection.each(&block)
  end

  private

    def collection
      @collection ||= @after_load_callback.call(@lazy_collection.call)
    end
end

As you can see I had to create some callback functionality to process the the raw collection before returning it to the caller. In this case it was creating BlogEntry instances. In the case there is no callback given I just pass the collection to a noop callback.

Backed by that new LazyLoadCollection class the BlogEntry#find method could be refactored to the following:

# app/models/blog_entry.rb
class BlogEntry
  class <<self

    def find(*args)
      options = args.extract_options!

      lazy_feed = lambda { Nokogiri::XML(open(MyApp::Application.config.blog_feed_url)) }
      create_blog_entries = lambda { |feed|
        feed.xpath("//item").collect do |item|
          BlogEntry.new(BlogEntry.extract_attributes_from_feed_item(item))
        end
      }

      lazy_load_collection = LazyLoadCollection.new lazy_feed, create_blog_entries
    end

    def extract_attributes_from_feed_item(item)
      attributes = {}
      
      item.xpath("*/text()").each do |text|
        attribute_name = case text.parent.name
                         when "pubDate" : :created_at
                         when "encoded" : :content
                         else
                           text.parent.name.to_sym if accessible_attributes.include?(text.parent.name)
                         end
        attributes[attribute_name] = text.content if attribute_name.present?
      end

      return attributes
    end

  end
end

And that's it! We got now a nice and clean little lazy loading collection, even with a callback feature included. All in less than 20 lines of code. Ruby is just amazing, isn't it ;-)

See you Space Cowboy ...
blog comments powered by Disqus
 
Fork me on GitHub