Adventures with Ruby and Jabber

A while back I discovered JWChat – a browser-based IM client that uses only HTTP to communicate with a server – in other words very friendly to restricted environments where you are behind a firewall and you can't install software.

JWChat uses a standard binding of XMPP (aka Jabber) over HTTP – in other words it speaks HTTP to a Jabber server. Now the good thing about many Jabber servers is that they can talk to MSN, Yahoo and AIM as well as other Jabber servers. This standard is defined in XEP-0124 and is implemented directly by some Jabber servers. Unfortunately not by the Jabber server that my ISP (dreamhost ) allows me to run. But wait! I can run a rails server on dreamhost, so it was just a simple matter of writing a gateway in Ruby that implemented XEP-0124. Of course, nothing in software development is ever simple (don't believe anyone who tries to tell you otherwise).

So, apart from the task of implementing XEP-0124 in Ruby, I also have to cope with the fact that my ISP uses Apache/FastCGI to provide Rails access. The problem here is that I need to open a single persistent connection to a Jabber server for each user session, whereas FastCGI spawns multiple processes to handle individual incoming HTTP requests.

Enter DRb. DRb is Distributed Ruby. With this I can create a single Ruby server process to which all of those FastCGI processes talk. At least I could if Dreamhost allowed me to run daemon processes. But they don't. They periodically go around and kill them. Besides, who wants to have the hassle of running a daemon process? So I made it so that the first FastCGI process that needs the gateway server becomes the gateway server. To prevent race conditions, where several processes try to become the gateway at the same time, I have to obtain a lock global to all the processes. A file lock was the most obvious way to do it, so that's what I did.

Now I have a neat little system whereby the gateway process comes into being just in time. Unfortunately Apache seems to have some pretty arcane rules about when it decides to kill FastCGI processes off. So, to prevent my gateway process from being terminated I had to fork off a daemon process from the FastCGI process. Ruby 1.8 doesn't do this. Fortunately some enterprising people have worked out how to do it and allowed people to use their code. So now I have a neat little system where an XEP-0124 gateway daemon gets run as needed.

Ruby on Windows can't fork, so on Windows I fall back to just not daemonizing the gateway process. This works fine for SCGI. It would even work fine for FastCGI if you have access to the httpd.conf file and can spawn off a few static FastCGI processes.

Anyhow, my little XEP-0124 in Ruby works. There's quite a few things that need to be implemented to make it fully XEP-0124 compliant (such as SASL support, support for more than two simultaneous connections per session, packet ordering etc.) but it works between JWChat and my host's Jabber server. Its hosted over at RubyForge , and if you want to give it a try there are some instructions here. If you want to develop it further, add a comment to this post and I'll add you to the RubyForge project.

I discovered an issue with how Dreamhost shared servers are configured. It seems they only allow two dispatch.fcgi processes to be run. Each process only serves one HTTP request at a time which means that the binding mechanism defined in XEP-0124 doesn't really work since if there is more than one active Jabber session, both processes are hung up on those sessions. I had to modify the Ruby module a little to make it correctly fall back to polling. To do this you need the new version and then set REQUESTS to 0 in xmpphttpbind.rb.

BTW I also implemented support for servers that use SSL, such as GTalk.

12 Responses to “Adventures with Ruby and Jabber”

  1. George Says:

    Hey, as it happens I find myself in need of this very thing, for precisely the reason you needed it…

    Looking to use JWChat hosted on dreamhost, basically…

    So I was wondering how this has worked out so far. It seems like it’ll be pretty easy to install and use (though I need to get more familiar with Rails and such) – so I guess you’ve been running this thing for a few months now. How’s it holding up?

  2. judge Says:

    It works just great. One thing: I found that the default install of rails was flaky – basically it seemed to have memory usage issues, so I had to install my own copy of rails and use that. There is information on the dreamhost wiki about how to do that so it should be no biggy.

    If you do it, let me know how it goes.

  3. George Says:

    Is this still true since DH upgraded their Rails install in May? If I try it without installing my own Ruby and Rails, what problems should I expect if the memory usage issue is still there?

  4. George Says:

    Hey, I’ve been trying to get this thing running without too much hand-holding but I’m feeling a bit overwhelmed trying to learn enough Rails to get this thing working. So I was wondering if you could help me out a bit…

    1: I have the xmpp4r gem installed in a subdirectory of $HOME – and in BASH sessions Ruby is told where to find this via an environment variable. How can I make that gem visible to Ruby running on the web server?
    2: The RhbController class – where does it go? app/controllers? How do I invoke it?
    3: Anything else I need to do to the rails app directory? (Prevent it from using SQL, anything like that? I suppose my case may be a bit odd since we’re not using Rails for anything else…)

    I think those three bits of information are all I need to get it running at this point. I saw the bit about installing xmpphttpbind.rb into the lib directory (simple enough) and the config file for JWChat – that’ll be easy enough to alter… And I’ve got the rails app running, though it doesn’t do anything right now apart from showing “welcome to Rails” and such…

  5. judge Says:

    The issue I had with the standard rails install at DH was that the FCGI process would get too big and DH would come around and kill them. This lead to a lot of server errors being reported by JWChat. You should try it and see I guess. As regards that you will want to edit dispatch.fcgi to stop some of the kills. This is mine:
    require File.dirname(__FILE__) + "/../config/environment"
    require 'fcgi_handler';

    class RailsFCGIHandler
    private
    def frao_handler(signal)
    dispatcher_log :info, "asked to terminate immediately"
    dispatcher_log :info, "frao handler working its magic!"
    restart_handler(signal)
    end
    alias_method :exit_now_handler, :frao_handler
    end

    RailsFCGIHandler.process!

    And you'll want to make rails use FCGI by editing the .htaccess file in your public folder, there are instructions in the file itself.

    Now the simple stuff. Yes RhbController just goes in app/controllers. You should make sure that index.html works the way you want it to – i.e. you could make it the JWChat main page for example.

    As for getting xmpp4r on the path for rails. Well, I guess there is probably a good way, but you could just copy the package into the rails lib directory. Some digging around in the DH wiki and/or Google might give a better answer though.

  6. George Says:

    Well, I finally got it working (maybe I’ll write up something later on about what I had to do – in case it helps somebody…), but it drops me every minute or two with “Service Unavailable” or (I think, sometimes) “Internal Server Error”…

    I don’t know if the FCGI processes are being killed off – when I’ve connected with JWChat, there are generally between four and seven instances of dispatch.fcgi listed in “top” – and each takes .8% MEM…

    I haven’t been able to identify a real pattern in the process behavior when I lose a connection – that is, I haven’t seen a process disappear from “top” at the time I get disconnected… So I’m not sure if I missed some critical countermeasure or if I just need to install my own copy of Rails as you did… At the moment, though, it’s run for a few minutes without crashing – so I don’t know, maybe it’s working, or maybe it’s just waiting to prove me wrong…

    Also, don’t know if this is normal, but there seems to be lag in sending outgoing messages (JWChat to somebody else) but apparently little or none in receiving messages (anybody else to JWChat…)

    Anyway, thanks for all the help – looks like I got my work cut out for me if I want to be able to rely on JWChat…

  7. judge Says:

    I think you should install your own copy of rails ;-).

  8. george Says:

    I’ll give it a try. Will it make any difference if it’s the same version of rails, though?

  9. judge Says:

    It made a huge difference for me.

  10. George Says:

    I found something – for a while I thought it was a JWChat bug but now I think it may be a bug in the xmpphttpbind.rb module

    Basically if someone sends me a message with a character in it that winds up HTML-encoded (like “&”) then when JWChat receives the message it displays the encoded text (“&”) rather than first decoding it…

    At first I thought it was a bug in older versions of JWChat which had been fixed in the newer versions or on the live demo on jwchat.org – still can’t rule that out, since I haven’t been able to log in from my copy of JWChat to jwchat.org’s server or from jwchat.org’s live demo to my Jabber server via HTTP binding…

    Easiest to see the effect when in a group chat – since messages are sent out to the server and then echoed back by it, instead of locally echoed.

    I’m looking through the module now to see if I can work out where to insert a fix – I thought I’d ask first to see if I’m at least on the right track. (Proxy’s working much more reliably now, BTW – I’m not sure why.)

  11. judge Says:

    BTW I discovered why there was a lag in sending outgoing messages. Dreamhost only allows Apache to run 2 dispatch.fcgi processes. Each process can only handle one request at a time. Therefore if you have 2 or more Jabber sessions going you will see the delay.

    I’ve modified the module to properly support polling and with Dreamhost shared hosting you should configure it to use polling. Set REQUESTS to 0 in xmpphttpbind.rb (this will only work with the new version at http://rubyforge.org/projects/rhb).

    BTW you can also use to to connect to secure Jabber servers (such as GTalk) now.

  12. Abhishek Says:

    Hi,

    I am trying to use Jabber:Simple for connecting to GTalk. It doesnt connect – I suspect that this is because I am behind a proxy. Does anyone know of a workaround?

    Abhishek

Leave a Reply

Judge’s Random Ramblings