Summer of Code final update

As of 20th August Google's Summer of Code program has officially ended. Students must upload code produced up until the deadline to Google's hosting, and are only judged on work up to that date.

Continuation of GSoC projects is encouraged however, and I shall be finishing off HTTPS (and have already committed bug fixes) and tidying parts of my code over the next few weeks.

There has been plenty of interest in my summer's work, on the gloox mailing list, and in private emails/IMs to me personally. It is great to know that people are finding my work useful to them.

I would like to thank Jakob my mentor for endless help with figuring things out when they went wrong, and for guiding me around the innards of gloox when I got lost (not to mention for his excellent library existing in the first place!). Thanks to Peter for taking on the hard task of orchestrating the XSF's involvement in GSoC, and for supporting the students throughout. Finally thanks to Google and the team behind the administration of GSoC, for, well, you know what :)

I will update this post with the URL to my uploaded tarball once it is on Google's servers.

However the latest, greatest code will always be found in SVN. Currently BOSH is at svn://svn.camaya.net/gloox/branches/work/mw/bosh but very soon will be moved to the trunk. More information on gloox's download page.

The server-side implementations that I know work with this code are Openfire 3.3.2 (or .3), and Araneo (a standalone CM (connection manager), which allows any BOSH client to connect to any standard Jabber/XMPP server, using the CM as a kind of proxy) from ff.bluendo.com (site down at the moment I write this).

PS. Apologies for the delay in this post... blame my vacation :)

Update: Official tarball uploaded here. Please use the version from SVN (see above) though if you would like to test or use my code.

Summer of Code Update

This week has been a good one for my project, despite my being ill for a few days of it. Firstly, I completed my initial implementation of multiple connections! I say 'initial' because it is currently only tested and working with one server (OpenFire), and I will be testing more, and likely fixing bugs, in the next few days.

I also received a lot of help from people following my previous blog post...

My problems with PunJab are fixed with minor code changes (although it still doesn't work correctly for other reasons at the moment).

badlop let me test against his ejabberd SVN server, and I discovered that it doesn't seem to support persistent HTTP connections, let alone pipelining. Now I have non-persistent connections implemented, I shall be testing ejabberd again.

OpenFire... well, it works (without the previous problems I had with a 30 second delay) with multiple connections, and works well too :)
The only downside is still the lack of XEP-0206 support, so my code won't work with SASL auth to OpenFire.

Which brings me to the news of a new CM, which I have high hopes for. If I can get my code working as well with this CM as it currently works with OpenFire, all servers can be connected to this way, and SASL login will work as well.

Summer of Code Progress

In the past 2 weeks, I have tried every server-side BOSH implementation I can find (although more are always finding me!), and all have their individual problems with my client...

Openfire
No support for XEP-0206, and hence no SASL login is possible. I also have the reproducible (yet still mysterious) problem I blogged about in my last post. I also have had problems with Openfire's overactivity detection. The XEP specifies that the client is always allowed to make a request if it is to send data. The old version of the XEP which Openfire implemented was not clear on this, and as far as I can tell I get disconnected when I should not be.

ejabberd
Well, it seems to work. Yet ejabberd does not support XEP-0206 either, and also refuses non-SASL logins. This makes it impossible to use with my code for now.

JabberHTTPBind (JHB)
A handy little servlet which implements XEP-0124 and a little bit of XEP-0206. Yet I have problems with this also, as it seems to take wait seconds before the session creation response is received. After that I get a reply, which triggers a reply from gloox, to which the server replies instantly (yes, it appears to be polling for some reason). This happens until I begin getting errors from Tomcat (I believe JHB disconnects the client for requesting too frequently). I suspect a bug in the logic for polling, which I will take a look at after implementing multiple connections.

Punjab
The problem with Punjab seems quite simple, yet I have been unable to solve it. All my testing so far is being done locally, and Punjab seems to try to resolve 'localhost' on the external DNS. I don't know Python well, so this is the best I can understand of the problem. I intend to speak with the Punjab developer(s) this week.


All of the above are problems I am having, I would not be surprised to learn that many of them are a problem within my code. Unfortunately not having a working server implementation makes it very hard to know where the source of any particular problem lies. If anyone has suggestions about issues I have listed above, please do contact me :)

Today I managed to talk with Ian Paterson (author of the XEPs), who will kindly allow me to test against his server. Also I spoke with the anonymous commenter on my last post who has been very helpful, and offered me their Python implementation to play with as well. I learnt this week that Tigase has BOSH support in-development. With luck I will be able to test against Tigase in the coming weeks.

This week I also began the biggest change in the code for a while - enabling multiple connections to be made to the connection manager. This will help the client to work through proxies which don't support/allow HTTP 1.1 and/or pipelining.

Summer of Code Update

Time for an update on my progress with my project, now we are halfway through GSoC...

Disconnection (either by error, or as requested by the gloox client) gracefully from the server is working now. I have also made some small improvements in other areas where necessary, too many and too small to list here.

A big problem I am having is that there are regular delays in my test setup. I have a test client for gloox that replies to messages it receives. I have tried both tkabber and Psi as the second client, with the same results.

I checked that my BOSH code is doing everything correctly and it seems the answer is (insert disclaimer here) yes.

Here is the link to a network capture log: Wireshark capture file

The file is so named because the test message I sent was the text "ooo". (No, I don't know why...)

Below is my translation to English of the necessary parts of the log, showing a message going from Psi to gloox (via the server) and the reply going back again:

A: GUI client (Psi 0.10, port 32921->5222)
B: BOSH client (gloox svn, port 45069->5280)
S: Jabber server (Openfire 3.3.2, port 5222)
CM: BOSH connection manager (Openfire 3.3.2, 5280)

# GUI client sends message to server
0 sec: A->S (frame 37 in the capture)
0.000030: TCP ACK for the above

# Server sends message to gloox via BOSH
0.001999: CM->B

# Gloox sends composing event
0.003065: (Composing event) B->CM
0.003090: TCP ACK for the above

# Gloox immediately after this sends reply message
0.003358: B->CM
0.003370: TCP ACK for the above

# Server sends composing event to GUI client
0.004707: (Composing event) S->A
0.004725: TCP ACK for the above

# Server replies with empty response to gloox,due to 30s inactivity (as per spec)
30.007330: (Empty response) CM->B

# Server sends message to GUI client
30.012078: S->A

# All the acks for the last packets
30.012078: TCP ACK for last TCP ACK (RTT to ACK was 30.007 seconds) (??????!)
30.012106: TCP ACK for S->A
30.046637: TCP ACK for (Empty response) CM->B

If anyone could explain why the server appears to hold a message for 30 seconds, please let me know!

The story with ejabberd (which I finally managed to patch, compile and build) isn't good either. Because the new version of the BOSH XEP does not define how to restart a stream, XEP-0206 support is needed, which now defines this function. Unfortunately no server supports this yet, so I disabled SASL until I find a way around this. Unfortunately ejabberd forces clients that advertise XMPP 1.0 to use SASL (which gloox does) so it denies login attempts.

Detecting which version of the BOSH protocol is supported by the server, and using the old method when necessary would work in this case, but not in Openfire's (it reports version 1.6 already).

There are a few server-independent connection managers around, and I will test with those. My intention is to find one that I am able to bring up to date with the current specifications.

This is my focus at the moment, but very soon I will begin implementing multiple connections (a big step) for when pipelining can not be used.

Summer of Code Update

This post is somewhat delayed, for a variety of reasons, and from today I'll hopefully be posting more regularly again.

Despite the lack of posts, I have still been working, and I have made some progress too.

Code for pipelining (a single connection) to the BOSH connection manager is roughly in place. My first message was sent through BOSH last week!

I do have a couple of problems, both of which I believe are down to Openfire's lack of full BOSH support.

SASL authentication will not work, because Openfire does not support the xmpp:restart attribute defined in XEP-0206. I have temporarily overcome this by disabling SASL authentication attempts in the simple client I am testing with. I will see how I can make the code backwards-compatible with the old HTTP binding in this respect*.

The second problem is that Openfire disconnects the client if it sends requests too frequently.

As I understand XEP-0124 however, if no requests are outstanding (and hence no way for the server to push data to the client) the server should allow a request within any amount of time. Although the latest version of the XEP is more clear about this, version 1.5 (on which Openfire's support is based) does not explicitly state it. As a result Openfire is being more restrictive than it should be. I intend to file a bug report for this.

My last problem was a bug I encountered in gloox (resulting in message replies not always getting sent), and it affected the test client I have that is using my BOSH class. It happens that the bug was already fixed, and simply merging with the latest gloox trunk fixed it.

My current and next tasks are to fix an intermittent segmentation fault, and an out-of-range exception (which I believe to be both related), and improve the handling of disconnecting from the server.

* This is not helped by Openfire currently reporting the wrong protocol version...

Vote!

I just found this site, The Webware 100 in which Jabber appears next to many other networks and IM-related websites.

Do a good deed, and if you haven't already, vote for Jabber! :D

(That is, if you think it deserves the vote of course. If you don't, let me know and I'll correct you...)

Update: Whoops, I linked to the wrong category!

Summer of Code - Week 1 summary

As promised I shall give some update on my progress with the project so far.

We held the second weekly meeting, though with Kev temporarily in stpeter's seat - logs here.

I finally have a clear picture of what I am doing and where I am going.

As I said in the meeting, my project is not something I can make fancy screencasts from. On the other hand, I think I will try and produce some diagrams that show what BOSH is, and how it works. I realise that this is not clear to many people who might be reading this yet :)

Coding
Down to the important part. Today I finished my first steps and managed to get my initial code to compile (through the gloox makefile). I haven't finished the simple test client I will be using, so I can't actually test it yet. If I could however, it would hang immediately after connection... but it's surely a start! :)

Once I get some real code I think I will start producing tarballs of my progress. I am currently working in a local SVN repository, so it is not hard for me to do.

Back to work...

First week of Summer of Code

The coding period of Google Summer of Code has officially started (since the 28th of May). A meeting was held on Tuesday for students and mentors working on Jabber projects.

So far...

The laptop

I have set up my new laptop for development, including development tools (subversion, SciTE), and server that I will be using to test my code against.

The server

I have chosen Openfire to be my reference server initially, although obviously I will be testing to ensure compatibility with all servers later on (just in case...). Openfire is easy to install and use, and includes XEP-0124 support by default.

Other implementations

I am interested in finding other clients or their developers that have XEP-0124 implemented already. It would help me make the final decisions this week about the way I will be implementing it. If you know something about it, or know someone who does, please contact me!

The code and protocol

Gloox thankfully already has a nice framework for adding new network-level connection methods. It is this way that HTTP proxy and SOCKS5 support has already been added. Thanks to Jakob's foresight it is also possible to chain multiple classes so that BOSH will be able to work though a SOCKS5 proxy, for example.

I need to create a class descended from ConnectionBase and write implementations of the pure functions from it. I will also have to descend from ConnectionDataHandler to parse the replies from the BOSH connection manager, before gloox is able to parse and handle the XML.

The main difficulty I have right now is deciding which BOSH parameters should be fixed in the implementation, and which should be configurable. I am also looking to see if I can make the connection degrade gracefully from HTTP pipelining to multiple connections, to polling, depending upon what is supported/available in the current environment.

Identifying possible client environments is also one of my tasks. This way I won't end up, for example, only supporting cases where the connection manager is at the same address as the XMPP server (which in the setup I will be testing with, it is).

From now on

I will be posting weekly updates to this blog, and I have decided I will aim for every Tuesday, before or after the weekly meeting. Watch this space!

PS. In the past couple of weeks I have seen BOSH spelt as BISH, NOSH and BOX. Stop it! :D

Google Summer of Code

Dear Applicant,

Congratulations! This email is being sent to inform you that your
application was accepted to take part in the Summer of Code.

Yay! I applied to work on gloox, a C++ Jabber library, under the hood of the XMPP Software Foundation as the mentoring organisation.

This project is important to me in many ways. Firstly I know I need to get back to C/C++ coding, of which I have not done much for quite some time. Second, my task is all about adding XMPP-over-HTTP support, which is currently my only way around my ISP's restrictions. Gloox is also the library that Tessa (my new Jabber client, more coming later) uses.

All this, plus the (much needed) payment, and being able to add this to my CV, all is good :)

I will be posting development progress updates to this blog, tagged with 'Jabber' and 'GSoC' (view) (rss).

Some other interesting projects I really look forward to seeing:

• Data Form Designer Suite for XMPP (Jabber)
  
• Jingle Audio/Video for Gajim (Jabber)
  
• Revision Controlled Home Directories (Ubuntu)
  
• Mouse Gesture Recognition for the Desktop (Ubuntu)
  
• Proposal for the Development of a Graphical BZW 2.0 Editor (BZFlag)
• Interactive Wiki Maintenance with Jabber/XMPP (MoinMoin Wiki)