We just finished our first meeting of Jabber GSoC students and mentors, with everyone in attendance but Tomasz Melcer (who is working on Jingle support in Gajim). A meeting log is here for the curious. Conclusions:
To help me keep track of the student and mentor blogs, I've added them all to my blogroll (and ralphm will add them to Planet Jabber).
The second meeting will be held next Tuesday at 17:00 UTC in the jdev room. See you there!
If you're interested in following the progress of the Jabber-related Summer of Code projects, check out the following student blogs:
Maybe these will end up on Planet SoC and Planet Jabber sometime... ;-)
Following up on yesterday's post, today I cleared out the backlog on my "idletasks" folder by adding some more projects to the jabber.org software pages:
As mentioned I've been thinking a lot about social presence lately. Most recently I've been chatting with Steve Krulewitz, a Songbird developer who created the Audioscrobbler plugin and who is interested in how XMPP can play well with Songbird.
IMHO, one thing that would be cool in Songbird is the ability to find live chatrooms where people who are listening to the same artist right now can find each other and talk. That might make the music listening experience more social, but in a way that doesn't interrupt the music (since most people can listen to music and do text chat at the same time). Probably this could be done with xmpp4moz but I suck at programming so I don't have recommendations about how to make it happen at the code level.
Right now it's possible to push tune information into XMPP presence, but that's an evil hack for several reasons, because if everyone puts their favored bits in the presence status then:
Our preferred solution is a more granular approach called personal eventing, which is starting to be rolled out now and which can be used to push out things like user tunes.
But the presence hack leads to further thoughts about integrating music more deeply into the buddy list. Why not have special groups in your buddy list for people who like the same music (perhaps people you find in those chatrooms)? Or do a form of buddy list tagging? Make the whole experience more social.
As I've complained before, the problem I have with things like last.fm and Orkut and the social networking stuff in general is that it's all so static. With XMPP we could make it much more dynamic and much more, well, social! After all, it's not very social to post a message on a group forum at last.fm and then go back and visit that page again in 2 weeks and maybe someone has replied. True sociality is a lot more interactive than that, and Jabber technologies can add much of the interactive glue that makes the experience more social and sticky.
Or so it seems to me. :)
I have an email folder called "idletasks", where I place messages I receive to do things like add projects to the software pages at www.jabber.org. Unfortunately, I don't process these tasks very quickly, but I did just add the following projects:
More to follow in the coming days.
Oh and speaking of living in the presence, my recent post about XMPP and social networking has gotten me interested again in a wide range of "social presence" applications, so I've refreshed my accounts at last.fm (got iscrobbler working again) and Plazes as well as the aforementioned Jaiku and Twitter. Experimentation continues...
VON Magazine is running an interesting article by Ross O'Brien entitled Living in the Presence -- read it to grok more deeply why information about network availability matters so much.
We've had yet another long email discussion thread in the last few days about what we call our technology. Is it Jabber? Is it XMPP? Is it Talk?
The short version of my philosophy in the matter was best expressed by Shakespeare in Act II, Scene II of Romeo and Juliet:
What's in a name? That which we call a rose by any other name would smell as sweet.
The long version is here.
Britt Selvitelle and I had a good chat just now about Twitter, Jabber, personal eventing, and the concept of lifestreams. Both Twitterers and Jabberites share a vision of making the Internet a more dynamic, alive, pulsating kind of place, focused less on pages and more on people and their endlessly fascinating activities. Services like Twitter and Jaiku have their finger on the pulse of what people are doing -- a kind of collective unconscious. So in their own specialized ways do services like last.fm and Pandora for music, Joost and presumably YouTube for video, Plazes for location, and much more. I see the need for two things to bring these many streams together into a mighty, pulsing river of real-time interaction:
Hint: I think the Jabber/XMPP community has a great deal to add to this conversation because it's the people in your buddy list who care about what you're doing, presence information provides a way to know where the information needs to go, and if every Jabber ID is a real-time eventing service then we have a way to funnel all this data through your online identity off to the people in your buddy list.
Ploum reminds us that it's only ten days until Open Discussion Day, when those of us who care about open standards encourage those who don't to use open protocols and formats like Jabber, SIP, and Ogg Vorbis. Check out the ODD site for information about how you can help.
BTW, here are the conferences I'll be attending in the near to medium term:
See you there!
I finally got around to moving the proxy65 project from JabberStudio to Google Code Hosting -- the new location is <http://code.google.com/p/proxy65/>. See you there! And yes, one of these days I'll prepare the 1.1 release. ;-)
The German news site heise online is reporting that GMX.de, probably the largest ISP and hosting service in Germany, has added Jabber support for all the domains they host. I think that's the potential for about 5 million more servers on the Jabber network...
Today we announced the official XSF Summer of Code projects over at Extended Conversation, but a little digging reveals that there are plenty of Jabber-related projects happening under other mentoring organizations, too! Here's what I found:
We wish success to all the SoC projects!
Just a reminder that I continue to blog about Jabber/XMPP over at Extended Conversation, the official weblog of the XMPP Standards Foundation. Today I posted two entries, one on XMPP-related I-D Updates at the IETF and the other on Presence Scalability (specifically comparing XMPP and SIMPLE on bandwidth usage for interdomain federation).
The good folks at VON have posted video of a recent panel discussion in which I participated at Spring VON 2007, entitled "My Mother Uses Skype -- Why Bother With Standards?" I had fun playing devil's advocate, so the video may be somewhat entertaining in a geeky sort of way. :-)
Here is some Jabber sleekness I've noted recently on the web:
Yes, sleek is the word of the day. :)
Some weeks are more productive than others. This week was less productive than I would have liked, or at least it seemed that way.
The big issue this week has been personal eventing via pubsub, specifically whether to add an atomic publish+configure action to the spec. The XMPP Council chatted about it at length in our meeting on Wednesday, and we have had long and so far inconclusive discussions on our standards list this week. Earlier today I attempted to break the logjam with a middle way. We'll see what people think. But this I know: we must gain consensus, no matter how rough. And I think we will, because all the parties are working in good faith, even though they are getting a bit frustrated right now. :)
As a result of this week's Council meeting we also advanced XEP-0202 (Entity Time) and XEP-0203 (Delayed Delivery) as modern replacements for XEP-0090 and XEP-0091. The older specs used non-standard datetime formats, which we decided to finally deprecate so that developers don't need to have two different pieces of code for datetime handling. It's a minor cleanup in the grand scheme of things, but it feels good to clean out some cruft once in a while.
On a more practical note, we also made good progress on choosing our Google Summer of Code projects. This year the process of reviewing applications has gone more smoothly, in part because we have had more time (thanks, Google!) and in part because we have had more potential mentors reviewing the apps. I still need to go through them myself, but I hope to carve out time to do that in the next few days.
One thing I did not accomplish this week was to finish the migration to Drupal 5 on our webserver so that we can start on the new jabber.org website in earnest. I must do that next week. (I also need to finish our application for tax exempt status with the IRS. Fun.)
Oh, but one bright spot: by the end of the day today I got my email inbox back down to 0 messages. There's always a certain comfort in that...
Heh. I started to type the title for this blog entry as "JabberWocky" but I mistyped it as "JabberWorky" -- which is totally appropriate! So herewith I inaugurate a new tradition: a weekly status report of my work on Jabber/XMPP technologies, delivered via blog straight from vi to you. (The idea was inspired by a conversation I had with Matt Tucker at VON yesterday. Thanks, Matt!)
So let's see. This week I travelled to San Jose for VON Spring 2007. I attend conferences only when I'm invited to speak -- that keeps me busy enough, let me tell you. This time, as previously mentioned, I was on a panel discussion entitled "My Mother Uses Skype -- Why Bother With Standards?". Jonathan Christensen of Skype (formerly of FaceTime, Microsoft, and most recently Camino Networks) was brave enough to show up for the panel discussion. I did my best to question assumptions and play Devil's Advocate (maybe the good folks at Pulver will post video sometime?). My two main points were:
People don't want choice. A journalist in attendance questioned me about that statement afterwards because he didn't know whether to quote me on it, but I stand by what I said. If the dominant option is clearly superior -- think Skype or iPod -- then the vast majority of people, by their actions in choosing that option, vote with their dollars or attention that they are perfectly happy using what's offered and that they don't particularly care about choice or more options. They want stuff that just works, and Skype or Apple or whoever deliver. The market is working. It's up to other providers to offer something better. And so far they haven't. Whose fault is that?
Open standards are important, especially in the long term, because we need to enable innovation at the edges (think Internet ecosystem, not telco silo). The problem is, the traditional standards organizations have fallen down on the job. In particular, they have gotten away from the modus operandi that built the Internet in the first place: rough consensus and running code. Design by committee, IP ambushes, and analysis paralysis have led to de-jure "standards" that nobody can implement. The solution? Leaner, meaner standards organizations that are driven by delivering features and enabling ease of development. We've tried to do that at the XMPP Standards Foundation and I think we've done a pretty good job so far (but if you disagree, I want to hear about it!).
So I tried to shake it up a bit. The A/V guys (I always thank the A/V guys, don't you?) said I succeeded, so I take that as a vote of confidence.
Aside from travelling to VON, I made quite a bit of progress on the Jingle specs, as reported a few hours ago on the standards@xmpp.org list. I'm excited about these changes because (1) they represent a serious simplification of the signalling protocol and (2) they are based on feedback from implementors, in particular Rob McQueen's team (which works on Telepathy, One Laptop Per Child, and the Nokia 770 and 800). More feedback is welcome as always, so keep those cards and letters coming! (Join the standards@xmpp.org list, which naturally is totally open, or ping me directly.)
On Wednesday we held a meeting about end-to-end encryption ("e2e") over the Jabber network. At the least, we knocked four potential technologies out of contention: OpenPGP, S/MIME, XML encryption (no one has ever gotten serious enough about it to propose an approach for XMPP), and Off-the-Record Communication -- read the log to find out why (basically they don't meet our requirements). That leaves two approaches: encrypted sessions (see XEP-0116) and some form of Transport Layer Security over XMPP (hints about what that might look like here, here, here -- follow the threads for details). I chatted some more about e2e today with Ian Paterson and we realized that we need to clarify the requirements (probably by posting them as a separate specification) and then perform a "gap analysis" to figure out how "ESessions" and "XTLS" meet our requirements. So Ian and I will be working on that over the next few weeks (Ian will concentrate on the requirements and I will focus on writing an initial spec for XTLS).
On the Boring Business Side of Things, I filed an official "Certificate of Amendment" with the State of Delaware modifying the Certificate of Incorporation of the XMPP Standards Foundation to incorporate the proposal that the Corporation's Membership recently approved for Clarifying the Purposes of the XSF. This will (we hope) help us achieve Tax-Exempt Status with the IRS via Section 501(3)(c) of the Internal Revenue Code (though I have yet to fill out and submit the rest of Form 1023). [Tangent: why is it that everything legalistic has Initial Caps? Is that to make it seem More Official? Whatever the reason, I must say I find it Rather Annoying. ;-) ]
Naturally we continued the usual conversations about standards, implementation, certification, infrastructure, and the like, but I won't bore you with the gory details since this blog entry is already way too long. So until next week I say: Jabber On!
Over at the official blog of the XMPP Standards Foundation, I just posted an update about the Jabber/XMPP community's involvement in the Google Summer of Code for 2007. Follow the links for detailed information. Oh, and pay no attention if the web interface prompts you to fill out the application template -- there is no such thing. :)
As I posted over at Extended Conversation, our sessions at FOSDEM and afterward were a smashing success. Here are some other Jabber-related happenings I've noticed recently:
The mabber service for mobile IM has been relaunched, including support for Atom over XMPP (which reminds me, I need to submit an updated version of that spec).
The folks who run jabbim.cz have released as open-source software a Jabber server component that enables you to upload files to a WebDAV data store via Jabber file transfer. Cool stuff.
George Hotelling is tapping into the wisdom of crowds by using the power of prediction markets to figure out which of the big consumer IM services will support XMPP by August 26, 2007. Read the background, register with inkling, and then play the market.
I've posted slides from my FOSDEM 2007 talks on Jabber 101 (194 slides) and secure communications (246 slides). And as mentioned, the good folks at FOSDEM have posted video of my security talk. Enjoy!
Mike Gotta makes an interesting observation:
Communication vendors have held rank for the most part and have focused almost exclusively on SIP and SIMPLE. XMPP and Jingle have been viewed as "radioactive" by some vendors. Same goes for Microsoft.
I like it! Yes, XMPP is dangerous if you're trying to control what kind of apps developers can deploy. But if you want to encourage creative mashups and innovation at the edges, then the deeply hackable real-time XML streaming technology we've been building out for the last 8 years is just the thing you've been looking for.
The choice is yours.
A little bird just told me that the hot Internet TV service Joost is heavily using XMPP for all sorts of top-secret interactive features. Well, not so top-secret, since they don't obfuscate their beautiful JavaScript code. Or so I've been told -- I haven't had time to investigate further since I need to be writing my FOSDEM talk right about now. ;-)
(And speaking of XMPP integration with cool projects, check out this new Jabber notifications plugin for Drupal.)
Here are two bits of Jabber news in Portuguese: the big SAPO IM service in Portugal has migrated to ejabberd, and the Jabber-BR group is calling for participating in the FISL 8.0 conference in Porto Alegre, Brazil in mid-April.
The other day I mentioned that I have around 1400 people in my Jabber roster (that's a Buddy List [tm] for you non-Jabberites, but I prefer not to use that term since it's been trademarked by AOL). So one of those many people IM'd me overnight, asking how I (and the jabber.org IM server, and the Jabber clients I use) manage all those contacts.
First the server. Well, no problems there. Since I am one of the server admins, I can tell you that our trusty ejabberd deployment does experience a CPU usage spike when I log in, i.e., while my Jabber client slurps down my roster. But other than that ejabberd simply chugs along happily, processing all the inbound presence information I receive.
But not all Jabber clients are quite so happy. The "stpeter roster test" is somewhat legendary as a hurdle to pass in optimizing roster processing and presentation. I've tested clients that took three or four minutes to render my roster (and that's not counting presence information)! But after some optimization, most of those clients have gotten the time down under one minute.
As to my personal IM habits, I use a lot of roster groups. One group in particular is a kind of catch-all, and it contains probably 600 people (but folks that I don't contact often, if ever). Typically I wait for people to IM me (hey, I'm interrupt-driven). But when I need to IM someone and they're not in one of my smaller roster groups, then I need to find that person fast (i.e., before I get interrupted!); thankfully, some clients (e.g., Spark) make that task easy by enabling me to essentially search my roster via Ctrl-F or somesuch keystroke. In a client like Adium or iChat I'll disable the groups view and scroll through the entire roster, but that's not necessarily very efficient.
Managing multiple simultaneous chats (both one-to-one and multi-user) can also be difficult. I'm somewhat agnostic about tabbed chats in one window vs. multiple windows. Adium uses tabs and iChat uses multiple windows, and I can handle either approach. It's all about proper placement of the window or windows on my desktop -- I typically have quite a few applications open at once (Firefox with 10+ tabs, Thunderbird with my (still empty!) inbox and one or two draft emails, four or five terminal windows, calendar, a PDF viewer, etc.) and my chat and groupchat windows go toward the bottom left of the screen, where I can keep an eye on them for new messages. Seems to work for me.
Another challenge is reading through the flood of offline messages and presence subscription requests that I typically receive when I log in after 8 or 10 or 12 hours offline (don't even think about what happens when I return from vacation -- I try not to take those ;-). Some clients present one window for each chat or subscription request, which is sub-optimal if you have received 20 offline messages and three or four subscription requests, let me tell you. I have not found a client that handles this with aplomb (perhaps because until recently the major IM services have not supported offline messages).
I don't claim that my IM profile is typical. Few people have more than 100 people in their rosters, and vanishingly few have more than 500; 1400 is almost unheard of (let alone the ~3200 I used to have!). And I doubt that such heavy IM use will ever become typical (though I do think that people will tend to have more contacts as IM becomes more popular). So you client developers might not want to read too much into my feature requests. :-) But do feel free to ping me if you'd like me to subject your client to the "stpeter roster test"...
I'm thinking about submitting a paper for the fourth Conference on Email and Anti-Spam about instant messaging spam (a.k.a. "spim") on the Jabber network -- or, to be precise, the lack thereof. If you have experience with receiving (or even sending ;-) Jabber spam, I'd love to hear from you.
The good folks at IMified are having some trouble with MSN (HT: GigaOM):
After a week of back and forth emails with Microsoft trying to get our screen name to appear online to more than 1000 users at a time, we've been told no. The limit with MSN occurs when over 1000 screen names request presence for our screen name at any given time, meaning that if your not one of the lucky 1000, our screen name will appear offline to existing users and new users cannot be added to our buddy list.
Well, I have 1400 or so people in my Jabber roster and all is well (I used to have around 3200 but I pared it back). That said, running a service like this as a bot may not be ideal. It's probably better to run your own server and write a nice server-side component to handle the flood of traffic. Oh, that's right, you can't do that with closed silos like MSN and AIM. Maybe it's better to use open technologies that you can control and extend on your own, eh? :-)
IMified is a fun new service that enables you to keep to-do items and reminders, interact with Google Calendar, and so on -- all from the comfort of your Jabber client. It's well explained here with pretty screenshots. Just add imified@gmail.com to your roster and off you go. (I'm not sure how their bot will scale, but I suppose they'll figure it out ;-)
The generous volunteers who put on FOSDEM have published an interview with yours truly, since I'll be a speaker at the conference (my topic is Secure Communications with Jabber, be there on Sunday at 15:00 :-).
There are so many errors in this article about SIMPLE in the enterprise that it would take me quite a while to correct them all. Maybe later this week I'll find the time...
I just submitted a proposal to speak at OSCON 2007, the brief version is:
Jabber/XMPP technologies not only provide an open platform for real-time communication, they do so in a high-security fashion. This talk will delve into Jabber security, including behind-the-firewall servers, reverse DNS lookups, channel encryption, strong authentication, end-to-end encryption, spam prevention, interdomain federation, and the Jabber network's intermediate certification authority.
And here's the longer version:
Although Jabber/XMPP technologies are known as the open alternative for real-time communication, their security characteristics are less familiar. Yes Jabber provides high security for organizations and individuals alike. Companies can run their own Jabber servers behind the firewall, federate with select partners and suppliers, or open communication to the entire Jabber network (including Google Talk, Live Journal Talk, and many other public services). In addition, channel encryption and strong authentication can be required for all connections. The XMPP Standards Foundation provides cost-free digital certificates to Jabber server administrators through its own intermediate certification authority. The Jabber network is so far free of spam, viruses, and other malware. And there is an active effort underway to build robust end-to-end encryption features into popular Jabber clients and code libraries. Why use insecure technologies like the consumer instant messaging services when you can use open, secure technologies like Jabber?
The other day I almost got run over at the intersection of Wynkoop and 18th Streets in downtown Denver (no, don't panic, it wasn't that close -- but if you drive a dark green Ford pickup truck with Colorado license plate 660-KJK, please know that you are a stop-sign-running, cell-phone-talking idiot who needs to pay closer attention to his driving). The experience made me realize that I really need to write up a README describing everything I do in running the XMPP Standards Foundation -- routine tasks, necessary passwords, relevant contacts, and so on. I already have a README for the role of the XMPP Extensions Editor, but a more general "stpeter.readme" is in order, too. I'll add that to my .plan for sure.
After some discussion among the Board of Directors of the XMPP Standards Foundation, we have decided to launch an official blog: Extended Conversation. And yes, we agree with Stowe Boyd that the press release is dead, so expect official announcements at the weblog from now on.
Carlo Zottmann follows up on my reply to his original post by clarifying his concerns:
I was talking about wide-spread adoption of XMPP/Jabber by the average IM user. XMPP is a superior protocol in my eyes, and I was wondering why it didn't take the public IM landscape by storm. That said, as impressive these numbers are, in my eyes corporate or governmental clients and services [don't] really count, mostly because in these cases the employer (be it a company or a country) dictate which client to use. Now if all these people would use XMPP IM clients at home as well, then that would really make a splash. Now I was wondering why not everyone is using an IM client that uses this superior protocol, and the reason is: there is no client that does really impress the public.
OK, here I reveal our grand strategy. Remember what email was like in '92 or '93? You had CompuServe and Prodigy and MCIMail and so on, and you couldn't communicate with people on other services. Then those services got the religion of open standards and they started using SMTP and everyone had a common language and all was well.
Unforunately, we're still trying to get to that point in the world of real-time communications (IM etc.). So if you're online using your AIM client, you can't chat with your friend on MSN or Yahoo. It's as if you needed a phone for Cingular, a phone for Sprint, a phone for Verizon, etc. Stupid.
Now, how do we solve that problem? While I agree that we need some really polished Jabber clients, I don't think that will solve the problem of communication silos, because the problem is more political than technical (yes, it's also social, because the buddy list is the center of the universe, but we'll get to that).
One way to attack the problem is to try to get one of the major IM services to use XMPP. Sounds easy, right? Well, not really. How are you going to talk Yahoo! or AOL into ripping out their entire infrastructure just to switch to an open standard that happens to do everything they can do today but not all that much more? Ain't gonna happen.
So we need to find a pain point for the consumer IM services, and that happens to be corporate IM. Big companies don't want all their IM traffic going through some third-party data center in Reston, Virginia or Redmond, Washington. They need and want to have control over their communication services. And they have shown that they don't think of AOL or Yahoo as a vendor of corporate IM solutions (Microsoft is a different story, though MSN is a different beast entirely from Microsoft's enterprise IM offerings). So these big companies use XMPP-based server software that they can host in-house (or they just use whatever IBM or Microsoft gives them).
This userbase of enterprise IM users has grown dramatically and continues to grow. There are tens of millions of such users. And contrary to what Carlo asserts those enterprise users do matter, because the consumer IM services would love for their siloed users to communicate with those corporate IM users (the corporations are where all the interesting commercial services exist).
How to make that happen? Enter XMPP. It's an open standard with a strong client-server model for interdomain federation. If you have an XMPP gateway and some smart federation policies in place (perhaps even a common CA for server authentication), all of a sudden your users can communicate with users at the banks and healthcare companies and so on in a secure, authenticated, spam-free environment. Like email, but really fast, with presence -- and the bad guys can't assert that they are service@paypal.com or whatever and indiscriminately spam your users.
So a big part of our strategy all along has been to build out the network of companies and service providers who are using XMPP. In 1999, when I joined the Jabber movement, there were perhaps 500 or maybe 5000 users. Now there are 50,000,000 or more (it's a decentralized technology, so we can't count them all). And there are more every day, as more companies and universities and governments roll out XMPP-based servers and as service providers like Google Talk and Live Journal Talk and radiusIM keep joining the network (sometimes adding the potential for millions of new users in one fell swoop). That huge army of Jabber users is starting to put pressure on other providers of enterprise IM software, as witness IBM's announcement that they will federate with native XMPP servers through an XMPP gateway for Lotus Sametime. And it's starting to put pressure on the huge consumer IM services, too, as witness persistent rumors about an AOL XMPP gateway.
True, those are "gateways" or specialized connectors, not native functionality (Lotus Sametime and AOL would still use their own proprietary technologies for IM). But that's how SMTP took over, too -- gateways first, then native support. It's no surprise that none of the consumer IM services or big enterprise IM vendors have fully converted to XMPP yet, because it's a big job. But the likes of AIM and IBM are inching closer as that army of Jabber users exerts some subtle and not-so-subtle market pressure. The key is to keep growing the network and userbase of companies and services providers who are deploying XMPP-based services to their employees and users, but that doesn't seem to be a problem because the Jabber juggernaut seems to have tremendous momentum.
As I mentioned in the Jabber Journal #27 or a recent blog post, we have been working on Jabber technologies for eight years now. Once upon a time it was easy to ignore the Jabber "movement" because it was just a few open-source hackers. But we have persistently kept building out those technologies -- standardizing them as XMPP through the IETF, extending them in the XMPP Standards Foundation, building more and more client and server implementations (open-source and commercial), deploying tens of thousands of XMPP-based services at companies and universities and government agencies, winning over the likes of Google and NTT and Live Journal, and perhaps most important never disappearing from the technical radar screen.
All that work isn't necessarily glamorous. It doesn't make a splash in the way that immediately winning over a major consumer IM service would (yes, Google Talk was such a splash, but it's not one of the established players). Yet it was necessary to do all that unglamorous work in order to lay the groundwork for what we have achieved so far, and it is necessary to keep plugging away at that unglamorous work in order to enable the progress yet to come. I happen to think that our future progress will involve some big splashes. Maybe one splash will be a major consumer IM service opening up to the broader XMPP network. Maybe another splash will be the kick-ass Jabber client of your dreams. I don't have a crystal ball and I don't have all the answers, but I do know we have proved that we are willing to do the unglamorous work that makes more big splashes increasingly likely. And that's why the Jabber juggernaut just keeps on rolling along...
Carlo Zottmann complains that "it's hard to find anyone who has a clue Jabber even exists" because "Jabber" lacks that one shiny client with all the fun features that end-users crave.
There are two things here. First, there are 40-50 million people using Jabber technologies these days, but most of them probably don't even know it since they think they're using Google Talk, Live Journal Talk, Chikka, IM services from NTT or BellSouth or Gizmo or whomever, presence services like Jaiku and Twitter, etc. Or they work for FedEx or HP or Adobe or EDS or just about any Wall Street bank and those companies all use Jabber for their in-house IM service. Or they're in the Marines or work for some other government agency that has deployed Jabber. Or they're using something that doesn't even look like IM because it's in fact a network monitoring service or workflow system or whiteboarding app that just happens to use the Extensible Messaging and Presence Protocol to send around some XML in real time. Or. Well, you get the picture. Jabber/XMPP is fundamentally infrastructure, not a shiny client. Think HTTP, not Firefox.
Now I grant you that it would be really really cool if someone came along and wrote a killer IM client that used XMPP to the fullest with all the doodads and gewgaws your average Internet user loves and some that they didn't even know were possible (which we can do in the Jabber community because we have this deeply extensible XML transport). IMHO building on top of Mozilla would be just the ticket (think Thunderbird or Songbird but for IM -- MynahBird perhaps?). But I'm not a coder so I'm not the one to make that happen. All we've done so far is build out the core infrastructure and standards, which has laid the groundwork for some enterprising open-source coder to make a real name for himself (or herself!) by building a kick-ass Jabber client for the ages. Will someone do it? I don't know, because that kind of thing can't be planned from the top down in a standards organization, it needs to bubble up from some mysterious wellspring of creativity inside some lone developer or small team. But if you're interested, drop me a line and I'll see how I can help.
I got word today from Tyler Johnson of Internet2 that our application for XMPP addresses to be added to the H.350 LDAP profile has been approved by the ITU. It's a dreadfully boring geeky thing, but it will enable organizations who want multimedia-friendly user directories to include Jabber IDs for better integration between the IM world and the voice and video world.
You may have heard some buzz about Me.dium, a browser plug-in that enables you to share presence and chat with people who visit similar sites on the web. What you may not know is that Me.dium is powered in part by Jabber technologies. I've been beta testing it for a while now and I think the folks at Me.dium have done a great job using XMPP to add a dynamic element to the web experience. Because they are so excited about XMPP, they have decided to open up their beta to members of the Jabber community. You can find a special invitation here.
Check it out and provide them with some feedback!
Oh, and naturally my Me.dium username is "stpeter", so feel free to add me to your friends list there. (No, Me.dium doesn't yet integrate with the roster at your regular Jabber account, I'm sure they're working on federation...)
Jabber on!
It's been 8 years since Jeremie Miller released the first open-source Jabber code. Over the years we've focused more on our open protocols, which have been standardized in the IETF along with Internet standards such as HTTP, SMTP, and SIP. When we standardized the core protocols in the IETF we called them the Extensible Messaging and Presence Protocol (XMPP). That branding has helped our open technology grow beyond the open-source world -- it has widely infiltrated the financial and defense sectors and has been embraced by the likes of Google, Apple, Sun, Nokia, and Adobe (heck, even AOL is rumored to be developing XMPP connectivity), and there are an estimated 40-50 million end users of Jabber technologies now. What keeps all these disparate interests in sync is our focus on open protocols, which has been guaranteed through our standards work in the IETF and also on XMPP extensions through the Jabber Software Foundation. Given our focus on protocols, we have just renamed the Jabber Software Foundation to the XMPP Standards Foundation.
But it's not just protocols. Unlike what you might find in relation to some Internet standards, we still have a great deal of open-source activity in the Jabber/XMPP community, which helps to keep the commercial vendors honest. We have found that the combination of open standards, open source, and an open community (no fee-based consortiums for us!) has really helped our technology grow. It sure takes a long time to break down the barriers to communication that have been put up by the closed silos of the IM, VoIP, and telco providers. After all, eight years later we're still working to make Jer's vision of the freedom of conversation a reality. But we've been doggedly persistent (yes, we're still here) and that is really starting to pay off.
So what's next? We continue to work on standardization of the core XMPP protocols through the IETF -- currently we're clarifying and updating some details in the specs and will be pushing them forward from Proposed Standard to Draft Standard this year. We continue to work on standardization of a wide variety of XMPP extensions in the XMPP Standards Foundation as published in the XEP series, including the Jingle extensions for voice and video chat. We will soon re-launch the jabber.org website as an information and communications hub for the community of people using and developing Jabber technologies.
Yes, I called them Jabber technologies, not XMPP technologies. Why? Because I think of it this way: Jabber is to XMPP as the Web is to HTTP. I still use the term "Jabber technologies" to refer to this whole universe of real-time communication products and services that companies and open-source projects and independent developers have built over the years. But I use "XMPP" to refer to the XML wire protocols that folks use to create those Jabber technologies. Jabber:XMPP::Web:HTTP. So I'll keep publishing the Jabber Journal at the new jabber.org website, and I'll keep talking about Jabber technologies and the Jabber network and the Jabber juggernaut and all the rest. And speaking of the Jabber juggernaut, I think 2007 is going to be a big year for Jabber technologies. So don't touch that dial! :-)
If the rumors are to be believed, Adobe has invested in XMPP technologies by acquiring Antepo and AOL is working on an XMPP connector. More signs of impending world domination? You be the judge.
I just published Jabber Journal #27. Enjoy!
Even before reading this post today I had been thinking more about REST and XMPP. At most, the principles of REST apply to XMPP request-response semantics (i.e., the <iq/> stanza), because that's the only time we transfer representations from one entity to another. Consider the example of an XMPP roster (a.k.a. contact list or Buddy List [tm]) from a server to a client, as explained in RFC 3921. On logging in to its server, an IM client requests its roster with an IQ get "operation" (cf. HTTP GET). We can see this as a request to read or copy the roster from the server to the client, where the XML namespace of the IQ's child element defines the "content type" in question. The server returns an IQ result containing a copy of the roster according to the server's current understanding (i.e, the server "transfers" a "representation" of the roster to the client -- perhaps we can even say that the roster is a "resource"). The client can also update the roster via an IQ set "operation" (cf. HTTP PUT) containing a roster item or deleting an existing roster item. The update is pushed out to all other connected resources via roster pushes, obviating the need for clients to poll for changes. And there are no cookies required, because the client is authenticated with its server and the authenticated connection provides enough state for the server to do its job.
So far, so RESTful. This line of thinking doesn't apply to XMPP <message/> or <presence/> stanzas, which have far different, non-request-response semantics because they don't involve the "transfer of representational state" (or do they? when I receive information about your network availability, isn't your presence state transferred to me "automatically" based on a standing subscription rather than an initiated request?). And REST principles may not even apply to all uses of the <iq/> stanza (which we use in protocols like Jingle). But thinking through how REST does or does not apply to XMPP is probably a useful exercise, and may help us devise better protocol extensions. So no, I don't consider REST to be a religion (even though some people seem to); but then again I don't consider XMPP to be a religion either... ;-)
The IETF recently published RFC 4732: Internet Denial-of-Service Considerations. It looks like a helpful summary of what to do -- and what not to do -- in building Internet-scale protocols. I'll definitely read it closely before we finish work on rfc3920bis.
I've been trying to grok REST of late (yes, I've even read Roy Fielding's dissertation). The concept is much-hyped but, to my mind, vague. Or, at the least, I don't (yet) see how it applies to the wonderful world of Jabber. For instance, in the comments to a post by Adam Bosworth from 2003, RESTafarian Mark Baker said:
REST says a few things, but one of them is not that HTTP is the only protocol to use. What it does say is that the use of other protocols is limited to HTTP-like semantics. What that means, roughly, is that you have to, in effect or in actuality, use an HTTP proxy in front of all other services. For example, this is how browsers use FTP.
Well, it pretty much all boils down to the same thing, doesn't it? I mean, in XMPP we have some HTTP-like (i.e., request-response semantics) semantics, but they don't happen through an HTTP proxy; instead, they happen natively in our protocol (via the <iq/> stanza). Does that make XMPP IQs unRESTful?
And what about semantics other than those familiar from HTTP? In XMPP we have three kinds of semantics:
It seems downright silly to say that it's a good thing to limit the use of other protocols to HTTP-like semantics, given that push semantics have launched not one but two killer apps -- email and IM.
And presence too opens up a whole world of new applications (it forms the bedrock for IM). Presence is one form of pubsub semantics, but not the only one, which is why in the XMPP community we abstracted from our more basic presence functionality to define a generic pubsub protocol. And pubsub semantics seem to be of interest even to folks in the HTTP community -- heck, there are even proposals to do pubsub over HTTP by defining some new HTTP verbs (if those are standardized, are they automatically included in the universe of "HTTP-like semantics"?).
By "RESTful" some folks seem to mean "it's available at a URI". Well, that's nice, but is it everything? Sure, we too have an XMPP URI scheme, but we don't typically use it to express availability of all resources. Does that make XMPP unRESTful?
AFAICS, Adam Bosworth's five questions are still apropos (especially since he explicitly mentions Jabber as a desirable transport protocol). I paraphrase them as follows:
How does REST enable the sender of the message to know reliably that the receiver has received it?
(In XMPP we do this via IQs if you're comfortable with receiving errors on failure but nothing on success, or via advanced message processing if you need greater reliability.)
How does REST correlate responses with sent messages in the absence of HTTP cookies?
(In XMPP we do this via the 'id' attribute on IQ and message stanzas.)
How does REST provide transparent service descriptions in the absence of WSDL?
(In XMPP we have service discovery and we endeavor to define our protocols clearly enough that the programmer's job is made easy.)
How does REST push out context-specific data in real time without requiring the client to maintain a large amount of state?
(In XMPP we can do this with our pubsub extension, although programmers haven't yet tapped into its full power.)
How does REST enable an entity to subscribe to events and receive unsolicted messages when someone publishes something of interest?
(Here again in XMPP we do this with our pubsub extension.)
It strikes me that XMPP satisfies quite a few of the REST principles. It's client-server and stateless (no cookies here) and layered (data is separated from presentation). It has a small number of well-defined operations (IQ get and set, message, presence, pubsub publish and subscribe, data forms of type form and submit) and content types (the various XMPP extension "payloads"). Etc. But we don't make "resources" the center of our universe (I guess you'd say that entities and messages are primary in XMPP), we don't use URIs to identify all "resources", and cacheability is not critically important in our world.
So I don't know that the Jabberites will ever be good RESTafarians. But given that REST seems to be something approaching a religion rather than a set of practical, helpful guidelines for building interesting services, I also don't know that it really matters all that much. :-)
Sorry I've been too busy for blogging much of late. So much to do, so little time. By golly, world domination is hard work! :-)
Among other things, I've been working on the following:
Debugging the recently-launched XMPP Federation certification authority. It seems that we now have most of the kinks worked out, so this evening I will install a new digital certificate for the jabber.org service (I'm just waiting for the number of concurrent users to fall below 5000). Many, many thanks to Eddy Nigg of StartCom (the root CA to our intermediate CA) for his fabulously fast, flexible, and friendly service!
Updating a bunch of specs-in-progress. The list is long (go here and sort by date): Link-Local Messaging, Chat Session Negotiation, Jingle and friends, XMPP Registrar Function (see also draft-saintandre-xmpp-urn), Proposed Stream Feature Improvements, Recommended Order of Stream Feature Negotiation, Best Practices for Message Threads, Entity Time, Delayed Delivery, Best Practices to Prevent JID Mimicking, and probably a few others.
Working behind the scenes on end-to-end encryption technologies for XMPP. More on that soon. But if you're curious, read (or even implement!) the following specs in the following order: XEP-0155: Chat Session Negotiation, XEP-0200: Stanza Encryption (we could use constants for initial values of the various parameters to jumpstart things), and XEP-0116: Encrypted Session Negotiation (if you're really hardcore, also peruse XEP-0188: Cryptographic Design of Encrypted Sessions and the papers referenced from there).
I think 2007 is going to be an extremely good year for Jabber/XMPP technologies -- and an extremely busy year for those of us who work on them... ;-)
One of my minor duties is retrieving lost passwords for users of the jabber.org IM server. The usual process is that the clueless user sends email to stpeter@jabber.org with answers to some questions and responds to the spam challenge I send, I ssh into the server machine to verify the information and then reply via email, and the clueless user receives my emailed reply. Unfortunately, email sucks. By which I mean, I receive so much spam that I needed to institute a challenge-response system (which some users don't seem to understand) and email gets blocked by various ISPs and end-users based on blacklists and spam filters. So for instance one particular clueless user is now irate about the fact that I have not emailed him his password, despite the fact that I sent it several times -- clearly I receive his email but he doesn't receive mine.
If your name is Bernd Voglmeier and you've lost your password, do the right thing by creating a new IM account (they're cheap) and contacting me via Jabber, OK?
Yesterday we launched an intermediate certification authority (ICA) for the Jabber/XMPP network. Here's a short rundown:
Who: The Jabber Software Foundation through its XMPP Federation website, under the auspices of root CA StartCom.
What: The ICA enables us to easily and cheaply issue real, RFC3920-aware digital certificates to administrators of Jabber servers (in fact they'll work for your HTTP server, too).
Why: Easily obtainable digital certificates will result in more widespread use of channel encryption among servers and between users and servers, which will make the Jabber network even more safe and secure than it already is.
Server admins are encouraged to register at xmpp.net, which is the first step in obtaining a certificate. (Eventually we may also issue end-user certificates, too -- stay tuned for details.)
Many thanks to Eddy Nigg of StartCom for working with me in making this happen, to Alaric Dailey of Pengdows for encouragement and beta testing, and to Drupaleers Boris Mann and James Walker of Bryght for their help with the XMPP Foundation website.
At the request of MattJ in the jdev chat room, I've created a machine-readable list of XMPP extension protocols. The XML format should be pretty straightforward. If you want more info in there, let me know.
Back in September, Ralph Meijer and I presented a tutorial (called the "Jabber Bootcamp") at EuroOSCON. I meant to blog this earlier, but our slides are online and in the public domain here. Feel free to re-use and modify as desired!
How is it that I've been working on Jabber stuff for seven years, yet I'm busier than ever? Don't you think perhaps things would've slowed down by now?
But no. I receive so much email I can't keep up with it all. Today I participated in a JSF Board meeting (I still need to write up the minutes done) and a meeting of the XMPP Council, which as usual resulted in a good number of action items for yours truly. Discussion continues apace among the membership regarding my proposal to rename the Jabber Software Foundation to the XMPP Standards Foundation. We're working to get an intermediate certification authority for the Jabber/XMPP network off the ground. Plus there's the usual raft of specs to work on, liasing to be done with the IETF (e.g., regarding XMPP URNs), infrastructure to maintain, conferences to plan, etc. Oh, and did I mention that today the jabber.org IM service went over 200,000 registered users?
Sheesh! It's a veritable whirlwind!
A few months ago, Google launched its developer network, but I have to admit that I haven't looked into it much. It turns out that there are 17 projects that mention XMPP and 33 projects that mention Jabber. Those coders have been busy!
It seems that Zimbra is adding IM and presence to its webmail service. Based on this thread we can conclude that it will be based on XMPP. The Jabber Juggernaut rolls on... :-)
I just finished writing the first draft of an article on Jingle for IEEE MultiMedia. It was helpful to write it all up in one place -- now I can see some places in the specs where things don't quite hang together correctly. Expect further clarifications soon... :-)
Here are some XMPP-related blogs that aren't on Planet Jabber yet:
I'm sure that Ralph will be adding them to Planet Jabber soon. ;-)
My recent proposal to change the name of the Jabber Software Foundation to the XMPP Standards Foundation has been well received so it looks like we will be moving forward on a formal proposal that the membership can vote on. I'll write that up soon (probably next week).
In other proposal news, yesterday I updated my proposal to establish an intermediate certification authority for the Jabber/XMPP network and today I updated my proposal to strengthen trust in Jabber/XMPP technologies (the old version is here).
I'm also proposing (thanks to a poke from Dave Cridland) that we start using Uniform Resource Names (URNs) instead of HTTP-style URIs for XMPP extension namespaces. Details are in draft-saintandre-xmpp-urn.
Feedback is welcome as always.
Bob Wyman asks an interesting question: Does scaling require protocol variants?
In particular:
What works for light load doesn't work for heavy loads and what is good for some heavy loads isn't useful for light loads... What we need here is is two protocols -- or, in some cases, a single protocol that incorporates support for two usage models.
Of course, nobody likes to define multiple protocols to get a single job done. Thus, you're not seeing a rush in the IETF to define "high volume" or "large scale" versions of the many protocols that quite adequately meet the needs of most users today. But, in the future, it might make sense for us to recognize that at some point quantitative differences translate into qualitatively different problems. Perhaps we should start thinking of serving both small and large systems as two different jobs and recognize that while we have many of the protocols we need to address small systems needs, we still don't have what we need to support the larger systems.
As an example from the XMPP world, we might want to design different transport methods for one-to-one or small-group whiteboarding as distinct from the use cases with 10,000 people monitoring the same whiteboard. Similarly for multi-user chat, publish-subscribe, and Jingle. It's worth considering...
BTW, the user-friendly public Jabber server list is now pulling its data from the XMPP Federation database and also shows only those servers that are flagged as supporting open registration via the In-Band Registration protocol. And http://www.jabber.org/servers.xml is up to date as well (it's used by client developers to auto-populate their server registration dropdown boxes). The lists aren't completely user-friendly yet since they could show more information (supported features, included components), but that's coming soon.
Oh and if you want your server listed, simply register at xmpp.net.
I've just posted a message about potentially changing the name of the Jabber Software Foundation to the XMPP Standards Foundation. Follow the link for details.