MSNP16 and SLP-rewrite merged

I have just pushed the revision that merges my MSNP16 and SLP branches to the main development branch in pidgin. I'm very happy to have this branches merged since they represent almost all the code I have been writing on the last year.

Yes I have started coding MSNP16 support almost a year ago and it took a lot of effort, reverse engineering, debugging Wireshark dumps and a lot of pidgin debug logs to get it working. That is a lot of time!

It is true that the MSNP16 code was almost complete when I started my SoC work but I though it would be better to start the SLP rewrite over the MSNP16 branch to be able to easily test both codes at the same time and try to get it in a better shape before merging it to i.p.p.

I know I have announced this merge like two weeks ago, but you know, I wanted this merge to be followed by a reasonable "beta" testing before being released and at that time it got that we had an security issue and needed to release 2.7.4. Once it was out, there were some ICQ issues that needed a quick release to fix that bugs, so we got a 2.7.5. Now I was able to merge and get a normal release cycle to get beta testers to find bugs in this new and nice code.

I hope this code will fix more issues than it brings up, specially the ones related to data transfer. Since most of the code on this area have changed due DirectConn and SLP-rewrite, I guesss it would be a good idea to review and close most of the related tickets since the traceback and debug output would be really useless now. Yei for smashing tickets!

I hope you all like 2.7.6 when it get released!

Use ImageMagick to convert a set of images

While reporting my experiment output from LTSpice I need to save the plots I get as wmf (because it's the only supported image format in this software) then change the format to png to be easily loaded in my latex file.

To achieve this goal I have used ImageMagick. ImageMagick is a really powerful set of tools to manipulate images in command line which allow me to just type a easy command to get my png ready to be used in my tex file.

I have been using the convert command:

convert foo.wmf foo.png

Today I needed to convert a bunch of images so using convert would be a little painful. A quick google show me the answer to my problem: mogrify

With mogrify command you can change the format for every image you tell it in your shell. So to modify all wmf images in a directory now I just need execute:

mogrify -format png *.wmf

And that's it. I hope some of you find it useful.

Merge Plan for MSNP16 and SLP

As you know I have been working on a refactor for the SLP code on the msn-prpl for libpurple as part of my Summer of Code. Before I started this project I have been working on adding support for "Multiple Points Of Presence" which are part of the MSN Protocol 16 (MSNP16).

Since by the time I started the SoC the msnp16 code was almost complete, I started to refactor the SLP module over the msnp16 code. At some time in the process of the refactio I got an ugly crash from one of the features of MSNP16, the P2P version 2 which uses a different binary header for SLP transfers than the one used before. This bug was caused by some clients not paying attention to the capabilities we expose, we clearly say we don't support P2P V2. To avoid this crash I have disabled MSNP16 in the SLP branch.

I have been testing the SLP code for a while with the MSNP16 feature enabled and it looks stable to be merged. The crash has gone, there are some minor changes that must be done, specially UI stuff, the new SLP stack have no known bug.

I have updated the SLP branch with the latest changes from the MSNP16 branch, so this branch [1] have every work waiting to be merged to pidgin's main development branch. This was the first step to get it merge-ready. There have been some testing form some of our closest geeky-friends and now I think it's ready.

My plan is to merge the SLP code with the MSNP16 feature enabled the next week to im.pidgin.pidgin. I ask everyone interested in this features to test it before it get to main branch so we can have a softer merging process.

[1] im.pidgin.soc.2010.msn-tlc

New SLP stack working

Note: This post was wrote 2 months ago and never published, my bad.

After some bug squash, I finally got p2p transfers working. I have tested sending and receiving custom emoticons and display pictures, smallers than the SB limit and bigger than that. They just work ok.

I guess I need to test some file transfers before thinking my work is done, I also have some cleanup that I want to do but I guess I can declare that the new stack is working nice and is very close to be merge ready. I'm really glad to say so.

Implementing new architecture in msn-prpl

Split SLP modules:

The first thing needed was to move SLP code to where it belongs. 

Every high level calls like requesting a msn transfer, an user display and stuff to the upper layer of the stack (slp.[ch]) this was easy because most of the SLP code was there so the file needed a cleanup.

Every SLP protocol decode and management, like sending acks, an 200OK after the invite and stuff, to the SlpCall module, most of that code was on slp.c and some other on slplink.

Every interaction with link layers, splitting the SlpMessage in Parts in SlpLink module.

Most of this changes were mostly cut-paste changes, still a lot of functions needed to be exported as public so the other modules can use them, it was not so painful.

Get MsnMessage out of here!

Since the code was designed to be used to transfer data using the Switchboard, all the code was populated with MsnMessages which is a representation of the MSG SB command.that is used to send the SlpMessages to the SB server. Since I wanted to get this stack agnostic of link layer it was needed to get the MsnMessage out of the way. It was hard since this object was used as the core of some of the functionality, most of the cases it was replaced by a SlpMessage.

This change was really big because it imply get away every SB code  from the SLP stack and abstract it in some way that can be used any of the link layers.

Msn-prpl Slp Refactor

I have been chosen to refactor the SLP module of the MSN-prpl as my Google Summer of Code work. This module have not been touched since ages, if you read it closely it's clear that it's a hack over a hack with some primitive and old structure mostly lost by the hacks that were added over the years.

As the first part of my work I have somehow re-designed the SLP stack as shown in the SlpArchitecture page, this stack is mostly based on the original structure. I will not elaborate a lot here about the modules because that documentation must be done on the wiki page. ;-) Still I want to explain a little the major changes I did to the original stack.

Msn have different ways to do p2p, libpurple now support both, Switchboard and DirectConnection. The first one is the one that have been always supported where we send the binary data to the Switchboard Server and it redirect it to the buddy client. The recently added DirectConnection support does open a socket on both clients so they can communicate "directly" so the transfer is really faster this way.

To be able to use the same SLP code to manage both types of p2p I have added a Conn layer which aims to abstract the Switchboard(SB) or DirectConn(DC) link. They both manages just the SlpMessageParts and send them outside.

Both SB and DC have some data size restriction, when the binary data that needs to be sent is bigger than the link limit, they must be split in different messages. This is the reason of the creation of the SlpMessageParts which is the fragmented representation of a SlpMessage that can be sent to the link layer down to the stack allowing the upper modules to manage a SlpMessage just as single message without noticing if it's big or small, that is abstracting the need to split messages to the upper layers on the stack.

This is basically why I have done the changes in the architecture, I hope this new architecture will be extensible and easy to maintain.

Stream Divx/xvid from linux to your xbox 360

For everyone of us who use ushare to stream video from our linux box to the xbox 360 it is common to see an annoying message telling us that the file could not be played because a bad codec.

I have lived with this for a while but today I was not going to accept MS interfere with my just downloaded torrent. So I researched a little and found a solution that actually works. It can be hacky because you need to change your mime info and maybe it will break something.

The problem is related mime info since it looks like ushare doesn't know what to do with "video/x-msvideo" which is related to avi files in the /usr/share/mime/packages/freedesktop.org.xml mime file. Having noticed this, and knowing that "video/x-ms-wmv" is supported by ushare it is easy to change the unssuported msvideo mime to ms-wmv in the mime file. After editing any file in /usr/share/mime/packages/ you will need to `update-mime-database /usr/share/mime/`to apply your changes before they take effect.

Doing this, ushare send the avi file just as any other and, since an old update, the xbox is able to decode it and play it for you.

Problem: I do not feel ok hacking over freedesktop.xml mime file, I'm not sure if there is another way to tell mime that .avi is an video/x-ms-wmv or maybe  video/x-divx.

I think I will need to look for a better way to get it working but that will need to be later since today, I will procrastinate in favor of my now-possible-to-play-on-my-xbox just downloaded video.

*I found all this info here: https://lists.ubuntu.com/archives/ubuntu-us-nm/2007-December/000368.html
(yeah, I never expected to find something useful in ubuntu archives)