Hell Oh Entropy!

There have been a number of code changes in libyahoo2 since some time now, with Gennady Feldman and later me pitching in. So it was finally time to do a release. My primary motivation for this was to remove libyahoo2 code base from ayttm and have it link dynamically to libyahoo2. I hope Ray van Dolson packages libyahoo2 for Fedora soon, so that I can pester Minto to package ayttm with that patch too.

The YMSG16 support changes in 1.0.0 will break things for many people using libyahoo2, but that was necessary. I wanted to avoid tying libyahoo2 to a specific crypto library for SSL use and also reduce the overhead of maintaining that code base. The result of this change is that it removed all of the networking code from libyahoo2 as well, which reduced maintenance overhead even more.

Another big change was the removal of the ability to bypass the callback struct and link functions directly. It would make things difficult when you're trying to find out in your configure script if the library is installed. One would have to write a test program in the autoconf, which would have to have all the callbacks implemented, to be able to compile successfully. That is painful. Also, The approach is much more extensible compared to earlier the approach.

There are a number of things that we still need to get done in libyahoo2:

• Chat room support
• Webcam support— I am not sure if it is broken since I don't have a webcam to test, but I want to make sure anyway
• Are mail notifications working?
• Voice support

I will be submitting a Fedora Summer Coding idea for libyahoo2 soon to add chat room support as a mentor. The major skills required for this would be C and the ability to read and understand wireshark dumps.

Recently someone asked me if there was a way, with gdb, to find out for which signal a particular signal handler was called. The catch was that there were no debugging symbols, but the binary was not stripped either. That way I could set breakpoints and do some general debugging on the flow of the program but not view variable values directly. So the backtrace does not show the signal number that was passed into the signal handler and I would have to do some magic to get at that value. So here's a little demo to explain what I did:

So I had a program that looked like this:

#include <signal.h>
#include <stdio.h>

void foo(void)
{
    printf ("that's all folks!\n");
}

void handler(int sig)
{
    foo();
}

int main()
{
    signal(SIGIO, handler);
    while(1);
}

I built the program, started it under gdb and set a break point at foo(). I then ran it and from another terminal, did a killall -IO a.out. here's what I got:

(gdb) bt
#0  0x00000000004004dc in foo ()
#1  0x00000000004004f8 in handler ()
#2  <signal handler called>
#3  0x000000000040050d in main ()

Notice that the argument value for the handler() function is not shown, which is our problem. This is because there is no way that gdb would know anything about the arguments without any debug symbols, i.e. the number of arguments, the size of each argument, etc. But we do. A signal handler has only one argument and it is an integer. It is the signal number.

To go any further, we need to check out the stack frame of handler(). As you can see in the stacktrace above, it is frame 1, so here's what we get:

(gdb) info frame 1
Stack frame at 0x7fff84277c30:
rip = 0x4004f8 in handler; saved rip 0x301e2301b0
called by frame at 0x7fff84277c38, caller of frame at 0x7fff84277c10
Arglist at 0x7fff84277c20, args:
Locals at 0x7fff84277c20, Previous frame's sp is 0x7fff84277c30
Saved registers:
  rbp at 0x7fff84277c20, rip at 0x7fff84277c28

So we know now that the arglist starts at 0x7fff84277c20, so the signal number argument should be in this region. Since it is a 32 bit value, we do some basic math and go down the stack by 4 bytes to see what word is stored at 0x7fff84277c1c:

(gdb) x/1 0x7fff84277c1c
0x7fff84277c1c:    0x0000001d

Do some more college math and convert 0x0000001d to decimal and we get 29. Lookup signum.h (search for it in /usr/include) and you'll see that it is SIGIO, the signal I sent in my kill. W00t!!

One may use this method to also get local variables of that function. It is just that we will have to do some guessing to find out the sequence of the locals in the stack if one does not have access to the source code. It should not be that hard though. This may also be one of the bases for those try to use buffer overflows to overwrite some value in the stack and hence modify the flow of a program. I'm yet to figure out how one could overwrite stuff far enough to drop from the program right into a shell. Learning computer science bit by bit now after wasting 6+ years of formal schooling in it :)

My first package submission was finally completed a few days ago with everything from ticket overload, personal issues to my US trip making sure that I had little time to get things done on time. I had started this in the first week of December and it took all of 2 months and some days to get my stuff together and put out an rpm package for gource. It is finally done now and you can install gource on your Fedora 12 box (or later if you're nutty like me) with just a

~# yum install gource

I've built gource only for F-12 and later since F-11 does not have the required version of ftgl. That is OK I guess since F-11 is way too old anyway ;)

I've been in the US for a week now. I've reached Raleigh, the final destination in my short trip in the US. I'll be here for another week before I go back /home. I was in Westford last week and in Boston over the weekend.

I was in Westford to get trained to support Red Hat MRG Messaging. The upstream for this is the Apache QPid Project. This seems to be a very promising product, especially with its speed of messaging. The training was fun and I got to meet some really cool people here, including fellow GSS geeks Ryan Mitchell and Andrew Blum. I got a chance to interact with the qpid hackers too, which was really cool. I attempted to dig a bit into the code to understand some random stuff, but haven't made much headway since.

I left for Boston on Friday and spent the weekend there. I only had enough time to visit the Museum of Science and the Aquarium. We managed time pretty badly though; we could have squeezed in some more. I met up with Philip for lunch today at an interesting place called the Toro in South End, Boston. You gotta go there to try the bone marrow tapas. It was awesome.

Waiting now to get to the mother ship, the GSS headquarters tomorrow; a chance to meet people who I work with across the oceans and across time zones.

Warning: This is a bit of a rant, so read on only if you’re in a bad mood and want someone to cuss at (either me or the entity I’m cussing at ;) )

I had recently bought email hosting services for my Mom from Net4 following decent service on the domain and web hosting front. It was a stupid decision. Net4 support for email pretty much sucks. Maybe I have only been lucky with the web hosting support too as I did not have to deal with morons for the most part. For the last two days I haven’t been so lucky.

I noticed the problem soon after the email hosting got activated — emails that I would send to any gmail accounts would go into the spam folder. I ignored it in the beginning mainly because I wasn't interested enough to look hard for what is wrong. But recently mom started complaining about it since her customers were not getting her emails. So we opened a ticket with Net4. For good measure, I also called so that I get immediate feedback. I was told to attach a sample email that went into spam. I did that. The support person then closed the ticket the next day asking for a sample email. Interesting.

So I open a support ticket once again and this time I try to do some research on why gmail may be doing this. I noticed the following header:

Received-SPF: neutral (google.com: xx.yy.zz.aa is neither permitted nor denied by best guess record for domain of mom@example.com) client-ip=xx.yy.zz.aa;

So I look up google and come across this article on SPF records. Apparently, net4 ought to be adding an SPF entry for my domain associating it with their mail servers. This entry will lead to the Received-SPF header being "pass" instead of "neutral". This is something that gmail takes into consideration while filtering for spam. While I realize that there may be other matters (email content, subject, etc.), all those factors do not come into the picture with my sample email. This seems to be the only missing link since my domain, mail server, etc. are not blacklisted either. I mention it in the support ticket and again attach the sample email. The ticket gets closed again with a request for a sample email. WTF?

I call up and ask the support tech. I try explaining about SPF and how that might be affecting things and he goes all ballistic (not figuratively, he's positively shouting) on me saying that this is on Linux and it does not work the way I am saying. With my ego hurt, I shouted back saying that I work for a Linux company :P

We end the call with the tech telling me that I am only throwing keywords around without knowing anything about it. He asks me how I would fix this and I admit that I don't know right away but I can do his job and look it up. He then bullshits me about how it is "the pop3 end which marks the spam with either SPAM tag in subject or moving to a spam folder" and so on. According to him SPF is something different and there is nothing wrong with it. He ended it saying that I was being an ass and that he will escalate the issue.

No, there is no hope. The last two issues were escalated too. Apparently everyone at Net4 is just a bunch of incompetent fools. Either that or those who aren't are eyeing management positions since that is the "way up the ladder". Or they're on their way out.

I think I need to be on my way out of this service. I may not get a refund this time but I can surely make sure that I don't renew my services with them and definitely not recommend their services to people I care about.

I finally got myself to start working on the pjpdns stuff and the cdb vs tinycdb question resolved itself pretty soon as far as pjpdns is concerned. Some functions and their arguments (cdb_findnext, cdb_findstart, etc.) have changed in tinycdb due to which pjpdns would not link against it as is. I would have to patch it to get it working.

Of course, being a lazy bum, I took the easy route and linked against my fork of cdb, which worked. Here is my fork of pjpdns with the changes. There are still some things that can be trimmed out in pjpdns as a result of this. There's also the entire "unlicensed code" thing I need to sort out since a lot of code in the original cdb does not have any license.

There are also a bunch of other changes I made to pjpdns, which included removing some auto-generated cruft from the repository and adding a bit of autotools candy so that any future autotools related changes don't result in git-log spam.

PJP had given a presentation some weeks ago on what would probably end up being pjpdns. One of the problems pointed on on his review request for inclusion of djbdns into Fedora was that it included code for cdb, which exists as a separate package and hence should be dynamically linked to instead. Someone even pointed out that (s)he had autotoolized cdb but did not respond to any further requests for more information.

I figured it was something I could do, so I went ahead and ported cdb to gnu-autotools. Here is my cdb fork. There is still some code which is unused, but I haven't really bothered to update in much since the time I found tinycdb, which seems to be maintained too. So my next project is to try and get djbdns to either use tinycdb or if that fails, my cdb fork.

I wonder how I can license the cdb fork though (if I decide to distribute it at some point). A lot of code is unlicensed, which is why (according to wikipedia) the tinycdb fork was written in the first place.

Oh, and let it be recorded officially and publicly that if pjpdns goes live, it was me who misread djbdns during the beginning of the presentation as pjpdns. Yes, that is my desperate attempt at being relevant to something completely unrelated :D

I have finally got myself an account on github. My first fork (obviously) is ayttm. Here I'll be putting in all the major destructive changes I cannot put into the main repository right away since even I'm not sure if it is a good idea in the first place. The first commit is the revamped conversation stuff. Note that chat_room.* is out and conversation.* is in. And a lot of code has changed. And a lot of things have broken.

The last day of FOSS.in. Kartik wakes me up saying "You have to pack, get up". I got up, went across the hall and rang Ram²'s bell a few times to wake him up.

So I packed up my stuff and got ready. This time I did not forget to wear my fedora. We had our breakfast and realized that a number of people had left early, either for the venue or to meet someone in the city. When we went out to our bus, we found that it was not there. We started talking to rickshaw drivers and bargaining with them when one guy came up to us saying that he was here to take some people from the hotel to NIMHANS. That was us. But we weren't sure, so we had him call his boss and confirmed that it was indeed us.

Susmit: "Well, if he wanted to drop us to the venue then why are we wasting time calling people? Why not just pile into the bus and go ;)".

So we finally reached the venue, only to find that it was almost deserted. People started crawling in quite late and the place eventually filled up. I sat down with Kital at the Fedora booth and fixed my package submission for Fedora. At 12:00 I went up to Lennart's FSH talk, but did not find it as useful as the previous talk. He could just have combined this one with the last talk and done one marathon talk, but I guess that would have been too long. Anyway, it was time for lunch.

I was pretty listless after lunch because there wasn't enough time to hack on anything significant, so I decided to take a nap at the booth. I woke up to find that I had only 20 minutes to say good bye to all the people I came to know during these 5 days. Wait, a phone call. An automated message telling me that my flight had been delayed by half an hour. Well, I had some more time now, so I went up to the hack centre to check out who was doing what. I bugged Gopal for the Speaker kit and he finally shooed me off with some stickers. I told him that the coordinators had done a great job, something I managed to notice this time since I was not sitting in the corner and hacking like last year.

While I was roaming about, I entered the Gnome POTD and saw Oliviere Crete giving a talk on Telepathy. I sat down and listened. And I have to admit that I was quite impressed with the layout. I remember wondering why I was working on getting ayttm and libyahoo2 up to speed when such a wonderful effort was underway to do messaging the *right* way. Now that I think of it though, it is not the only right way to do things. While ayttm may not be as extensible as the Telepathy framework+Empathy, etc, it has a totally different use case. To be able to fit into a sub-1 MB package and work well with minimal dependencies. And it can work on *BSD. The workout participation for ayttm unfortunately fizzled out since it was pulled off the itinerary and it is not popular or cool enough for people to want to contribute to it. But I managed to make some headway into the chat window stuff and will hopefully get that out of the way soon. Next week I plan to work on the webcam stuff for yahoo at Rahul's request. OK, I digressed.

I had to leave the talk halfway since I was getting late. I stepped out of the hall and managed to say goodbye to everyone I recognized and could catch hold of. I managed to find Pradeepto just as I hauled my bags up to leave. We said our goodbyes and I left for the gate, trying to find a rickshaw. But what I found was a bus, a BIAS-7 going to the airport :)

So I'm now sitting in the airport lounge reflecting back on one of the best events I've attended so far. This was very special for me because I got to know a lot of very good guys: Pradeepto, Aanjhan, Sayamindu, Joerg, Gopal, Susmit and so many others. The best part of the week was the BoF sessions all of us would have in Pradeepto's room, talking about foobar and sharing stories. I did not have too many uber hacks to talk about through this week, but a lot of cool incidents that seemed to make a big difference to the way I thought about the Indian FOSS movement and the people involved.

This was fun. I hope I get to do this more often.

The penultimate day of FOSS.in/2009. And me and Kartik got up even later than usual. We managed to get ready quickly and reach on time for breakfast, only to realize that practically everyone else was late. We left for the venue (eventually) at about 10:30. Philip had already arrived at the venue and he asked me about the ayttm workout, which was to happen today. I thought it was cancelled, but we decided to go for it anyway. So we spent the time before lunch in the hack centre with Kartik and Ram ² going about their Debian workout and Philip scp'ing his latex presentation back and forth on his server since he was stuck with his Mac.

I came up to the hack centre to work on ayttm but saw Russell Nelson teaching a bunch of people how to solder stuff. This got me excited since I remembered my my childhood, with me trying to make some weird stuff with wires and pins (yes, a couple of them exploded too :P) and trying to "repair" stuff, eventually breaking them. So I had to get up and see what he was doing. That way, I would be putting the soldering iron my wife recently got me, to constructive use.

Russell explained some basics of soldering and then went on to show a small example of attaching an LED and resistor to a small microcontroller board. I stuck around to see how soldering was done and then finally decided to skip the software part and get back to ayttm.

We sat down to hack again after lunch, but Philip vanished after a while, probably to prepare for his talk. I continued trying to get some stuff together while we also battled with a relatively unreliable wireless connection. The organisers finally pulled down the wireless in the hack centre since they were overloaded. I spent the rest of the time mucking about with nothing in particular. All the while, it was quite entertaining listening to Rahul Sundaram, Kartik and Ram² discussing Debian and Fedora release cycles.

Finally it was time for Philip's talk. While I got up to go down to the hall, we were joined by a bearded guy who came up to talk to Ram². I casually looked at his tag and realized that he was Baishampayan Ghose. I have only probably conversed with him or read his emails on mailing lists or heard about him, so meeting him in person quite suddenly was a surprise, which kinda left me a bit tongue-tied ;). He recognised me instantly (probably due to the tag) and became the first person other than Philip and Kartik to ask me about ayttm/libyahoo2 :). We then walked down to the hall to attend Philip's "Shut up and Hack" talk.

I thoroughly enjoyed the talk, mainly due to the spontaneity. He basically talked about reasons for which hackers hack. Not just software hackers, he talked about hackers in general. He then went on to make some useless and some other useful little snippets of code to explain how one could hack. The point was to not call a committee to decide when, where and how to scratch an itch; just scratch it now.

Bluesmoon was followed by Blues before Sunrise, which was strictly OK, because of which I walked out after the first few songs. We had a fun time with Himanshu and Kartik finally getting the better of Rahul Sundaram in pulling his leg. Pradeepto was lying down in a corner and soon after, everyone was gathered up around him and were chatting away. Gopal came up to chat too and once again managed to call me the "bakra who Philip caught to work on ayttm" :P. He had said exactly the same thing at last years FOSS.in. It turns out that Philip had asked Gopal if he could contribute to ayttm, which Gopal was able to conveniently turn down since he soon joined Yahoo! :P

All this was followed by a speakers dinner at the hotel, which was quite a lot of fun too. Kushal had 7 bowls of soup till the time I was hanging around and was probably looking to have more. There were occasional shouts of "PulseAudio sucks!", "PulseAudio rocks!", "KDE sucks" and "Lenny rocks!" just to pull each others legs. It was quite cool to watch a lot of people get a little drunk on beer and having fun in general. Me, Kartik, Aanjhan, Ram² and Pradeepto then came down to Pradeepto's room and were chatting about foobar right up to about half an hour ago. Now here I am sitting in the hotel room with Kartik fast asleep. I'll go to sleep now.

This has turned out to be a long post. Yes, it was quite a long day.