Caching autotools outputs for fun and profit

GNU make parallell building feature is cool. And ccache which speeds up rebuilds of same files is cool too. But even with this great tools rebuilding some project is still not as fast as it can be. Why ? Because preparing GNU autotools files (./configure & friends) takes quite long time. And there is no way to parallelize it.

But we can cache autotools products just as ccache caches compiler products. And now there is tool that's capable of doing that! You can grab it here. I've adapted fabricate.py for this job.

Just put fabricate.py into PATH and prepend it to your command. Like:

fabricate.py ./bootstrap

or

fabricate.py sh -c './bootstrap && ./configure'

And it'll copy command outputs from cache if all dependencies are same. Nearly instantaneous!

Welcome to 21th century, Aliaksey!

I'm used to slow, expensive & unreliable Internet. Because of that I tended to keep as much as possible on my local hard drive. I had tons of specs, articles, e-books. And I even kept local mirror of entire Debian unstable repository! That's because at home and until recently at Altoros office I had quite limited and expensive Internet access. I tried (successfully) to reduce my dependency on Internet as mush as possible.

Around 2 weeks ago I accidentally deleted most of my 'stuff'. And now I'm forced to use online resources. And quite surprisingly that's not bad experience at all! Internet here, at Bay Area, is fast enough. And googling for information and reading it online sometimes seems to be even faster than locating and opening it from local disk.

This is unusual and new experience. When I, for example, need some information on how something is done in python I just ask Google. And within mere seconds I have precisely what I need! I've even started reading PDF articles online via Chrome's plugin without downloading them first.

I'm slowly getting used to 21th century Internet. I'm still not quite 100% comfortable with depending on Internet availability. And some lookups are definitely much faster against local data. For example, I don't think anything can be faster than local supermegadoc documentation lookup of some function. But I'm going to stop downloading interesting stuff and will instead bookmark it. Let's see how it'll go. I'm especially curious how it will work when I'll return home. Hopefully, we'll have speedy ADSL connection by then :)

How to use two monitors in IntelliJ IDEA

I'm working on some educational Scala project. And IDEA is natural choice for that. Normally I use Emacs for my work, but for anything Java related it's not as good as IDEA. And I cannot express how powerful and polished this nearly free software IDE is.

However, I was missing one very useful feature of Emacs, which is ability to open several frames to effectively use two monitors. Until few minutes ago.

When you want to open same "buffer" on two monitors you can first split it horizontally (C-x 2 if you use Emacs key bindings) and then you can simply drag one of the windows out of it's frame to second monitor! If you want different files you can just drag tabs out of main frame.

How to testament... childs' death

Often Unix processes spawn childs. Contrary to the popular belief childs DO NOT automatically die when their parent exits.

Unix job control has a concept of sessions and session leaders. Session leader is the process that 'owns' terminal, and when this guy dies, each process of session gets SIGHUP. This usually leads to death. Every time you open new tab in your terminal emulator, you're creating new session. And bash (or other shell) is leader of this session. So your childs will likely die when terminal emulator tab will be closed, but they will easily outlive your process. And they will live forever when your main process is spawned from cron or your favorite continuous integration tool.

Of course the whole topic of Unix job control is slightly more complex. You can read more by reading GNU libc manual (or aptitude install glibc-doc && info libc). Posix man pages (aptitude install manpages-posix{,-dev}) contain lots of details, much more than usual Linux man pages.

So one of the ways to ensure your childs die is by using sessions. This is usually implemented by creating pseudo terminal pair (PTY) and giving slave side of that pair to child. Session leader (i.e. child) gets SIGHUP when master side of PTY pair is closed. Master side is closed when your main process exits (if you don't pass it's fd to child by mistake). When child dies, this causes delivery of SIGHUP to whole session. {spawn,fork}pty-like functions are used for that.

And there's another even less widely known way. There is a concept of orphaned process group that can be used to ensure SIGHUP delivery without relying on PTYs. I'll refer to posix man page for details. But basically you fork and deliver SIGSTOP to child. When your process group becomes orphaned (e.g. when main process dies), each member will be sent SIGCONT & SIGHUP. Added benefit of this approach is that if parent of your main process dies, your process and it's child will die too. This relies on main process or it's parent being process group leader, but that usually holds.

I'm using this to control a bunch of erlang VMs when running Membase in development mode. You can grab python code for doing that here. Ruby code for doing same is removed, but you can dig it's grave.

UPDATE: unfortunately this trick doesn't always work in OSX/Darwin due to different definition of orphaned process group in OSX: http://developer.apple.com/library/mac/#documentation/darwin/reference/manpages/man2/intro.2.html

Two envelopes sophism puzzle

I'm currently on vacation. And it allows me to return to some small topics that I postponed earlier. One of them is Two envelopes problems. This is quite interesting and difficult sophism to solve and 'get'. This morning I'm not in 100% work-ready mood so I decided to return to my list of 'Later'. Surprisingly, this time it took quite small amount of time to solve this puzzle. Actually, I'm a bit puzzled why I postponed it earlier. Maybe I should take vacations more seriously ?

I'm copying definition from wikipedia:

The basic setup: You are presented with two indistinguishable envelopes containing some money. You are further informed that one of the envelopes contains twice as much money as the other. You may select any one of the envelopes and you will receive the money in the selected envelope. When you have selected one of the envelopes at random but not yet opened it, you get the opportunity to take the other envelope instead. Should you switch to the other envelope?

The switching argument: One line of reasoning proceeds as follows:

1. I denote by A the amount in my selected envelope.
2. The probability that A is the smaller amount is 1/2, and that it is the larger amount is also 1/2.
3. The other envelope may contain either 2A or A/2.
4. If A is the smaller amount the other envelope contains 2A.
5. If A is the larger amount the other envelope contains A/2.
6. Thus the other envelope contains 2A with probability 1/2 and A/2 with probability 1/2.
7. So the expected value of the money in the other envelope is
   
   
   
8. This is greater than A, so I gain on average by switching.
9. After the switch, I can denote that content by B and reason in exactly the same manner as above.
10. I will conclude that the most rational thing to do is to swap back again.
11. To be rational, I will thus end up swapping envelopes indefinitely.
12. As it seems more rational to open just any envelope than to swap indefinitely, we have a contradiction.

The puzzle: The puzzle is to find the flaw in the very compelling line of reasoning above.

IMO the most easily understandable flaw in that reasoning is that it adds values of A from two possible realities. And those values obviously differ. If you analyze this problem correctly then it's obvious, that switching envelopes does not make sense. So the lesson is: be careful with your variables.

Reflections on Ruby vs. Python meetup

This is cross-posted from Altoros' developers blog. I want to thank Altoros PR department for editing this text.

I never understood developers and customers that willingly choose Python over Ruby. Ruby does feel like a native language to me, while everything else doesn’t. No, this doesn’t mean that I don’t like working with other technologies. Over the last year and a half, I was really inspired by my positive experience of using Erlang and JavaScript, while working on the Membase project (where Python is widely utilized). I really enjoyed doing my work, so it’s obviously possible to have fun with languages that are less well designed than Ruby.

However, I always wanted to understand why this happens. Why do people choose other languages than Ruby so often? That is why I was excited to accept the invitation to the “Ruby vs. Python” meetup that was sponsored by our company. I chose the topic “Why Ruby Is a Brilliantly Designed Language” and wanted to summarize what makes Ruby different.

The Holy War

The event was very dynamic and intriguing. The speakers and attendees were split into two groups: one defended Python, while another fought for Ruby. Each group prepared presentations describing history, community, main concepts, development features, and perspectives of their language of choice.

I was a part of the group that presented Ruby. It was my first public speech in many years. My previous presentation—the defense of the diploma—was a big failure. I’m glad that this time I’ve learned some lessons and wasn’t so bad. 

So, why is Ruby a brilliantly designed language?

With hindsight, it is obvious that I picked up a very hard topic. Squeezing my experience into a 15-minute speech was really tricky. While preparing for the session, I reinforced my belief that Ruby is the best designed programming language I’ve ever used. However, when it was just one day left before the meetup, I realized that I could bring in only the most essential arguments to prove that.

1. My first pro-Ruby argument was that, from the moment the language was created, there were no missing features to add and no design bugs to fix. With over 10 years of its history, Ruby remained—mostly—stable with only minor syntactic features added/tweaked.
2. Second, Ruby represents all powerful object-oriented programming (OOP) features, which it adopted from Smalltalk. It is a “pure” OOP language, where everything is an object. The pureness, open classes, method_missing, and other OOP features provide for a huge semantic power. Surprisingly, nothing game-changing was introduced to the programming language design over the last 30 years.
3. After that, I talked about what powerful features Ruby adds on top of the traditional form of OOP. For instance, mix-ins, a powerful alternative to multiple inheritance is a great invention. During the presentation, I demonstrated how it makes functional programming even more natural than in LISP. Another remarkable Ruby feature worth mentioning was the ability to extend individual objects. It really makes defining “static” methods natural and can be useful in some other cases.
4. I concluded with touching upon Ruby’s concise syntax. As a part of this argument, I demonstrated Ruby’s syntactical killer-feature – blocks.

Revealed pros and cons

There were some other nice discussions during and in between the speeches. For example, the Python group argued that its language is more widely used. However, I noted that one could resort to such argument only because of the lack of other worthy pro-language arguments.

Another controversial point was that Python is good for education and is adopted by Massachusetts Institute of Technology as the primary programming language for students. I objected that Python is not the best language to start with. I think it is wiser to begin developing your programming skills with a pure OOP language, such as Ruby, because it can build a better solid ground for object-oriented design and way of thinking.

The Ruby group was also presented by Aliaksandr Rahalevich, Altoros alumni, who spoke about how Ruby allows him to design great APIs, and Hleb Stiblo, who presented an “Introduction to Ruby.” At the end, my university classmate Pavel Gabriel called upon pro-Python and pro-Ruby folks to summarize their arguments and language trivia.

The discussion ended up with the confessions from both sides about what sucks in their language of choice. Python 2.6/3 incompatibility was the biggest issue that Pythonistas admitted. Ruby developers noted the lack of speed, the lack of support for real concurrency, and cumbersome multiple encodings support in Ruby 1.9. The first two negative points were well understood by Pythonistas, since they suffer from them, as well. Ruby’s incompatibility of 1.9 vs. 1.8 was also mentioned, although nobody said that the difference between these versions is far less than the difference between Python 2.6 and Python 3.

I really enjoyed attending this event and making a presentation there. However, it seems like everyone—including me—forgot to mention that it is hard work of human beings and their attention to details which mostly contributes to an end-result. I think that as long as the technology doesn’t actively interfere with the programmer’s job, it affects the end-result very slightly. The language is merely the tool, while the engineer is the master.

How to use time travel for testing javascript libraries

The project that I'm working on right now is Membase. It is and was awesome experience in many ways. One cool part of that project is that I'm allowed to do quite advanced stuff for membase UI. Part of that is Cells library that is growing as part of Membase UI effort. But this post is not about cells. I promise to write about cells later. This post is about mocking JS clock for testing cells.

The code is here. You're interested in files named: mockTheClock.js and wrapped-date.js. It is inspired by equivalent code in JSUnit, but it's more full featured and more efficient. In particular I'm using binary heap to order pending timers.

The code is Apache-licensed, so feel free to grab and re-use it for your own projects.