While browsing through kde-look.org I found a cool idea for visualizing a daily agenda.

This reminds me of the Akonadi architecture diagram and I even have code (probably not up to date) for drawing this kind of diagrams. So I guess it would be doable without too much effort to implement a daily agenda viewer like this. Would be a fun project.

A while back I blogged about querying dbpedia with sparql. The queries in that blog were pretty simple. Today, I present a more complicated example.

You could also look for stuff near Gran Canaria or Berlin.

I noticed that four of the book lists that I have near the top left corner of my main blog page, have acquired bit-rot. Most of the images of the book covers were no longer being found.

Now I don't know whether this is a result of Amazon changing their API's earlier this year, or the cache URL's are just no longer valid, but I thought I'd fix them up. Rather than taking the URL that Amazon supplies when you search for a particular ISBN programmatically, I'm now using the "standard" medium size image URL that seems to nicely work with most books at Amazon with a recent publication date.

The four lists are:

• 1001 Books You Must Read Before You Die.

• Pulitzer Award Winning Novels.

• 100 Favorite Mysteries of the 20th Century.

• Salon.com Reader's Guide to Contemporary Authors.

I'm one of those people that finds it easier to remember an unfamiliar book by its cover than its title. That doesn't always help as they seem to need to change the art-work for the same book with each new release. It did enable me to find a few of them at the Los Altos library book sale last Friday evening.

I also highly recommend the Salon.com Reader's Guide to Contemporary Authors. The list only gives the suggested book to start with for each of the authors, but the actual book goes much further than that with mini biographies and bibliographies plus various essays and digressions under a multitude of topics.

[Technorati Tag: Books]

[Technorati Tag: Lists]

> Read More... | Digg This!

Since I now have a true SMP system, it’s obvious that I’m expected to run parallel make to make use of this. Indeed, I set in my make.conf to use -j8 where I was using -j1 before. This has a few problems in general and it’s going to take some more work to be properly supported.

But before I start to get to those problems, I’d like to provide a public, extended answer to a friend of mine who asked me earlier today why I’m not using Portage 2.2’s parallel emerge feature.

Well, first of all, parallel emerge is helpful on SMP system during a first install, a world rebuild (which is actually what I’m doing now) or in a long update after some time spent offline; it is of little help when doing daily upgrades, or when installing a new package.

The reason is that you can effectively only merge in parallel packages that are independent of each other. And this is not so easy to ensure, to avoid breaking stuff, I’m sure portage is taking the safe route and rather serialise instead of risking brokenness. But even this, expects the dependency tree to be complete. You won’t find it complete because packages building with GCC are not going to depend on it. The system package set is not going to be put in the DEPEND variables of each ebuilds, as it is, and this opens the proverbial vase to a huge amount of problems, in my view. (Now you can also look up an earlier proposal of mine, and see if it had sense then already).

When doing a world rebuild, or a long-due update, you’re most likely going to find long chains that can be put in parallel, which I sincerely find risky, but they don’t have to be. When installing a new package, on a system that is already well installed and worked on for a few weeks even, you’ll be lucky (or unlucky) to find two or three chains at all. If you’re doing daily updates, finding parallel chains is also unlikely, as the big updates (gnome, kde, …) are usually interdependent.

Although it’s a nice feature to have, I don’t find it’s going to help a lot on the long run, I think parallel make is the thing that is going to make a difference in the medium term.

Okay, so what are the problems with using -j8 for ebuilds then?

• we express the preference in term of (GNU) make parameters, but not all packages in Portage are built with make, let alone GNU’s;
• ebuilds that use a non-make-compatible build system will try to parse the MAKEOPTS variable to find out the number of parallel jobs to execute; this does not always work right because there can be other options, like -s (which I use) that might make parsing difficult;
• even -s option can be useful to some non-make-compatible build systems, but having to translate every option is tremendously pointless and boring;
• some people use a high number of jobs because they have multiple box building as a cluster, using distcc or icecream; these won’t help with linking though, or with non-compile jobs; forcing non-compile tasks to a single job is going to discontent people using SMP systems, using a job count based on network hosts for non-compile tasks is going to slow down people with single-cpu and multi-host setups;
• some tasks are being serialised by ebuilds when the could be ran in parallel;

And this is not yet taking into consideration buildsystem-specific problems!

What should be doing then? Well, I think the first point to solve is the way we express the preferences. Instead of expressing it in term of raw parameters to make, we should express it in term of number of jobs, and of features. For instance, a future version of portage might have a make.conf like this:

BUILD_LOCAL_JOBS="8"
BUILD_NETWORK_JOBS="12"

BUILD_OPTIONS="ccache icecream silent"

And then an ebuild would call, rather than simply emake, two new scripts: elmake and enmake (local and network), which would expand to the right number of jobs, for make-compatible buildsystems, that is. For other build systems eclasses could deal with that by getting the number of jobs and the features from there.

More options might be translated this way without having to parse the make syntax in each ebuild, or in each eclass. The ebuilds could also declare a global or per-phase limit to jobs, or a RESTRICT="parallel-make", that would make Portage use a single job.

The last point is probably the most complex one. Robin already dealt with a similar issues in the .bdf to .pcf translation of fonts, and solved it by having a new package provide a Makefile with the translation rules, the conversion could then be parallelised by make, instead of being serialised by the ebuild. I think we should do something like this in quite a few cases; the first one I can think of is the elisp compile for emacs extensions, and I don’t know whether Python serialises or execute in parallel the bytecode generation when providing multiple files to py_compile. And this is just looking at two eclasses I know doing something similar to this. But also Portage’s stripping and compressing of files should probably be parallelised, where there are enough resources to do so locally.

I guess I have found yet another task I’ll spend my time on, especially once I’m back from the hospital.

Systems Administration is a lonely job. There is very little support for professionals and we're all very divided. There is, I find, a common desire to know what "the other guy" is doing or thinking, what tools they use, how they arrange their day, how they deal with stress, how they select one gig over another, how they provide for a wife and family in it all.

A podcast is a great way to do this, but there isn' t much out there. The closest was Kevin Devin's "From the Trenches", which appears to be dead, and always focused on the Windows/Cisco network segment which is frankly uninteresting to me.

Enter SAPro...

This new podcast will be interview and round-table focused, unlike my previous podcast(s) which were all in the "one guy with a mic" format. Even I didn't want to listen to them unless I was really bored.

I'm organizing several "admin roundtables" for diverse discussion and starting to line up several interviews with industry super-stars. If you want to participate please let me know. I am particularly interested in interviewing vendor backline support engineers, if you are one please contact me.

Keep your eyes open, I plan to roll out episodes soon.

While I’m writing this, Yamato is up and running a world rebuild to updat to the new CFLAGS. It also spares me from having to run a further rebuild to get stuff updated. I’ll write more tomorrow about Yamato, the problems getting it to run, my idiocy on a little detail, and the changes in software I made for this incarnation of my workstation.

As I now have access to my main data repository again, I’ve also started a boring and long task that I should have done a long time ago. I started looking through all the bookmarks I had in Konqueror (and there are lots), and tag them on delicious (has it been renamed Yahoo! Bookmarks yet or is it soon to come? The fact that Firefox extension revers so often to “ybookmarks” makes me wonder).

My reason to use delicious is that I have a huge amount of bookmarks saved, and I have to organise them somehow; tags are better suited than folders for this, as it makes it possible for me to remember that Dave’s Trailer Page is, yes, a page related to Movies, but it also involves my multimedia work as it contains important samples.

Thanks to the extension, also, I can easily access all the bookmarks, from Yamato (Linux) or Intrepid (Mac OS X/Windows), which makes it quite important when one of the boxes is inaccessible, or if I have to find something and I’m not at home, but rather at a friend of mine’s (as long as I decided to share the bookmark).

So if you’re monitoring my delicious page or my friendfeed, you’ll most likely see a new wave of pages being tagged down, it’s not something I think I’ll be doing for a long time, hopefully.

On the other hand if you have been following me a few weeks ago you might have seen I tagged down a huge amount of medical-related pages on Wikipedia; I was transferring to delicious the list of pages I bookmarked last year while I was at the hospital, integrating it with the new stuff they gave me this year.

I suppose that if you’re monitoring me, you might now know more about me than you’d ever have wanted to.

Some time ago I expressed my feelings about C# ; to sum them up, I think it’s a nice language, by itself. It’s near enough C to be understandable by most developers who ever worked with that or C++ and it’s much saner than C++ in my opinion.

But I haven’t said much about Mono, even though I’ve been running GNOME for a while now and of course I’ve been using F-spot and, as Tante suggested, gnome-do.

I’ve been thinking about writing something about this since he also posted about Mono, but I think today is the best day of all, as there has been some interesting news in Java land.

While I do see that Mono has improved hugely since I last tried it (for Beagle), I do still have some reserves against Mono/.NET when compared with Java.

The reason for this is not that I think Mono cannot improve or that Java is technically superior, it’s more that I’m glad Sun finally covered the Java crap. OpenJDK was a very good step further, as it opened most of the important parts of the source code for others. But it also became more interesting in the last few days.

First, Sun accepted the FreeBSD port of their JDK into OpenJDK (which is a very good thing for the Gentoo/FreeBSD project!), and then a Darwin port was merged in OpenJDK. Lovely, Sun is taking the right steps to come out of the crap!

In particular, the porters project is something I would have liked to get involved in, if it wasn’t for last year’s health disaster.

In general, I think Java has now much more chances to become the true high-level multiplatform language and environment, over C# and Mono. This because the main implementation is open, rather than having one (or more) open implementations trying to track down the first and main implementation.

But I’d be seriously interested on a C# compiler that didn’t need Mono runtime, kinda like Vala.

My previous post touched on the horrid FPS you can expect from any graphics intensive Qt app on the N810 at the moment. Ariya has pointed out one reason for the bad numbers: the overhead of having all Qt paint operations on a 32 bit backingstore and then converting to 16 bit when blitted to the N810's screen.

Well, here is a workaround I found while we wait for some pretty big changes in the Qt painting engine for 4.5 that Ariya hinted at. Mind you it is a big hack, but it seems to work well for my use case and perhaps it would be useful to others.

Introducing QX11RasterWidget:

This week is hackweek at SUSE and people are frantically hacking on all kind of stuff. Fun.

My project is the Social Desktop, which is the buzzwordy title for an implementation of the Open Collaboration Services API (see specification on freedesktop.org). Frank Karlitschek has joined the fun and is at the SUSE offices for hackweek, so server and client implementations go hand in hand. The idea is to bring the community to the desktop and take benefit of the fact that free software projects are not only about software but also about community. This can provide a lot of extra value for our users, especially as the desktop is the place where all the social web data from different sites comes together and the user is in full control of what happens to the data and how it is combined. For some more background have a look at Frank's Akademy keynote.

As a first result I have now implemented a client which accesses opendesktop.org through the Open Collaboraton Services API and makes its users available on the desktop. Next steps are searching for people and enabling communication.

For more info and progress updates have a look at my hackweek blog or the occasional tweet.

Chris mentions JavaScript embedding using SpiderMonkey.

When GNOME first started out, there was much talk of using Guile as an embedded language. The concept was similar to the architecture of Emacs, Firefox, or all the games using Lua (such as World of Warcraft).

I would define an "embedded" language as one that doesn't come with a platform. It's small and easy to glue together with an app written in C or C++. It uses the same platform as the app. If the platform has an introspection capability like XPCOM or GObject-Introspection, then the platform can be exported to the embedded language in an automated way.

At some point, and I was one of the people to blame for this, we changed the focus of the Linux language discussion to grand unified schemes involving component technologies and virtual machines. The result is that too much desktop stuff is still written in C.

A "C/C++ core plus embeddable language" architecture for an application has many pragmatic advantages.

• Higher layers of code such as UI code are so much easier in a dynamic "scripting" language.
• Proven to be practical and scalable for huge apps such as Firefox and World of Warcraft, even with dead-simple languages such as JavaScript and Lua.
• No platform silos. In a GNOME app, the embeddable language uses GIO, GTK+, GStreamer, Clutter, or whatever you are using - it does not have its own http stack and everything else.
• You can choose to write each part of the app in the easiest language; some stuff is easier in C/C++.
• Sandboxable / well-modularized. The interpreter is generally an object that can be destroyed; no heavy language runtime or VM takes over the entire app.
• Embeddable languages don't seem to create a huge jump in memory footprint, for whatever reason, so they are still sensible for "swarm of many processes" environments like GNOME and KDE that multiply any per-process overhead by a significant factor.

(There are advantages to the heavier full-platform languages also, but I'm guessing everyone knows them already so I won't make a list.)

JavaScript is a great choice for an embedded language. I did not realize at first that SpiderMonkey has no dependencies; it is not tied up with the rest of the Mozilla platform (XPCOM, etc.), it's completely standalone. And most programmers understand at least some JavaScript already.

There are some problems right now, for example there's no standard packaging and .pc file for jsapi.h/libmozjs.so, so it's hard to make an app build on multiple distributions unless you suck in an internal copy of the library.

It's necessary to add a couple things to JavaScript before it's truly usable; most notably, some kind of module system. I'm not sure SpiderMonkey itself should have that though - it starts to define a platform and make the language "not embedded". Maybe modules are introduced by higher layers (in fact xulrunner adds a module system if you use the whole xulrunner platform).

I don't mean this as a SpiderMonkey advocacy post, though we're using it successfully at work. Other options are the WebKit JavaScript engine, Guile, Lua, Tcl, etc.

My main point: the "C/C++ core plus embedded scripting" architecture is a very smart way to write an app, and doesn't have some of the real downsides of heavier platforms.