ProFUSION | Aggregator

ANNOUNCE: kmod 2

2011-12-21T00:00:00Z

I’m glad to announce the second version of kmod. I’m sorry for not sending the first version to the mailing lists. Now I’m both writing it here and sending to the mailing list. I thank very much the feedback received for the first version and that now Jon Masters, the maintainer of module-init-tools, is helping us with kmod [...]

kmod announcement (and how to help testing it!)

2011-12-21T00:00:00Z

introduction and motivation

At ProFUSION a common topic is how to optimze the system. Not just the speed, but also what and how it is done. Not just for our embedded systems, but also our desktops and laptops.

These discussions led us to be involved in projects that consider this goal, such as Enlightenment, ConnMan, oFono and systemd. With great projects, come great people and thus enlightening discussions on how to improve things. From these discussions Lennart Poettering and Kay Sievers put together a A Plumber’s Wish List for Linux, with one of the items being of special interest for ProFUSION as it had the potential of also helping embedded systems and speeding up or boots:

* module-init-tools: provide a proper libmodprobe.so from
module-init-tools:
Early boot tools, installers, driver install disks want to
access information about available modules to optimize
bootup handling.

One of the reasons for this is that udev will search /sys/devices for all “modalias” files and call “/sbin/modprobe -bv $ALIAS“, however many of these calls evaluate to an empty list, thus an useless execution of program that will open /lib/modules/`uname -r`/modules.*, load resources, search for something that is not there and return. This could be way cheaper if done inside udev workers. [note: Lucas did measure and noticed 2.5x speedups, stay tuned for benchmarks]

kmod

Then our developer Lucas De Marchi proposed to do the libmodprobe and we funded it. After some discussions with Lennart and Kay, it was decided to rename it to kmod for shortness and good namespace. The initial goal was to achive a 1:1 replacement of module-init-tools as a library to be used by udev and systemd-modules-load and we are close to it with our second release done today! While we miss depmod tool (planned for v3), our kmod-modprobe should be fully functional and if this proves to be true, the logic will move to libkmod to be used by udev.

we need your testing!

To ensure kmod-modprobe does what it is supposed to do, we need extensive testing, there is where we need you! Try it on as many systems as possibles and let us know. To do this we recommend:

$ git clone git://git.profusion.mobi/kmod.git
$ cd kmod
$ ./autogen.sh
$ ./configure --enable-logging \
              --enable-debug --enable-tools \
              --enable-zlib # if you have module.ko.gz
$ make all
$ sudo make install
$ sudo mv /sbin/modprobe /sbin/modprobe-bkp
$ sudo ln -s /usr/bin/kmod-modprobe /sbin/modprobe
$ ls -1 /sys/module/ > ~/original-modules
$ reboot
$ ls -1 /sys/module/ > ~/kmod-modules
$ diff ~/original-modules ~/kmod-modules

current users and feedback

Being a new project in a critical area of Linux system, we expected lots of criticism and rejection by people, but interestingly enough after Lucas’ announcement and LWN article the feedback was highly positive! We even had some testers and people to help with ideas and experience.

Among the people that joined the project is the current maintainer of module-init-tools package Jon Masters, which announced that kmod replaces module-init-tools. He is helping a lot with his knowledge and cases from Red Hat Enterprise Linux, also providing modules from non-trivial platforms such as s390. Last but not least he provided insights to improve module handling on Linux, particularly replacing modules with better alternative as required in enterprise systems.

With Jon’s blessing we’ll use the same mailing list linux-modules@vger.kernel.org and our git will move to kernel.org soon.

People from ARCH and Debian were also interested and even created packages for it! These guys were extremely helpful to test cases such as gzip modules, 64bits modules with i386 user space and so on. They are: Tom Gundersen (tomegun), Dave Reisner (falconindy) and Marco d’Itri (Md).

next steps

Based on our TODO, we have the following ideas for next steps:

Finish libkmod-elf.c to provide information required by depmod. I’m working on this in my depmod branch;
tools/kmod-depmod.c: create a 1:1 compatible tool to generate /lib/modules/`uname -r`/modules.*;
libkmod should export parsed configuration to be used by kmod-modprobe.c –dumpconfig;
create kmodmock library to be LD_PRELOAD’ed to redirect some syscalls such as init_module(), delete_module() and open() of /sys and /proc. I’ve started a branch for this some time ago as “unittest” branch but stopped due lack of ELF support at the time;
create unittests and measure coverage. Given kernel modules are user-input they may be broken files and libkmod-elf.c needs to be extensively tested to avoid segmentation fault due out-of-boundaries access. This mean not trusting null terminated string in .strtab section and so on.
create optimized modules.archive that would contain optimized search index and all modules compressed independently, but in the same file. This would save file access and could provide information we don’t have today, like the uncompressed size of gzipped modules.

Given enough eyeballs, all bugs are shallow

2011-12-16T00:00:00Z

So, in last post I said kmod 2 could be released sooner than expected if there were major bugs. Not as much as a surprise, there was 1: depending on the alias passed to the lookup function we were blocked iterating a list. It’s now fixed in git tree. Thanks to Ulisses Furquim for fixing [...]

ANNOUNCE: kmod 1

2011-12-15T00:00:00Z

For some weeks now I and Gustavo Barbieri at ProFUSION have been working on a new library and a set of tools, libkmod and kmod respectively. This is the announcement of its first public release. Overview The goal of the new library libkmod is to offer to other programs the needed flexibility and fine grained [...]

LinuxCon Brazil presentation

2011-12-07T00:00:00Z

Recently I attended at LinuxCon Brazil, did a presentation and still had the chance to attend to some interesting talks.

The presentation was given together with Bruno Dilly, about EFL focused on embedded devices. Besides some previous similar talks about EFL, this one was more interesting and quite different, since we didn’t talk much about how to write a program using it, but instead focused on its advantages, presented some real use cases and tried to show off where one can take more advantages from these libraries.

You can see the presentation slides online on Bruno Dilly’s post, or get the slides here.

Overall the event was very interesting, with some nice keynotes and presentations. Two of them that draw my attention were Eugeni Dodonov’s talk about the Linux Graphical Stack, and Daniel Frye’s keynote about the 10+ years of Linux at IBM, which exposed how IBM’s LTC (Linux Technology Center) started trying to contribute with Linux, what they did wrong and how they fixed it.

In addition to our talk, 2 more guys from ProFUSION were presenting their talks: Lucas DeMarchi did a talk about how to become an open source developer and Gustavo Barbieri presented two talks: “Tips and Tricks to Develop Software for CE product on Low-End Hardware” and “Demystifying HTML5” with Sulamita Garcia (Intel).

That’s all for now, and hopefully there will be more presentations coming on next events.