Comments on: The afterlife of systems research code http://www.emilsit.net/blog/archives/the-afterlife-of-systems-research-code/ Fri, 13 Aug 2010 22:58:01 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Trackback Spam Resources » Blog Archive » Hibernation. http://www.emilsit.net/blog/archives/the-afterlife-of-systems-research-code/comment-page-1/#comment-27059 Trackback Spam Resources » Blog Archive » Hibernation. Tue, 03 Mar 2009 16:25:32 +0000 http://www.emilsit.net/blog/?p=160#comment-27059 <p>[...] A common fate for research code. Unless there’s (1) new work to be done or (2) someone else picks up the [...]</p> [...] A common fate for research code. Unless there’s (1) new work to be done or (2) someone else picks up the [...]

]]> By: Emil Sit http://www.emilsit.net/blog/archives/the-afterlife-of-systems-research-code/comment-page-1/#comment-27055 Emil Sit Mon, 16 Feb 2009 14:49:23 +0000 http://www.emilsit.net/blog/?p=160#comment-27055 <p>VMs are a great way to let people test out a system, though reproducibly building good VMware appliances is still a somewhat difficult task, as far as I can tell. There are some tools like <a href="https://help.ubuntu.com/community/JeOSVMBuilder" rel="nofollow">the Ubuntu VM Builder</a> or <a href="http://www.vmware.com/download/sdk/studio.html" rel="nofollow">VMware Studio</a> but they are very new. <a href="http://pdos.csail.mit.edu/~baford/vxa/" rel="nofollow">Bryan Ford's Virtual Executable Archives</a> are a lighter weight example. Chord only speaks SunRPC so it isn't very Web2.0-plug-n-play but yeah, some sort of VM image might be a good way to make it easier for researchers to use as a basis for comparison and to have a guaranteed functional build system to hack on. Now then, how to make sure it runs on EC2, Emulab, PlanetLab, VMware and qemu? And sadly, I probably won't have time to do this either.</p> VMs are a great way to let people test out a system, though reproducibly building good VMware appliances is still a somewhat difficult task, as far as I can tell. There are some tools like the Ubuntu VM Builder or VMware Studio but they are very new. Bryan Ford’s Virtual Executable Archives are a lighter weight example. Chord only speaks SunRPC so it isn’t very Web2.0-plug-n-play but yeah, some sort of VM image might be a good way to make it easier for researchers to use as a basis for comparison and to have a guaranteed functional build system to hack on. Now then, how to make sure it runs on EC2, Emulab, PlanetLab, VMware and qemu? And sadly, I probably won’t have time to do this either.

]]> By: David Molnar http://www.emilsit.net/blog/archives/the-afterlife-of-systems-research-code/comment-page-1/#comment-27054 David Molnar Mon, 16 Feb 2009 06:30:05 +0000 http://www.emilsit.net/blog/?p=160#comment-27054 <p>Thanks for the insightful post. I am also facing this issue. I've spent a fair amount of time hacking on the code that will underpin my dissertation, and I don't want to see that work go to waste. At the same time, keeping code in a working state as compilers, glibc, etc. advances is next to impossible.</p> <p>The cleanest solution I've found so far is to publish VM images with everything required to build the code (plus the project already installed). This at least gives people something they can download and play with. For example if you go to http://www.metafuzz.com we provide a link to a VMWare image and instructions for installing. We also have Amazon EC2/Xen images, though they are not publicized at all yet (just use them for running experiments).</p> <p>One downside of this is that creating these VMs for me is kind of heavyweight. As a result, the VM image lags behind the current working version of the code. This is a problem because people can then form opinions about the project based on old code without all the latest improvements, bug fixes, etc. For me this is an OK tradeoff, at least for now. </p> <p>I note the asbestos project (also from David Mazieres and friends) has a qemu image as part of its build process. So whenever someone makes changes, they can build a redistributable image of the new OS. That's pretty neat. Ideally such an image would include all the tools+code required to make a new such image; I don't know whether the asbestos people have that or not. </p> <p>A second downside is that my work needs to test other software to be useful. I've already found cases, like the vlc media player, where the latest and greatest just won't play nice with my VM due to requiring new glibc, new sound libraries, or whatever. I don't have a good solution for this in general.</p> <p>Maybe that is less of an issue with a Chord implementation? To do an experiment with a new twist on the basic Chord protocol, would it be sufficient to have a bunch of Amazon EC2 images running old distros of Linux with the tweaked software? If so, you could partially address the bit-rot issue by publishing a basic "Chord Node" Amazon EC2 image which others could rebundle, using it as a basis for their own new Chord implementations.</p> Thanks for the insightful post. I am also facing this issue. I’ve spent a fair amount of time hacking on the code that will underpin my dissertation, and I don’t want to see that work go to waste. At the same time, keeping code in a working state as compilers, glibc, etc. advances is next to impossible.

The cleanest solution I’ve found so far is to publish VM images with everything required to build the code (plus the project already installed). This at least gives people something they can download and play with. For example if you go to http://www.metafuzz.com we provide a link to a VMWare image and instructions for installing. We also have Amazon EC2/Xen images, though they are not publicized at all yet (just use them for running experiments).

One downside of this is that creating these VMs for me is kind of heavyweight. As a result, the VM image lags behind the current working version of the code. This is a problem because people can then form opinions about the project based on old code without all the latest improvements, bug fixes, etc. For me this is an OK tradeoff, at least for now.

I note the asbestos project (also from David Mazieres and friends) has a qemu image as part of its build process. So whenever someone makes changes, they can build a redistributable image of the new OS. That’s pretty neat. Ideally such an image would include all the tools+code required to make a new such image; I don’t know whether the asbestos people have that or not.

A second downside is that my work needs to test other software to be useful. I’ve already found cases, like the vlc media player, where the latest and greatest just won’t play nice with my VM due to requiring new glibc, new sound libraries, or whatever. I don’t have a good solution for this in general.

Maybe that is less of an issue with a Chord implementation? To do an experiment with a new twist on the basic Chord protocol, would it be sufficient to have a bunch of Amazon EC2 images running old distros of Linux with the tweaked software? If so, you could partially address the bit-rot issue by publishing a basic “Chord Node” Amazon EC2 image which others could rebundle, using it as a basis for their own new Chord implementations.

]]>