At today’s VMworld keynote, CTO Steve Herrod included a brief demonstration of the project that I have been working on at VMware since last May: the Mobile Virtualization Platform (MVP). One of the downsides of working in industry as opposed to academia is that you have to wait for big release dates such as this before being able to talk about your work, so I’m excited to have the opportunity to blog a little bit about MVP today.

Steve chatted with Jerry Chen, who showed a Nokia N800 running the MVP bare-metal hypervisor which was hosting a Windows CE guest and an Android guest. The relevant part of the keynote, including the N800 loading and launching the Android VM can be found on Vimeo. The demo was also shown in a booth: Senior R&D Director Julia Austin walked through the demo for ITPro who has posted a 3 minute video of the full demo, which I’ve embedded at the end of this post.

Running Android and WinCE on an N800 is pretty cool in and of itself, but our demo also highlights one of the key use cases for mobile virtualization, namely the separation of work and personal settings. With your “work phone” as a virtual machine, it will be possible to easily backup and transfer your work settings from one phone to another, while keeping these settings cleanly partitioned from your personal settings. Similarly, when you upgrade a device to the next generation, you will be able to port your phone over with relatively little hassle.

By providing a stable virtual platform, MVP will also hopefully simplify application development by allowing developers to target MVP instead of a myriad different devices. Defining the virtual platform and mapping it into different physical platforms is an interesting challenge.

Some of the other challenges and benefits of virtualization in the mobile space are discussed in an eWeek article by one of the founders of VirtualLogix, a competitor in this space. At VMware, we are working hard on addressing these challenges with the aim of providing benefits to all stakeholders, from phone OEMs to enterprises to end-users. Hopefully, you’ll be able to see the results of this work in the palm of your hand in the near future!

When a graduate student completes their PhD, the software they wrote for that degree begins an almost inexorable decline into obscurity. Other things become more important and, unless that code serves as a platform for further research or has spun off into a startup, there is precious little time to maintain and further develop code written for the degree. The code has already served a purpose—namely, advancing the state of the art in the form of research papers and a dissertation.

For example, one student I know wanted to test a new Byzantine Fault Tolerant system against an earlier one and was told that the old one only worked on one specific machine in the corner of lab (running some ancient release of RedHat). Another student in my group was trying to compare file system synchronization tools and found that the published source for some tools was hopelessly incompatible with modern compilers and libraries; he wound up finding some old RedHat isos and testing in a virtual machine.

I’m thinking about this today in the context of some recent activity on the Chord mailing list about code I wrote for my PhD. The entirety of my time as a graduate student was spent hacking on the PDOS Chord/DHash implementation, which has served as the reference implementation of Chord in numerous papers. While I hesitate to declare this implementation dead, it has become clear from numerous mailing list posting that it will be increasingly hard for others to use the implementation if it is not more actively maintained.

Chord and DHash are implemented using the libasync C++ library, an obscure and mostly undocumented creation of Prof. David Mazieres, who wrote it as part of his thesis on a Self-Certifying File System (SFS) ten years ago, back when C++ compilers sucked and the STL didn’t work very portably. libasync offers a lot of nice features such as managing TCP buffering for nonblocking I/O and a comprehensive framework for writing user-level NFS servers; this came in handy for early Chord applications such as the Cooperative File System. As a result, the libasync toolkit was adopted by many PDOS research projects.

Unfortunately, as Prof. Mazieres moved on to other research projects, SFS releases and the libasync codebase languished. The libasync library was fortunately adopted by Max Krohn in the form of sfslite. I switched Chord to sfslite, so that it would be possible to point people at a real release instead of asking them to checkout some recent version of SFS from its (now defunct?) CVS server.

As part of sfslite, Max went on to develop tame—a language designed to simplify writing event driven programs with callbacks using libasync, and as part of my thesis, I started to write code in tame to help test it. When sfslite 1.x was released, tame had evolved from its v1 to v2 which changed the syntax. Since I never had the time to take the potential stability hit involved in upgrading, Chord still relies on sfslite 0.8, which Max no longer has time to maintain.

Which brings us back to recent activity on the chord mailing list: as people try to build Chord with ever newer compilers and Linux versions, they run into problems finding a version of sfslite and Chord that build successfully. I still don’t have the time to update the Chord tree to use newer versions of sfslite (though it might not be too hard), nor do I have access to all the resources I used to test it in deployment. So, people are left to patch their own source trees and cobble together working bits.

I wish I had better news for people interested in Chord—there are a lot of useful lessons embedded in our implementation, and many reimplementations that might benefit from those lessons. Perhaps one day soon I will try to write up some of those implementation lessons. Or maybe someone would be willing to take over maintaining Chord. If you volunteer, I’d be happy to help you get started!

The New York Times has an article today about how the inauguration crowd will test cellphone networks. They wrote:

Sprint Nextel, which said it had been planning for the inauguration since April, has also increased capacity of its cell sites and terrestrial transmission lines to prepare the network to sustain 10 to 15 times the number of users it would serve on its networks during a normal day.

Web services (such as Twitter, also cited in the article) also have to deal with scalability and planning to provide adequate (network) capacity. It’s interesting to see how difficult (and expensive!) it can be to deal with flash crowds in other systems.

Most companies keep resources associated with on-going but as-yet-unreleased projects hidden from public view. Working at a big company has made me realize that many tools for organizing various bits of data, that I took almost for granted as a grad student, are not available inside the firewall. Here are just a few:

• delicious, for social information management;
• iGoogle, to allow per-user customizable homepages, organizing info from other services;
• Google Reader, a web-based RSS reader (to aggregate internal RSS feeds);
• Facebook/LinkedIn/Twitter, to develop internal/professional social networks, gaining awareness into what others in the company are working on and what their skills might be;
• QuickTopic, to create quick impromptu archived discussions;
• GMane, a central archive of all internal mailing lists, searchable, linkable, easy to use;
• GitHub, high quality, self-service hosting for source code;
• SlideShare/Scribd et al., for document sharing (instead of mailing around incompatible PPTX and DOCX files) and management.

Some of these services like Yammer or GitHub that allow you to host functionality externally. However, how much do you trust them with your company’s secrets? Can you convince corporate IT to go along with this? If you can’t review the source code, how do you know if it isn’t vulnerable to trivial password guessing attacks?

Open-source clones exist for many of these services but most are not at the level of polish that one can expect from the public services. Still, this can work well, for example with laconica or ReviewBoard.

Similarly, commercial alternatives exist but set a higher bar for internal deployment because money must be allocated to deploy them (in addition to people). On the plus side, commercial alternatives (e.g., ClearSpace) have support. On the minus side, commercial alternatives are generally less amenable to you hacking on the source for your own purposes.

In any event, the space of open-source/commercial alternatives simply doesn’t cover the space of cool web tools that exist today. The AGPL—which copylefts software used to provide services over a network—can help in the future but its impact is still some years off. I hope in time, these tools like the ones I’ve highlighted here are made open-source or otherwise self-hostable because without them, some things at work are just a bit harder than I’d like them to be.

Reading the Wired writeup on the Twitter password hack, I’m incredulous to read that there was no rate limiting on failed authentication. Given Twitter’s stringent rate limiting for API requests, this seems surprising. Not to mention that online password attacks are practically older than time. Fortunately,

As for addressing the security issues that allowed the breach, [Twitter co-founder Biz Stone] wrote in a follow-up e-mail that the company is doing “a full security review on all access points to Twitter. More immediately, we’re strengthening the security surrounding sign-in. We’re also further restricting access to the support tools for added security.”

Thank goodness. I feel safer already.

MoinMoin supports sub-pages, which is a great thing for organizing the lots of wiki-pages. However, the way you write links violates the principle of least surprise: to specify an absolute link, you simply write the target name (e.g., HelpOnLinking), but to specify a relative link to a sub-page, you prepend the sub-page name with a slash (e.g., /Errors). This is of course precisely the opposite of the behavior for standard file systems where absolute paths begin with a slash and relative paths do not.

I don’t know if other wikis exhibit this problem but it wouldn’t be surprising if they did.

After my review of Clipperz and PassPack, I received comments and e-mails suggesting that I consider Mashed Life and LastPass as well.

The most interesting feature of Mashed Life is that it supports logins with a YubiKey–a USB dongle that Mashed Life uses as part of either one or two-factor authentication. This is a very cool feature, as two-factor authentication is harder to beat even than PassPack’s (now potentially weaker) packing key: an attacker must literally have your YubiKey to login as you. But, the downside of the Mashed Life architecture is that it relies on the security of their servers, which, as far as I can tell, have an unencrypted version of all your authentication data–the despite any secret splitting they talk about in their FAQ (Q6), it must be programmatically possible to extract your password since the login action retrieves your password from their server over SSL. Of course, this is probably what enables them to provide an API for synchronization with KeePass and Password Safe, but for me, this is a show stopper.

LastPass

The other day, I complained on Twitter that it was impossible to check-in occasionally to see what people I know had been up to, and separate all that activity out from the prolific people that I’ve followed just because they occasionally say interesting things. And just to prove that fact, I totally missed the two replies to that tweet for several days. One commented that FriendFeed was the solution because of its friend lists. I quickly discovered two things:

1. It is not trivial to find the FriendFeed usernames of people you are following on Twitter. The best solution seems to be from Internet Duct Tape but it is Windows only, not open source, and also doesn’t create imaginary friends for people without FriendFeed. Not useful.
2. You can’t import your friends with private Twitter feeds that require authorization.

Arguably this is just an example of the problem of re-creating the same social networks over and over again. A huge problem of APIs, data formats, trust, … FriendFeed in this case could make it easier by at least recognizing people by their other service profile URLs and hey, even offering Twitter (and FaceBook) import instead of just a few e-mail services.

Somewhat tangentially, I’ve been playing with Yammer at work and they make it very hard to miss a message: compared to Twitter, your home screen auto-updates via AJAX and you can get an daily summary of activity e-mailed to you. Not bad.

Update (22 Jan 2009): Importing your Twitter followings is now a solved problem. Yay! I am definitely pleased with how fast FF is able to improve, as opposed to how slowly core Twitter does (e.g., Twitter search still doesn’t appear on the main web UI page.)

Seen via High Scalability, this seems related to my Chord/DHash work – Ringo: Distributed key/value storage for immutable data. A cursory glance at the Erlang source (a language I don’t actually know), suggests that Ringo does simple successor only routing. I think it uses something like Merkle synchronization trees though there are also comments about potentially switching to Bloom filters (a la Glacier). Also, as of this time, no changes to the core Ringo code since May. For the Erlang DHT aficionados, see also CouchDB, which may be a bit more complete, though with a very different API and possibly (last I checked) less focus on resilience.

On Wednesday (12/17), Collin Jackson will be giving a talk at MIT titled, ”Extracting Passwords from JavaScript Password Managers”. I can’t go due to scheduling conflicts but it seems worth considering if you are local and interested in my post about PassPack and Clipperz. I didn’t see any obvious papers from Collin’s website about the attacks he is describing but there are some that look worth reading.