Thoughts on Systems

Emil Sit

How to Use the Git Protocol Through a HTTP CONNECT Proxy

Many corporate firewalls prevent git from using its efficient binary protocol by blocking outbound network connections. Sometimes, you are lucky and are trying to clone a repository that is hosted on a site like github which exports their repositories over HTTP, which would enable you to get through the firewall using the http_proxy environment variable. However, you are usually not that lucky and are only given a git:// URL to clone from.

Fortunately, most corporate firewalls allow for tunneling connections through their HTTP proxies, using HTTP CONNECT. This is normally used for allowing browser to connect to secure websites (using SSL over port 443), but if you are lucky, you can have your firewall administrator configure the proxy to also allow CONNECT for port 9418, which is the port used by git.

Once they have appropriately configured the proxy, you should then be able to use tools like netcat-openbsd or socat to connect through as follows…

  1. Install `socat`. For example, on Debian/Ubuntu, just `sudo apt-get install socat`.
  2. Create a script called `gitproxy` in your bin directory; You will need to replace proxy.yourcompany.com with the name of your proxy host and the port with the port used by the proxy (common ports include 3128, 8123 and 8000). (If the javascript is broken, visit [Gist 49288][gist] and download the raw file; or use the original commands as reproduced in the comments.)
  3. Configure `git` to use it:
    $ git config --global core.gitproxy gitproxy
    

That’s it! Your git clone commands should now transparently accept git:// URLs.

Update: Fix quoting and add link to Gist 49288.

How to Securely Delete Files Before Returning a Computer

A friend of mine who is switching jobs has asked how to delete all personal information from the company computer before returning it. Simply deleting files can prevent the casual observer from finding your data, but file deletion typically does not (to make an analogy to paper) shred the file but merely throws away the information that tells you how to find it. A sufficiently dedicated person could indeed search through every possible place on your computer’s drive and re-construct your deleted files.

In brief, the best tool I am aware of to prevent this is Darik’s Boot and Nuke (dban). This software goes on a floppy or bootable CD image that you create and then completely overwrites every place any file could have ever been stored with garbage data. This is the ideal solution if your employer has a standard software image that they will use after you return your computer.

If you need to be a bit more precise (e.g., your company really needs that one Windows license and can’t re-install), you may need the Heidi Eraser which runs in Windows. This uses the same secure wipe methods as dban, but can be applied to “empty” space on your drive. Thus you can manually delete your user account and wipe any windows cache directories (e.g., everything in C:\Documents and Settings\YourUsername) and then run Eraser as the administrator user to clean off any bits. Heidi Eraser includes dban so you need only download one

Secure deletion takes a long time (many hours!) so be sure to budget sufficient time to do this.

The Difference Between Clipperz and PassPack

Clipperz and PassPack are two web services that store your passwords for you, and provide one (or two) click login to those sites whose passwords it keeps. In doing so, these services hopefully encourage you to select strong passwords that you wouldn’t otherwise be able to remember and thus improve your overall security. Aiming to be the one basket in which you keep all your eggs, both services work very hard to provide you with adequate security.

There are clearly technical differences in the services. PassPack has more user visible phishing protection; Clipperz doesn’t transmit your password anywhere providing mutual authentication using SRP. PassPack requires much longer passphrases; Clipperz has a better (though somewhat harder to set up) system for automatically logging into remote sites. PassPack autolocks your data, though it’s not clear what that does in a technical sense (e.g., does it scrub memory?) But both only store encrypted data and have your browser do all the crypto, which is (for me) a major attraction.

I find some more interesting differences in their terms of service. For their protection, both PassPack and Clipperz use standard lines such as: “You expressly understand and agree that Clipperz shall not be liable to you for any direct, indirect, incidental, special, consequential of exemplary damages […] resulting from unauthorized access to or alteration of your transmissions or data” (taken from Clipperz’s Terms of Service). That is, you are allowed to store access to your million dollar retirement assets but please don’t hold Clipperz responsible if somehow your use of Clipperz results in those assets being stolen. However, PassPack’s User Agreement has an interesting caveat:

If your Account is provided to you without charge, you will not use the Service to handle financial data, information and credentials for accessing bank accounts or services provided by financial institutions, including brokerage services;

It’s not clear to me what additional CYA this is supposed to provide. It’s impossible by design for them to verify compliance and it should be a user choice whether to trust PassPack with encrypted credentials to their financial data. To me, it seems more secure than say mint.com.

Clipperz and PassPack differ in whether you are allowed to audit their service. Clipperz:

Clipperz grants you a license to use the source code related to the Service for the sole purposes of performing security reviews of the code or enhancing the interoperability of other products and services with the Service.

whereas, PassPack:

You are prohibited from violating or attempting to violate the security of this Site, including, without limitation, (i) accessing data not intended for you or logging into a server or account that you are not authorized to access; (ii) attempting to probe, scan or test the vulnerability of a system or network or to breach security or authentication measures without proper authorization; […]

Clipperz’s terms in this regard are the most reasonable I think I’ve ever seen.

The other main difference is that PassPack is a commercial service so you put your trust in them by establishing a contract. As the Digital Railroad failure fiasco shows, that’s no guarantee. Clipperz is currently not a paid service but is open source; you trust their goodwill and can always roll your own in case of emergency. On the other hand, PassPack has revenue, which means they can continue to innovate whereas Clipperz needs investors or volunteers.

It’s a tough choice. I’m leaning towards Clipperz philosophically but there is a lot to like about PassPack as well.

Strobist Meets Niekerk

Continuing in the photography/lighting vein… Neil van Niekerk’s techniques are geared towards the time-limited on-the-go (wedding) shooter; David Hobby tends to prefer carefully constructed off-camera light. He’s been exploring on-axis fill recently and his ”run and gun” post is where I’d like to go next with lighting: using a single off-camera light with on-camera light, and using TTL for speed. Too bad Canon built-in flashes can’t act as TTL masters; that’s a nice plus for Nikon users.

Flash Photography With Neil Van Niekerk

Shooting with natural light can produce a beautiful portrait, but in many real life shooting conditions photographer-added light can improve the quality of the result. In his flash photography workshops, such as the one I took last week in Boston, Neil van Niekerk explains the technical knowledge you need to produce the artistic look you may be seeking and then works hands-on with you to put that knowledge into practice. The techniques that he teaches grew from his experiences working in studios, applied to on-the-go wedding photography: practical, fast, effective.

Neil favors soft directional light that ideally looks as if flash was not used (motivated light). To achieve this, Neil teaches you to deliberately control the ambient light by setting your exposure manually and then to bring in directional light, either by bouncing an on-camera flash off a nearby surface or by using off-camera lights. He reviews the theory of controlling flash exposure manually (aperture, ISO, power, distance) and automatically (flash exposure compensation, leaving shutter speed, aperture and ISO transparent). With diagrams and live demonstrations, he explains flash technicalities like second curtain sync, high speed sync and the magic of the maximum sync speed.

In the afternoon, we worked with two extremely patient models Tanja and Melissa and went through a number of practical exercises including using the histogram to meter, bounced fill flash and FEC to control contrast, off-camera lighting in bright sunlight, use of gels to control color temperature, indoor bounce flash with modifiers, and a few more. In warmer climates, Neil takes the class downtown to work but things were a bit chilly in Boston so we spent the evening shooting in-doors, working more with off-camera manual flash.

One technical note: Neil recommended that we use evaluative (or matrix) metering instead of spot. He recommends this in order to be sure that the TTL flash metering will correctly set the power for the desired exposure of the subject; he suggests zooming in to a white patch (e.g., of the wedding dress) to correctly set the ambient exposure based on the histogram. Since I use primes mostly, I would prefer to be able to leave the camera in spot metering. Fortunately, the Photonotes Flash Photography with Canon guide seems to suggest that I can:

E-TTL II […] examines each evaluative metering zone before and after the E-TTL preflash. It then calculates the weighting for each zone independently, biasing against those zones with high reflectivity in the preflash. This means that E-TTL II does not have a flash metering pattern as such, since it’s calculated dynamically.

This seems consistent with my usage on the 30D so I will stick with spot metering for now. (Nikon users, pipe up in the comments.)

Neil’s work is meeting a growing demand to incorporate sophisticated flash techniques into photography for better results. He encourages an understanding of the full manual approach and off-camera aspects (a la strobist), but doesn’t shy away from using the intelligence programmed into today’s TTL flashes. He’s willing to answer questions from complete beginners, but the workshop is best (I think) for people comfortable with their camera controls and basic flash usage, and wanting a taste of the more sophisticated. Neil’s workshop is definitely recommended: I came away more confident in my ability to get results using flash having better internalized the fundamentals and gotten some great practical tips. For more pictures from the shoot, check out my photostream on Flickr.

Securing the Web Browser.

Google is soon going to demonstrate Google Chrome: a ground-up re-written browser designed with security in mind. Wow, render each tab in a separate process (and more). Compare that to what we saw at SOSP 2007: MSR presented some improvements to deal with IFRAME within MSIE (MashupOS (PDF)) and how to track Ajax performance in a distributed way (Ajaxscope), and researchers at Cornell described a new programming language that allows for static partitioning of data (Swift (PDF)).

Wuala: Buy or Trade P2p Storage

Wuala uses erasure codes and crypto over a p2p network (backed by managed servers) to provide “social” storage. The crypto builds on work by Kevin Fu and others. One nice idea is that you can trade local storage for remote storage. I wonder how good the latency is when reconstructing data fragmented across distributed peers.

Opis

Pierre-Evariste Dagand developed Opis, an OCaml-based framework for developing distributed systems. It includes yet another Chord implementation, tested in ModelNet against Macedon and MIT Chord. While I still think OCaml is kind of ugly, it appeals to me more than P2 or Macedon did. (via Anarchaia)