Hugo’s blog

I’ve just realized that Flickr is now available in multiple languages. It looks like the days guiding my family through what I can best describe as pages full of words they have no clue about are finally over. Woohoo!

It’s going to take some time to get used to, but the new interface of Flickr seems really cool.

Editing a batch of photos is now so much easier! Kudos to the Flickr team.

I wanted to add a lot of pictures (an entire set, in fact) to a private group in Flickr. In the Flickr forums, I found a reference to febstools for that, but it didn’t work quite right for me, and it comes as a Windows executable.

So I put together Mass-add-to-group in PHP that others may find useful: it basically does what febs’ tools does, but you don’t need to install anything.

After my issues with photos having the wrong taken date, I quickly put together a tool to edit this information from the command line using the Flickr API. It took me 30 minutes to fix the dates of 150 pictures with it.

It’s nothing fancy: DateFix.py allows to to set the taken date of a set of photos given their ids. The ids of the photos showing problems are easily gotten from the calendar view, using the “edit as a batch” tool.

It looks like this:

DateFix.py -i 79818469@N00 -d 2001-09-02 121729217 121729192

I have been uploading old photos to Flickr, in order to have them all easily searchable by keywords, etc. However, I was reminded that a few years ago, I was not doing a great job at manipulating the photos metadata, and that quite a few did not have their EXIF data anymore.

And I have discovered that Flickr decides that, when there is no metadata in the photo, the date the photo was taken is the date that the photo was uploaded. I am not sure when this approximation is beneficial, especially these days when I expect cameras and tools to deal with metadata well; anyway, it certainly is not the case here. I would have preferred that this field be left blank.

ExifTool has helped me set the DateTimeOriginal and DateTimeDigitized in the photos that did not have one, and now Flickr reads the information from this data that I manually entered.

However, as it took me a while to notice this, I am left with hundreds of photos whose taken dates are wrong, and the interface to change those is painful, as AFAICT, it needs to be done on each photo individually. Maybe I’ll have to upload them again, though finding the originals again and uploading just the ones that were missing this information without duplicuting the others is going to be tricky.

Let this be a warning to others: check the EXIF of your photos before uploading them!

This is me waking up 1.5 months after everybody.

I worked at 3Com and lived for about 1 year in the Hemel Hempstead area, home of the magic roundabout, and I am finally realizing today while playing with Flickr that the Buncefield fuel depot which exploded last month in the UK was right in front next to where I was working. I even drove in front of it every day! Shame on my British memories.

Flickr has some photos of the damage that the explosions did to the building. Flickr is a really cool tool. Computer World even has an article about it.

I wrote my first Flickr desktop application, Offlickr, but something has been bothering me in the way the authentication is done.

The Flickr Authentication API Desktop Applications How-To explains how user authentication works. Basically, an application has an API key which will appear as an api_key parameter, and a shared secret which is used to sign every call, the signature being included as a api_sig parameter. If a call needs to be authenticated, an authentication token will be used (which is given to you by Flickr when you as a user authorize an application to access your data) that you include as an auth_token, and again, the application signs the whole thing with the shared secret.

A closer look at the authentication procedure

So the security relies on the api_sig signature, i.e. on the shared secret. Only the application with the shared secret can generate the right signature to do a call.

However, replay attacks are easy: everything is contained in the query parameters. Just do the HTTP request again. This is probably why people use HTTP POST instead of HTTP GET even for things like getList: to try to hide what’s going on and not have the authentication token show up in a log or in a Web cache.

In addition, one can get somebody’s authentication token somewhere (e.g. in a Web proxy log file), and then could reuse the same application (after all, the api_key identifies the application) and pretend to be this somebody. But this limits the attacker to using one particular application, as the so-called shared secret prevents you from doing arbitrary operations as you need it to sign the HTTP requests.

This is where another problem lies: the shared secret is not so secret; it doesn’t appear on the wire, but if you have an open-source application, you can easily find it in the source. If the code is compiled, maybe the developer can hide it from the public, but I was writing a program in Python. I spent some time wondering if one should or not include the secret in the code, but looking at other programs, it seems that the answer is that shared secrets are always included as you don’t want everybody to register the program with Flickr before they start using it.

So, if you can get a hold of somebody’s authentication token for say an uploader tool for which you can get the shared secret (e.g. because you have the source code), as the uploader will have read-write access, then you have full read-write access on somebody’s account. And again, the authentication token is shown in the clear on the wire, so you just need to go to a conference and snoop on the WiFi network to wait for somebody to use the uploader for example, and bingo.

The real secret here, since you already have the API key and the shared secret of the application, is therefore the authentication token that the user has gotten from Flickr. This is a piece of information that only the user which authorized the application has. It stays a secret until it gets on the wire as a query parameter. Well, it’s currently returned on a standard HTTP response so one could technically get a hold of this when the token is issued, but that’s a minor problem as it could easily be changed by having flickr.auth.getToken use SSL.

The question is therefore: why send the authentication token on the wire then? As it seems to be the real secret, it should be kept hidden.

I am assuming that Flickr doesn’t want to use SSL as it would be expensive CPU-wise. It seems that a stronger authentication technique could be used fairly easily and would improve security greatly.

One way to keep this authentication token secret would be to use it to sign the call, i.e. have a parameter auth_sig akin api_sig to hold a signature generated with the authentication token.

Using standards

It may seem that I’m excessively picking on the Flickr API, when actually this is a general HTTP problem. After all, basic authentication in HTTP is weak, and the current practice of using cookies is not better; capturing just a little unencrypted HTTP traffic is guaranteed to give you access to something you shouldn’t have. Plus I like Flickr, which is why I started playing with it and looking at the API in detail; I’m sure similar comments could be made of others. Qui aime bien châtie bien as one would say in French.
However, it seems that, with an API, we have a greater power and therefore can achieve better solutions as we’re not limited to the solutions supported by the intersection of what the major browsers implement.

A way other than the auth_sig solution I was talking about to solve this problem, could be to use the authentication token obtained from Flickr as the password in HTTP digest authentication (RFC2617) and it would even be standard so people wouldn’t need to do much in their code.

I admit I’m a little surprized that with all the standardization work happening on the Web, ad-hoc solutions are used. Digest authentication has failed to be adopted by browsers, so it sadly cannot be used for Web applications targeted to browsers (maybe that will change), but it certainly be used in the context of APIs (e.g. in Python, we have urllib2.HTTPDigestAuthHandler, Jakarta has a AuthPolicy.DIGEST, etc.).

Also, the Flickr API, like many others, exists in several flavors: REST, XML-RPC, and SOAP. But it’s basically the exact same messages, with a different envelope. It’s no wonder that people prefer REST APIs: they’re basically identical to the SOAP ones, except that you need to use less libraries in your code and your messages are simpler.

In this particular case, SOAP-based Web services bring you WS-Security that could solve this authentication problem nicely. Then the SOAP API would show the advantages of SOAP, and people may actually see the point of all this standardization (including Web services) work.

I’m really wondering how Flickr orders photos by interestingness.

Here’s the attributes of ours:

1. 30 views / 1 person counts this as a favorite / 0 comments
2. 5 views / 1 person counts this as a favorite / 1 comment
3. 26 views / 1 person counts this as a favorite / 2 comments
4. 51 views / 1 person counts this as a favorite / 3 comments
5. 17 views / Nobody counts Mac dudes as a favorite / 4 comments
6. 45 views / 1 person counts this as a favorite / 0 comments
7. 36 views / Nobody counts France vs Cyprus - 2005-10-12 - 4-0 as a favorite / 1 comment
8. 6 views / 1 person counts this as a favorite / 0 comments
9. Etc.

This looks random to me.

I have published a new version of Offlickr. It now enables to back photosets up, which is particularly useful as there is quite a bit of information in them.

There is now an Offlickr class also, which can be reused in other programs, in case somebody would like to reuse the code in another UI.

I started uploading my pictures to Flickr, and spending time writing descriptions, titles, adding tags. Other people have also contributed by adding notes, etc. So I really want to be able to get this data back, as I don’t want to solely rely on Flickr to keep my data.

I also wanted to play with the Flickr API. I looked into toolkits provided, and they are all built on the REST API. I ended up choosing Beej’s Python Flickr API. However, it uses an HTTP POST for all its calls. It’s especially weird for Offlickr which is a series of information retrieval (the permission requested is read-only). As there is some authentication information in the parameters sent, maybe it makes sense. It might have been better to use another authentication technique (digest auth?).

Anyway, I ended up with a small Python script which does the trick for now: Offlickr. I probably will improve it, but I now feel confident that I can get my data back from Flickr.

I called it Offlickr as once you have the metadata in XML form and the pictures, it should be fairly easy with some simple XSLT to have an offline copy of your Flickr space.