flickrfs - Online in the Cloud

PROGRAM:

NAME

flickrfs - virtual filesystem for flickr online photosharing service

ABOUT

Flickrfs is a virtual filesystem which mounts on your linux machine like any other
partition. Once mounted, it retrieves information about your photos hosted on your flickr
account, and shows them as files. You can now easily copy photos from your local machine
to this mount, and it will automatically upload them to your flickr account. Similary, you
can copy the files from your mount to your local machine, and it will download your images
from flickr.

All the files in the mount have a meta file attached to them, which provides access to
title, description, tags, and license information. Modifying any of these fields and
saving the meta file, will update them on the server as well.

You can easily specify what is the default resolution of photos that you wish to upload,
and the fs will automatically resize (read downsize) the photo, and upload it.

That's not all, the latest verion of flickrfs, provides a syncing mechanism. This
mechanism automatically syncs any changes done online directly to flickr account (using
flickr's web interface or any other third party apps), to your mount. Thus, any
addition/deletion/changes of sets/images is reflected back to your mount. No stale
information! You just to mount it once, and it can be used forever!

USAGE

Configuration
The first time you run flickrfs a directory .flickrfs is created in your $HOME directory.
The directory contains a file named config.txt. Edit the file, to set the following
options:

[configuration]
browser:/usr/bin/x-www-browser
image.size:1024x768
sets.sync.int:300
stream.sync.int:300
add.default.tag:yes
APIKey:f8aa9917a9ae5e44a87cae657924f42d
Secret:3fbf7144be7eca28

The sync intervals are in seconds. The default interval set is 5 mins. The sync of /sets
and /stream is irrespective of each other, so you can change the interval values as you
like.

image.size field specifies the default size that the photos will be resized to before
uploading. If you wish to upload the original size of photos, leave the value blank as
follows. Note that, only a copy of photo is resized, your original photo will not be
resized.

flickrfs will by default add the tag 'flickrfs' to all pictures the user uploads. Set
add.default.tag to 'no' to disable this feature.

The default values of API key and Secret used for authentication are specified in the
config file. You may change them if required.

Create a screen terminal (Optional)
What is screen? See screen(1)

Why? This will keep the fs mounted even when you log out. Next time you log in, you can
directly access the fs, without having to mount it again. Saves effort!

$ screen -S flickrfs

Mount the filesystem
Check if fuse module has already been loaded in kernel.

$ lsmod | grep -i fuse

$ sudo modprobe fuse #If not loaded

First time you are mounting the fs, you need to close all the browser windows. Secondly,
if you are behind a proxy server, set you http_proxy environment variable in the command
line.

$ export http_proxy="http://proxy.address:port"

$ flickrfs /mountpoint

The default web browser used is the system one. If you use something else, you can set it
in config.txt.

Your browser window will open up, and require you to login, and allow authentication to
flickrfs. Once you do that, close your window. Flickrfs will now start downloading
information. Wait a few minutes for your sets to populate. If you wish to view your whole
stream, create a directory stream in your mountpoint.

$ mkdir /mntpoint/stream

You are ready to GO!

See flickrfs in action (Optional)
If you wish, you can view exactly what is flickrfs doing, by checking out the log file
present in $HOME/.flickrfs

$ tail -f ~/.flickrfs/log

Unmount the fs
To unmount the filesystem, execute the following command

$ fusermount -u /mntpoint/

Structure

Flickrfs Home Directory $HOME/.flickrfs:
config.txt:
Allows you to set the default image size, and the time interval for the syncing of
/sets and /stream to flickr online server.

log: log file which keeps record of activity going on in flickrfs. Useful for debugging
purposes. In case you find a bug, post it along with this file on the mailing list.

The filesystem mount point contains these following folders by default:
/sets
/sets folder contains your sets, including your private photos. You can easily add/delete
sets using standard linux commands.

/tags (contains /tags/personal and /tags/public)
/tags/personal folder allows search based upon tags in your photostream. Just create a
folder (use mkdir) whose name is a colon delimited tags like 'tag1:tag2:tag3', and the fs
will search your photostream to match all the photos which contain ALL these tags, and
show them as files inside the directory.

/tags/public does the same thing, but searches for public photos even outside your
photostream.

/stream (not created by default)
/stream folder will contain ALL the photos that are present and accessible in your
photostream. Obviously these wd also include photos from your sets.

This folder is not created by default. You need to create directory named 'stream' in the
root of mount point for the fs to populate the photos. Once the folder is created, it
can't be removed.

Feature List and Usage

Integration with Berkeley DB - Minimal hardware resource consumption by flickrfs
flickrfs has been integrated with Berkeley DB using python's bsddb module. Which means
flickrfs would now store the image information in Berkeley db database, instead of storing
it in memory. Let come millions of images, flickrfs would still consume only negligible
amounts of RAM. Regarding hard disk space, it would be in order of 10s of Megabytes; not
an issue for today's computers.

Performance increase
flickrfs retrieves the sets information in parallel, background threads. Hence, the
directory structure is created quickly, allowing users to start working. If some sets
couldn't be retrieved the first time, they'd be taken care of when *syncing* kicks in.

Secondly, full metadata information of images would only be retrieved when asked for;
though they'll always be listed in the directory. Its just like the way flickrfs handles
images.

Robust execution of flickr operations on small bandwidth
Flickr operations, as in, interactions with flickr server are now fail-safe. flickrfs
handles the URLError exceptions thrown by the operation, checks its result, and retries
the operation multiple times in case of failure. Thus, even if connection is lost
temporarily, flickrfs would continue its operations unaffected.

config file would be automatically created
config.txt file, present in ~/.flickrfs would be created automatically, if not present.
Normally the default values would be fine, but feel free to change it according to your
needs.

Bandwidth information of flickr account
statfs system call is now tied to the bandwidth usage information of your flickr account.
You can view it by running a

$ df -h

Automatic periodic syncing of fs with online server.
flickrfs will automatically sync any changes done directly through flickr web
interface/organizer or any other third party apps, to its local mount dynamically. This
include changes to your /sets and /stream. The photos present in /tags are not synced,
simply because /tags is meant just for searching/downloading purposes. The time interval
can be specified through config.txt file. The default interval specified is 10 mins.

Directly link photos from your stream or tags to sets, without downloading.
Use your ln command to directly add a photo present in your /stream to a set.

$ ln /mntpt/stream/image.jpg /mntpt/sets/yourset/

$ ln /mntpt/tags/personal/../image.jpg /mntpt/sets/yourset/

OR, if you wish to add a new set altogether, just create a new dir before executing link
(ln). When you execute a mkdir, link your first photo quickly, so that the set is created
online. Otherwise, the sync thread will delete the new directory, seeing it as incoherent
with online server.

$ mkdir /mntpoint/sets/newset

Automatic resizing of photos while uploading.
NOTE: This does not affect the original photo. Only a copy is resized, and uploaded.

If your photo resolution is high, fs will automatically downsize (a copy of it, not
original photo) to the resolution specified in config.txt file, before uploading. This
also reduces the image's size, thus useful to people with limited 20MB upload limit. Note
that if the image is smaller than the specified size, it would not be resized, and the
original size will be uploaded.

Unicode Support
More robust unicode support. This feature is still in testing mode, so if you find any
bug, feel free to contact us.

Change photo meta data
Title, Description, Tags, License: Use meta data file associated to photo. For photo named
img1_ID.jpg, the meta data file will be updated to the servers.

$ vim .image1_ID.meta

You may also use 'sed' to do batch operations. For e.g., To change the licenses of all
photos from license:0 'All rights reserved' to license:1 'Attribution-NonCommercial-
ShareAlike License', execute this:

$ sed -i -e 's@license:0@license:1@' .*.meta

Permissions: On the image that you wish to change permissions, execute these statements:

$chmod 755 image_ID.jpg #to make photo public

$chmod 774 image_ID.jpg #to make it available to only friends and family. Not to public.

$chmod 764 image_ID.jpg #to make it available to family only

$chmod 754 image_ID.jpg #to make it available to friends only

$chmod 744 image_ID.jpg #to make it private

Time: Modification time of the image is set to its last update time on flickr, while
creation time is set to the time when the image was uploaded to flickr.

$ ls -l #Sort photos in asc. order according to 'last update'

$ ls -cl #Sort photos in asc. order according to 'date uploaded'

Uploading
Copy the image to either /sets/yourset or /stream, with the directoy name as follows:

$ cp /mycamera/myimage.jpg /mntpoint/stream:tag1:tag2:tag3/

and the photo will be uploaded with the three tags, and an additional tag 'flickrfs'.
Spaces in tags are allowed.

To create a new set, just create a directory in /sets/. The first photo that you copy or
link to that directory will become the primary photo of that set. When you execute a
mkdir, copy your first photo quickly, so that the set is created online. Otherwise, the
sync thread will delete the new directory, seeing it as incoherent with online server.

$ cp /mycamera/myimage.jpg /sets/newset:tag1:tag2:tag3/

Note the '/' at the end of the destination paths. This is required to differentiate b/w a
filename and directory.

Note: /tags/personal can no longer be used for uploading photos. Use /stream for that.

Deleting
Flickrfs doesn't allow/do deletion of photos. It is to prevent a 'rm *' accident!

You can easily remove photos from your sets though.

$ rm /mntpoint/sets/myset/image1_ID.jpg

If the photo is the last photo in the set, the set will be deleted automatically.

Searching
To search your personal photos for tags: tag1, tag2 and tag3; create a directory in
/tags/personal:

$ mkdir /mntpoint/tags/personal/tags1:tags2:tags3

To search public photos, do so in /tags/public.

The directory will store the results as files, with a maximum of 500 files.

Downloading
Copy the image from the mount to your local harddisk, and the original size of the image
will be downloaded. If original size if not present, the accessible largest size of the
photo will be downloaded.

$ cp /mntpoint/../image1_ID.jpg ~/myphotos/

Switch Account
If you wish to switch account, just delete the folder $HOMEDIR/.flickr. This will remove
your cache, and ask for authentication next time you mount the fs.

$ rm -R ~/.flickr/*

Flickrfs User Guide Online

Please visit <http://manishrjain.googlepages.com/flickrfs> to read more about flickrfs
usage and features.

Use flickrfs online using onworks.net services