I have lots of data. I have thousands of emails, pictures, videos, contacts, documents, settings, preferences, bookmarks, passwords….the list goes on and on. Among my data there are two major trends. Application specific data and universal data. My application specific data includes my settings and preferences, bookmarks, passwords, contacts, emails. The more universal data includes my pictures, music, documents.

The line between application specific and universal data is blurring. My emails can be downloaded into another application, but once they get there they are again application specific. My bookmarks can be synced across multiple browsers, but still…it’s application specific. My music can be uploaded to the cloud, turning it from universal to application specific again. In truth, it’s a bit of a mess.

I like to think of this problem in three major terms. First there is intra-device sharing, this is mostly moving data between applications on the same device. The second, understandably, is inter-device sharing. Like having my music available on both my ipod and my desktop. A lot of work has been put into solving inter-device data sharing challenges, but we have a long way to go. The third and final term is inter-user sharing. Like sending a photo to your friend.

Intra-device sharing

There have long been two kings of intra-device sharing. Number one is the file. A contained structure for your data that splits it up into meaningful chunks and stores it in a standardized manner in what we call a filesystem. Without a filesystem one application would find it very difficult to use the data of another application. It wouldn’t be impossible, it would just be very impractical. Files are great for sharing larger pieces of data that are naturally self contained and have a mean outside the format they are stored in. A picture is a picture even if it is stored as a .png or a .jpeg, so storing it as a file makes sense. The same goes for a document or a video. However there is more to the picture than just the visual representation. There is data on what camera was used, focal length, iso and those technical things. Also there is data about the picture that isn’t necessarily stored, like where the photo was taken, who’s in the photo, how important the photo is.

The second king of intra-device sharing is copy and paste. It all began as a simple way of taking text from one place and inserting it somewhere else. The concept is great and for it’s purpose it’s a great tool. You can take a picture from one application and use it in another without having to store it in a file. The user gets to choose what to copy and where to paste it. Just being able to take a picture from your web browser and move it into a specific location of your document is marvelous.

These two solutions are great for data that is stored in standardized ways, but sucks for more conceptual data. I usually hate it when an application makes sound. I have a preference that could be described as “Applications should make no sounds”. If I put this in a file and expect to be able to open it with an application, I’m surly mistaken. If I go to the preferences of one application and try to copy and paste it’s settings to another application, it’s not going to work. This does have a solution today, it’s called system wide preferences. In essence there is one big list of my preferences that all applications should look at before doing anything. So if I have a default printer in my system wide preferences. Any application that wishes to print should let me print to this printer without any hassle.

For settings that are universal this works great, like my sound preferences. For application specific settings though, not so much. Let me explain.

I use firefox (mostly) for my browsing and I have it set up just like I like it. When I upgrade firefox most of my preferences remain intact, a few new preferences are introduced and a few are removed. So not everything is preserved. I have one preference that I hold very dear, you may find it stupid, but to me it’s sacred. I like to keep my tabs under my address bar right above the actual webpage. It’s just the way I like, nothing special or too complicated. Yet when I open chrome, it’s doesn’t seem to know this. So maybe if I copy and pasted my preferences from firefox to chrome, at least I would expect the settings that are in common to make it over, but no, no luck. So I dig around the filesystem and find a file which seems to contain the preferences for firefox and try to open it with chrome, chrome has no idea what I’m trying to do. It can’t understand that I want it to look and behave as much like firefox as it can. Lastly I look through system preferences, but of course the location of the tab bar is way to specific to make it into a system wide setting.

I’m at loss. Why can’t there be a standard way of storing my settings so that I can move them from one application to another, or maybe even from on device to another.

Inter-device sharing

If we can’t even share simple things between applications on one device. How on earth would we be able to share them between multiple devices? You’d be surprised.

I have 5 devices. One server, one desktop, one laptop, one ipod touch and one old nokia phone (going on 6 years now). Sharing data between my server, desktop and laptop usually involves finding a specific file and moving it from one device to the other. I have several options for doing this, with each option having it’s pros and cons.

I can do a direct transfer or transfer through an intermediary. Small things I tend to email to myself. Like a document, a picture or a link. For bigger things I usually use SCP.

There is no copy and paste between my devices and no context awareness when transferring the data. I’m essentially stuck with files and text again. I can’t really take a contact from my address book on my desktop and use my laptop to call the person with skype (ok, so maybe I can export the contact to some file type, email myself the file, import it into skype and then tell skype to phone the number…or just bypass all the pleasantries and type the number in by hand).

What I’m saying is that inter-device data sharing is a bit of a manual task. It’s slow and painful and really just makes you want to stick to one device.

Enter the syncing

If you have two devices and you want the data from one of the devices on the other device. You connect them together and sync them. Now each device contains the same exact data as the other. I don’t want to diss syncing, but I just can’t help myself. So here goes. Instead of having one contact book, I now have 5. And all I really needed was that one phone number. With contacts this isn’t really a problem, but consider my email. I have gigabytes of it, if I had to sync that between every device it would be a huge waste.

So, enter the cloud

The cloud does what syncing doesn’t, it keeps your data in one place and let’s all you devices access it. Essentially instead of every device having to be able to communicate with every other device, they now only need to know how to communicate with the cloud. Instead of having skype on my laptop talk with address book on my desktop, they both talk to the cloud and that’s it. So one standard rules them all. Great. But wait…this seems too good to be true.

And it is. With the cloud there is one thing that you really, really need. That is an internet connection. Now 90% of the time I have one, but in those few hours a day that I don’t, it would really be a pity if I couldn’t access my data. To compensate for this the cloud has turned back to syncing for some data, like my contacts and calendar.

There is still one major thing that is lacking. Say I’m editing a document on my desktop and I want to keep working on it on my laptop. Now I don’t want to stop working on it, transfer it and start working on it again. That’s a waste of time. I want it to just move to my laptop in the exact state that it’s in so I can keep working on it. I want the window to look the same, I want my place in the document to remain the same and I even want the undo history to remain. (yes, I know, google docs does most of this).

For a lot of smaller and simple things there are specialized solutions. If you’re browsing a web page on your desktop and have to run, Send2Prowl, for instance, will let you do this, but it’ll only work for that special case and won’t let you transfer the state of other applications or other devices.

There are a few attempts at state transfer between devices, Deep Shot is a great example. It lets you take a picture of you computer screen with your phone and transfers the state by analyzing the picture to find out what content on what computer. Works great for maps and websites. If you haven’t seen it in action, you should check it out. Engadget has a nice article.

Inter-user sharing

The last type of data sharing I’ll talk about is that between users. Say I have a document open and I’d like my coworker to take a look before I send it off. So I email him the document and he emails it back to me with his comments. Other than having him physically come to your computer it’s the simplest approach.

It’s also the least content and state aware approach possible. All your doing is transferring the file from one computer to another and then back again. You can’t watch edits live or inline comments in a standardized way. In essence you are locked into what the file allows you to do.

So back to the cloud then. If you store the document there you can send a link to you coworker and have him watch as you guide him through the document and get his feedback live and in context.

For small things this is an ok approach, but once you have bigger things or more conceptual things it breaks. Say I want to transfer a home video, that’s going to take over and hour to upload to the cloud (on my connection) and an hour to download. So it’s back to direct transfer or through a physical intermediary like a disk. With a disk everything is usually fine, you burn it out and then give it to your friend. But if you don’t want to use a disk (or other physical media) and have to transfer it directly, you’ll be back to a myriad of options and solutions that may or may not work. Usually I’d give up trying to transfer a song from someones phone to my laptop, because they just don’t talk together very well.

The essence

Our problems boil down to two major fields. The field of data transfer and the field of semantics.

I want a standard way of transferring data from one device to another that doesn’t require me knowing technical things like SCP or how to setup up Apache. A simple interface based in reality rather in the virtual world. Deep shot uses images to connect to devices together and an underlying protocol to transfer. HP/Palm uses physical proximity (one device on top of another) to connect and also an underlying protocol. In the beginnning it’s always going to be a million different ways of doing one thing, but eventually companies will start adopting technologies from other companies and after a while the best (cheapest) solution wins and becomes a standard.

The semantic part of the solution is a bit harder. How can you tell one application about the state of another application (regardless of them being on the same device or not). In essence you have two options. Either make a common language or try to make every application talk to every other application. The latter is the approach that has been most successful to date. For instance firefox can import settings from safari and chrome can import from firefox. However this approach raises the bar for what is needed to build a good application, and smaller applications will be unsupported and will never gain traction.

So a common language it is then (right?). I propose RDF.

RDF gives us standardized syntax that is human readable as well as machine readable. More importantly though it gives us the option of using common vocabulary. Let me demonstrate. A contact in my address book has a phone number. Even if every address book program used XML, you’d end up with one application calling it a “phone number”, another calling it a “phoneNumber”, another would call it “tlf”, some would specify that is was a cellphone and another would say that it was a work phone.

If some of the people who made the applications sat down and decided on a common vocabulary that would be sufficient for them, and then shared it with everyone. It could be adopted by lot’s of other developers and suddenly make your data application-independent. Don’t just think of this as your contacts, pictures and documents, but keep in mind settings and preferences. Suddenly switching from chrome to firefox would allow you to keep most of your settings (at least the ones they have in common) rather than having to start form scratch. You want to switch operating systems? No problem, your settings are coming with you.

So we are far away from a time when I can take my data form my desktop to my laptop to my ipod touch, use parts of it on my phone and let my friends see what I’m doing. It’s going to take time and effort, but I believe in you all. In time my data will be mine, and not locked down into the device I’m using or the application that’s running.

Oh, and if we end up going for RDF, then we’ll also have RDFS and OWL to help us reason on our data. Say you took a picture in Paris. Since we know that Paris is in France and France is in Europe. You can now search for pictures taken in Europe, and have your pictures from Paris show up.

I’ve had an arduino lying in my cupboard for well over a year now and I’ve been wanting to actually make it do something. Today, this wish came true for me, and here is how.

If you don’t know what an arduino is, then let me tell you. It’s a cheap little opensource platform that lets you write programs is a C like language to control motors, servos, read sensors and in general build robots. In essence it can do two things. It can read electricity or send electricity, and as it turns out that that’s all you really need.

Above is the arduino. It has a usb port (1) to connect it to the computer. It has a set of digital connectors (2) that can either read or write electricity, and the ones with a “~” can be switched on and off really fast so as to simulate a variable current (see: pwm). It has a set of analog ports that can read or write a voltage (3) and power connectors and ground (4).

If you wanted your arduino to, say, blink a led. You connect one leg in connector 13 (2) and the other in connector GND (4), then in your program you say something like:

digitalWrite(13, HIGH);   //light on
delay(1000);              //wait 1 sec
digitalWrite(13, LOW);    //light off
delay(1000);

Now let’s have a look at this boat.

After opening up my boat I found it had an circuit board with an RX-2B chip and a bunch of transistors. The picture below is after I soldered on 5 red cables, so those weren’t there originally. The transistors are used to control a big power supply (high volt and amp) with a low power supply. It’s used because the chip can’t handle the power needed for the motors, so it instead supplies a control signal to a transistor that can handle the higher voltage and amp.

My first though was to buy a H-bridge off ebay and just connect it to the power supply and the motors directly, but it would be a waste to just chuck the circuit board and start from scratch, so I had a closer look at that chip (RX-2B).

Alas, google is my friend. I looked up the chip and the first link was to a specs pdf. In it I found the following diagram:

Let me explain. OSCI is the oscilator and connects the the antenna. VDD is power in, and GND is ground. RIGHT, LEFT, FORWARD, BACKWARDS are the signal connectors for the transistors.

SI is the really cool one though. It let’s you send in a coded signal to tell the chip to do things. This is what I first considered, but I got a bit confused (me? confused?) so I gave up on it and instead focused on the FORWARD and BACKWARD connectors.

It seems that if I connect the VDD (13) to FORWARD (11) the motors start spinning. It’s really dead simple.

Connecting the GND (yes, it’s not the VDD, but just keep with me) to the GND on the arduino and the FORWARDS to pin 13 and then doing a digital write to that pin also makes the motors spin. I couldn’t connect the VDD to the arduino, because the arduino uses a separate power supply, so instead connecting the GND to the arduino just lets me override the chip and it’s power supply entirely.

What I did next was go out and buy a soldering iron and solder wires to GND, FORWARD, BACKWARDS, RIGHT and LEFT. Then I soldered resistors onto the cables to match the amps the chip was running at and connected it to the arduino.

Then I wrote a simple test program to see that it works. In essence it goes something like this:

digitalWrite(12, HIGH); //forwards
digitalWrite(10, HIGH); //left

Compile, upload and watch the left motor on the boat spin up. Now I need to get hold of a poximity sensor (probably an ultrasonic rangefinder) and make a simple collision course detection and evasion program so that I can pilot my boat towards the shore and watch the arduino take over and steer it left or right to avoid crashing. Or maybe write a program for shore following (like the iRobot vacuums wall following, but on water). We’ll see. I need to order the sensor, so I’ll probably get back to this in a few weeks.