Sharing Is Broken

I have a big pet peeve on the web. I know. It’s another rant of mine that may or may not turn into a good idea. (Queue appropriate XKCD reference Here it is: There is no good way to post something in one place, and share it everywhere else automatically, in whatever format is best for the site it’s being posted too.

For instance, I’m sitting here writing this blog post, and I want to share my verbal tirade with the world. Specifically, I want to post it on my blog, which happens to be based on WordPress. WordPress has this nice clean RSS feed feature that cool people like you can use to subscribe to my blog updates. Now, I’m half in half out of this whole ‘Social Media’ thing, and I have accounts on most major sites, apps hooked up on my phone and so on, but it’s a pain to update everything at once. All I want to do is share this awesome post with the world in whatever medium the rest of the world is using, and keep control of my data so that if Facebook accidentally pushes out an update that melts every single hard drive in every one of of there servers, I still have a backup. But that’s another discussion. Unfortunately, RSS feeds don’t cut it because the rest of the world is uncool and doesn’t use Google Reader or some similar app. So, once I post my blog, I put the title and a link and post it on Twitter. Put a mini excerpt and put it on Facebook. Put another excerpt on Google+. Email the parents. Print it out for the Grandparents and mail it. You get the idea. Twitter I can automate. Facebook I can kinda automate via twitter. And Google+ is the awesome social media site without a programming interface in the corner. Damn you Google+. Why do I love thee so much?

We live in the modern age. So why can’t I have a service, or an app, or a tool that takes my post, and cross posts it across the “Big 3”? Heck. I’ll write my post in markdown so the app doesn’t have to strip and process out all the simple html tags out of my post incorrectly. So, until some smart person comes up with a GOOD solution, I’m going run this show on intern power. Good ‘ol human labor.

Thoughts on Amazon S3 with WordPress

Recently, I’ve been looking for a way to reduce my bandwidth and storage bill for this site. I’m currently hosted through a company called Nearly Free Speech which is a fantastic hosting company for small static or PHP / MySql based sites, and it’s been great to have a hosting provider that I’ve literally never had to worry about. However, given that I regularly upload and distribute a large amount of media around this site I tend to eat through storage and bandwidth pretty quickly, especially now that my site has been around for about 4 – 5 years. As a super cheep hosting provider for small sites, NFS if fantastic for those small start up sites, but prices do increase quite a bit as the site grows. A few months ago I signed up for Amazon S3 (Simple Storage Solution or S3), which is a simple, flat, ‘blob’ storage system. Amazon itself originally built this system to power the media distribution for its own site, and then opened it up for others to use as well.

The basic concept is quite simple, instead of worrying about complex structures or directories the core service simply assigns a file a huge unique number and a title to the blob of data your uploading. It doesn’t matter what the data is, it could be anything from an image, to a zip file, to an 3GB text document. It doesn’t matter. It’s then possible to distribute that blob around the world so that the data is physically closer to whoever is requesting it. Because it doesn’t matter, most of the programs that use it choose to use the title as a path, which means that it uses the “/” character, making the file appears to be in a directory. In essence, Amazon S3 is a simple flat file server with a simple HTTP layer on top of the storage system for easy access to these objects in web applications. Assuming you’ve allowed public access to the particular file you can request it by performing a get request on your account followed by the title of the file.

However, the biggest original draw was the incredibly cheap storage and bandwidth costs, NearlyFreeSpeech weighs in at around $1.00 / GB down to $0.20 / GB based on the total bandwidth over the life of the account and storage is at $0.01 / MB per month ($10.24 / GB per month). Now, for a super small site, it’s a non-issue and you can’t beat the low end tier for pricing and ease of setup (You can see the full pricing here on the Nearly Free Speech pricing page). Now, consider Amazon’s S3: $0.150 / GB for storage (Until your storing more than 50TB of stuff, when it goes down a cent per GB), and $0.150 / GB of bandwidth. (You can see the full pricing tiers at the amazon web services simple storage page.).

Now, my current bandwidth for the site is about 1.5GB a month and my storage is about 400mb at the moment, for all intents and purposes its not a huge site, but enough that I wanted to see if there was a way I could take down the pricing. I’ve tried several things in the past such as offloading my photos to Flickr (Which I don’t like because it gets blocked occasionally) or SmugMug (Which doesn’t have a good plug-in for inserting photos into a wordpress site, but interestingly they also uses Amazon S3 to host their content). After tinkering around with it for a few months and using it as a host for large downloads, I went looking for and found a plug-in that replaces the core upload / media storage and retrieval functionality with calls to S3, and now that I’ve pushed my uploads folder to be hosted through S3 plus a quick switch in the blog settings and everything is now pulling from Amazon S3. Much cleaner, much nicer, a much less out of my pocket each month.


– Paul Rohde