natevw proudly presents:

a glob of nerd­ish­ness

powered by work over time.

Cost to keep ShutterStem image masters backed up in Amazon Glacier

I was excited by this morning's announcement of Amazon Glacier.

I've been interested in automatic offsite backup of my photo archive but found the cost for "cloud storage" prohibitive so far. (I'm assuming that attempts to actually take advantage of e.g. Flickr Pro's cheap "unlimited" storage will not end well. Also, any sort of "we'll happily take all your stuff" offer also tends to imply insecure storage of data.) A penny per gigabyte each month is a game changer — an order of magnitude cheaper than anything else I've found so far.

There's a slight catch in that Amazon charges an extra fee for retrieving more than 5% of the total amount you have stored each month. What happens if I'm using it as a backup for all my photos and vidoes, and would like to quickly recover it all after a data–tastrophe? It gets complicated but here's my best estimate for the real-world situation I'm interested in. If I've crunched the numbers right, I like what I'm seeing!

Glaciers above Mount Rainier's old Paradise visitor center

Say I've got 500 GB backed up to cold storage, something goes horribly wrong, and I need to locally restore a complete copy from the cloud. My ISP at home can "burst" around 10 GB/hr but hates my guts if I download more than 250 GB/month. So, the quickest I can expect to have it all home is 2 months, ten times faster than Amazon's "preferred" 20 month free retrieval period. This means I'll be "charged a retrieval fee starting at $0.01 per gigabyte".

Let's work through Amazon's pricing example with my round numbers. For 500GB, my peak daily allowance would be 0.833GB. Assuming I steadily exceed that, downloading an even 8.33GB/day, my peak hourly retrieval rate for the month will be 0.347 GB/hr. This results in a billable peak rate of 0.312GB/hr after subtracting 1/24 of their daily allowance. So my excess retrieval fee would be only $2.25/mo (0.312 GB/hr * 720 hr/mo * $0.01 GB) — less than 10% of an increase on top of the corresponding $24.90 of standard data transfer cost each month!

The numbers get a bit better for 1TB of data (which I'm heading towards). My ISP is the limiting factor; doubling the data at rest actually decreases my retrieval overage fee to an even $2.00 each month. So as my photo archive grows into that ballpark I'd be looking at:

Four months seems like a long time, but it's way longer than I've been hobbying upon ShutterStem — which, of course would be a perfect fit for Amazon Glacier! In fact, since all my main ShutterStem apps rely only on the "always present" 512-pixel photo thumbnails, having such cheap storage means that my hard disk archive is almost just a local cache, mitigating Glacier's 4 hour storage latency. Theoretically I could toss all my big drives, run my medium-resolution ShutterStem library off SSD, and just process any full-resolution exporting for prints/uploads overnight. Of course, why would I do this, and what would happen if AWS were to lose/corrupt that copy of my data, but it's fun to consider…

Sketch of ShutterStem's architecture using Glacier in place of traditional filesystem for storage of original images

Overall, Glacier is looking like an excellent fit for ShutterStem's design and intended audience. When used as a true just-in-case offsite backup, anticipating catastrophic recovery happening over average US broadband speeds, the pricing is very competitive. The crazy numbers come only if you've got a gigabit connection at home or suddenly feel like slurping your entire Glacier archive into an S3 bucket por el pronto. The fact is that cloud backup is "slow" for mere mortals, and slow is what Glacier wants. I expect to see a number of backup apps spring up around this new offering, and am already pondering what the interface for ShutterStem should look like.

comments

Assembling a Printrbot Plus

Last week a friend gave me the Printrbot Plus he backed on Kickstarter! I suspect he gave up finding the time for it; the box label says it originally shipped on June 14 :-)

Opening the box with Toby's help

I'm super excited to have a 3D printer. Toby is too; he's eager for me to get done "fixing" it and start printing toys and parts we've found on Thingiverse. [He recently broke his femur and will be in a hip spica cast for a while. He's been doing a lot of Duplo at the table lately. Being able to extend his toy sets in interesting ways could be fun.]

Lots of wooden parts

Opening the box and pulling out myriad parts and pieces and fasteners, I was a bit overwhelmed myself. The official instruction page don't provide a very smooth start. (There are three different Printrbot models to sort through, none of the parts diagrams include big-picture context or step-by-step instructions, and the "Building the Printrbot LC and PLUS" videos I tried to start with were sort of confusing and lite on any actual assembly.) It seemed like the official stuff was all aimed at people who had already built other printer kits or who had been closely following the project from day one; I just had a huge box of parts and desperately wanted someone to please say which two pieces I should start with!

Base assembled (for the first time) Base assembled correctly, with z-axis rods

What really helped was an (in progress) series of unofficial Printrbot Plus–specific assembly videos by "Craig the Fabricator". The ones that have been uploaded so far got me off from a great start to a point where the official videos became easier to follow. I found a wiki with reasonably good step-by-step assembly instructions that has been indispensable as well.

Almost together, just needing missing bearings and some final assembly

I'm currently stuck waiting for Printrbot HQ to send some missing bearings for the z-axis as well as (hopefully) some replacement plastic parts for the extruder head that got cracked. When it's done, I'm hoping to get competent enough with it to have it be something of a community resource for Room to Think members and friends. In the meantime, I've got a replacement balloon to keep flying and plenty of work to keep up with.

comments

Argyle tiles preview

Last week I launched an early preview of the data that's been rolling out over at Argyle Tiles. There's a good portion of the global coverage already in place, as well as a single-city sample of what I'll be able to provide in most US urban areas. You can check it out via the signup page.

Faded screenshot of map on Argyle Tiles signup page

I have been prioritizing client work over this speculative venture, but I've also been outsourcing some of the tiles work to make sure the ball keeps rolling even while I'm meeting my other commitments. The list currently looks something like this:

  1. finish generating and clean up some global coverage issues (Finland spot, coastline resolution)
  2. start adding basic coverage for the whole Continental US
  3. beta launch: provide API keys and documentation for early customers
  4. begin processing more popular high-resolution urban areas

I'd love to hear what you think, and would also be grateful for any more leads — if you know developers who could benefit from the service in its beta, please send them over.

comments

Forth and onwards

It's the tenth Independence Day since 9/11.

Since then the government, the shadow government, the establishment and the suicide bombers have continued their fight against "democracy".

The rule of the people, by the people, for the people. Which people is in a constant state of turmoil, but it shall perish from the earth. So I must constantly defend against terrorism — against believing any mere people should have me afraid.

Freedom is a state of heart. Never forget.

comments

On the burners

In May, starting to acclimate as a freelancer, I started wondering how much client work and side projects I should consider. How many things should I be working on?

Well, most stoves have four burners. Here's what I set on in June:

  1. Room to Think organization — the Tri-Cities coworking group I'm in recently incorporated as a non-profit with me on its first directors board. I may be biased, but we've already got a great start: friendly members continue to gather, and we landed a wonderful location overlooking a park on the Columbia River for our space. We have plenty more to figure out as we settle in, but so far so good and the community has been very supportive.
  2. Inquirium client — it's an honor to work with this smart, hard-working crew — and getting to tackle an e-reading related app no less!
  3. Meograph client — this has been one of the most chaotic, but also the most shipping-est, product developments I've worked on.
  4. Argyle Tiles project — busy bubbling on the back burner, we have my speculative aerial and satellite tile hosting business.

ShutterStem and other personal data management apps are still lingering below (in the oven). There's a lot of infrastructure needed for most of my ideas there — not just technologic and financial, but also building with and for people who care. So it was really encouraging to participate in IndieWebCamp this past weekend, too.

An old kitchen stove — image from http://en.wikipedia.org/wiki/File:Gas_stove.jpg

comments


All posts

Subscribe