There is a hierarchy of use cases CouchDB fulfills:

• A central RESTful data store
• A chain of SLEEPy databases
• Many nodes sharing as much data as they agree to disagree about

The first is handy, but boring. It's great that with CouchDB I don't have to work through a bunch of boilerplate every time I want to spin up an API on top of a datastore. CouchDB is a datastore with a great API already included, but RESTful web middleware is nothing a little code generation or some developer typing practice couldn't already provide above any database. The only thing exciting about giving third-parties access to your database over HTTP is how easy it is to accidentally share too much of said data. (More on this in the context of CouchDB in another post, perhaps?)

The second starts to get interesting. As promoted, the "Syncable Lightweight Event Emitting Persistence" aspect of CouchDB's _changes feed allows clients to maintain an up-to-date mirror of an authoritative data set. This is powerful! I've often wanted "a CouchDB" of offline IETF RFCs, up-to-date Wikipedia articles, realtime Open Street Map edits, weather forecasts, you name it. Having "my own" copy of valuable information and being able to efficiently process secondary views of its contents would be liberating. I'm excited to see how far dat can take this idea.

Only the last use case makes CouchDB a truly masterless database, however.

What makes CouchDB [almost] unique among databases is not its _changes feed, but its revision trees! Let's call this feature DOZE (a stretch upon DecentraliZed ObjEcts), or perhaps NAP (for Normative-Agnostic Packratification), to be like the other two.

For every document, CouchDB also builds up metadata concerning its history, namely all known revisions ever. This revision metadata is not a simple ordered list, but a tree allowing multiple branches. When CouchDB replicates it uses this revtree lineage to decide whether an incoming change implies a direct modification to the local document, or a divergent "alternate universe" coming of age for said doc. For the "alternate universe" case, the revtree branches end up leading to multiple documents which co-exist simultaneously. DON'T WORRY "DOZE" GETS EVEN MORE PHILOSOPHICAL

So anyway, most of the documentation and even CouchDB's API itself, glosses over this important aspect of a given document's existence. When it must be explained the community calls it "conflict resolution", rather than the more accurate "branch management". What you need to understand is that the primary index CouchDB offers is not a key-value store. It is inherently a key-values store.

In most multi-node data architectures, "the truth" must be decided before a particular portion of the system is considered synchronized. The most common arrangement for keeping multiple "client" devices in sync to treat a "master" server somewhere as the Single Source of Truth, propagating all changes therethru. You could eliminate the central point of failure, but frequently all parties in such an arrangement still agree-to-agree that, in essence, there is One True Dataset; sync is a matter of eliminating deviations from this absolute.

With DOZE, there is no database. Sync becomes a matter of sharing state, not determining absolute truth. There is still an ideal; within a CouchDB "social network" a given key represents the identity of one particular document. All databases will eventually agree that doc._id exists/existed even if they end up with different contents. In practice, usually the document's type and usefulness within an app also remain consistent between all nodes. And with "standard" CouchDB there is even a veneer of complete system consistency — a polite lie of consensus that (based on its generation number or falling back to comparing random hash output) one revision will "win" against the others by default.

DOZE doesn't "resolve conflicts". CouchDB's actual replication algorithm simply compares and shares revision trees and the most recent known versions of each known document. This is a different model than people imagine based on what gets indexed locally or returned by a naïve GET request — even though on the surface CouchDB pretends like the conflicting versions don't exist, it is pushing and pulling all the options when it replicates. And they are options: you can continue to update a "losing" revision's branch just the same as a "winning" document's if you care to. Once you realize how winningly CouchDB's replicator treats all current versions of a given document you start to get extra annoyed at those 409 errors an individual database so adamantly throws to its own users.

Conflicts happen. They're harder to design an interface around, but this is the only reason CouchDB papers over them. If you want to "do masterless replication" you need to embrace conflicts. The difference between SLEEP and DOZE is the difference between publishing source code and accepting pull requests. SLEEP makes it easy to maintain a mirror, while DOZE makes it easy to maintain a fork. With DOZE you don't need to assume a neutral point of view exists, each database simply accepts only changes it agrees with. In theory, a future version of CouchDB could actually let in and replicate out document revisions that validate_doc_update rejects, without allowing them to "win" locally as far as the current style of indexes and app interfaces go.

In practice, I'm not arguing for an eminently tolerant database. CouchDB tends to assume disk is cheap, but I don't want my SSD wasted on your version of RFC 2616 replacing it with a copy of httpd's SVN repository. My disk, my definition of "normative" whenever possible. It's an interesting idea for allowing dissent "within" a higher level-application, e.g. Wikipedia itself — Ward Cunningham is building that, by the way — but in practice we're probably mostly interested in using CouchDB for offline apps (where the conflicts should indeed be resolved as soon as possible) and the more general DOZE for maintaining corrective patches to otherwise authoritative-but-changing datasets (where the "main server" branch should be left open and updates folded into the locally-patched revision).

In summary, if you're avoiding conflicts, you're probably not using CouchDB to its full potential. I'm as guilty of this as anyone, because developing on top of a single CouchDB database makes conflict resolution easy to "put off". And indeed, you don't have to resolve conflicts in CouchDB and when you do, you don't have to use the arbitrary "winning" revision to do so. Certainly if you're trying to join CouchDB's "masterless replication" ecosystem you'll need to donate some disk space to the cause — to wish there was only one current revision allowed is to wish the data model weren't really decentralized. Don't go back to throwing away data just because it looks old. Track your revtrees, deal with life's inevitable conflicts, and keep postmodernity weird!

Some five and a half years ago, I proudly quit.

… AD/HD-addled brain … I’ve been wanting to start my own company … Out of all my ideas, only a handful seemed to have much feasibility or market potential …

I've been thinking, it's probably time to quit again.

I'm still occasionally haunted by visions of a world where my dream career works, but at the end of the day I simply pour/dribble/squeeze my last reserves of mental energy into a book, and then: I go to bed. I wake up, spend another day trying to work and another evening trying to give quality time to family. And I'm not going to invent TCP/IP, or build the Macintosh, or personally secure the unhosted web's victory over both Googorwellian AND iHuxleyan‎ oppression. I'm not going to change the world.

Sorry, I guess?

It'd be crazy to think it is God's will that I dent the universe. It'd be foolish to try just because some TED talk–type claims trying is humanity's only hope. And it's kinda pointless to feel guilty about letting myself down. It was me who picked a ridiculous goal, and I can't think of anyone else who needs (or even wants) me to stick with it.

So I quit. It's official. Nate is too old to joust with giant windmills, to put his body upon the gears and upon the wheels, to spend so much time on such great things that everyone lets him decide what deserves to exist for a while.

I don't have the time to capitalize or even Kickstarter any of "my ideas with market potential" (as if). I don't have the money to "start my own company". I don't have the energy…oh wait…I do have my AD/HD-addled brain and that seems in no danger of going away.

A year and a half ago I was wrestling with goals and a lot has changed since then but not that. Not the wrestling, nor the answer I hope to find at the end. And between now and then, tiny goals are all I really need. So I leave the LLC renewal form on my desk, to expire like I did once before, in self-pity then but in confidence now.

If I ever need a corporation again for some little thing, I know an operating agreement and some filing fees should be the least of my worries, and will be one of the smallest expenses. In the meantime, worrying about the big picture has been a big expense the last few weeks…and years. The more time I spend dreaming and trying to decide what to do with my life, to the point where I've conquered it with logic and can codify it into cute catchphrases or zen ramblings or even concrete plans of action, the more that stress wastes my energy and loses me money.

It's waaaay past 5 o'clock again. I quit, and I'm going home…I have a lot of meaningless work to catch up in the next week of my inconsequential life.

Which I no longer consider a problem.

Officially.

DO YOU HEAR ME BRAIN HAVE YOU EVEN BEEN LISTENING PAY ATTENTION AND DON'T LEAN YOUR CHAIR BACK LIKE THAT ONE MORE OUTBURST YOU WILL BE STAYING IN FOR RECESS

Patience is starting to pay off in the greenhouse.

Plants

Are doing well. Perhaps a little too well — it's getting hard to see the vegetables for the jungle!

So far we've eaten: more green onions, several small strawberries, a bunch of lettuce and spinach, and two green beans. Coming up: more beans, some peas, and the tomatoes all have blossoms!

Equipment

Added battery backup for airpump, float valve for water pump. If power goes out (or gets unplugged by a young child), or a water pipe comes loose, these should give me a bit of extra protection against full fishdaster.

I'm pleased, but also not very proud, to report that the UPS does detect and avoid electrocuting those who pour water into its top vents.

My parents visited recently, so I enlisted my dad to do some heavy duty carpentry for the next set of grow beds. Still need to finish the plumbing and chase out some leaks (and wash a bunch more gravel) but "phase 2" is starting to come together.

Fish

Added tilapia. Twice.

Apparently, while my cursory research indicated that goldfish were mild and not likely to bother tilapia, however my search terms did not specify just how small of tilapia I was worried about. Apparently goldfish, while not terribly aggressive, do not mind small tasty snacks.

So the second set is currently growing out in their own bucket, while a small handful of tilapia survivors are managing to rapidly catch up in the "goldfish" tank.

Next up

I'm at the NodePDX conference right now and surrounded by people doing cool things with software and small sensors.

I've got all the parts for getting my own greenhouse online, and hoping to integrate them together in time for NodeConf.

It's that time of the year/quarter/day/hour — I really don't track these up and down cycles — another time of life that usually ends with me reading through Ecclesiastes in a single sitting and then knowing again this is normal, it's okay, and resting in that…at least for a time.

This time, like most times, it's about time. Here I have it, never enough but always more than I know what I'm to do with. What to do with the time I've been given? What should be on my "bucket list"? The sun sets, the sun rises, and what was the meaning of life again?

I'm always behind on my bucket list, and it always makes me feel guilty. When the guilt gets to a certain point, I shirk some other responsibilities, catch up a little, and then spend the next few months trying to catch up back up on sleep and/or finances.

Q. Why am I here?

It doesn't help at all when a friend writes:

The thing you’re really passionate about. It’s distracting you completely right now. It keeps you up, wakes you up, and catches you daydreaming in between the rude interruptions of real life. … There is something you truly want to do—and guaranteed if you fish around your heart long enough, that calling is there.

…but then also asks:

Why do you want to do what you want to do?

Because I don't know. There's a lot I just want to do.

Here is a sampling of projects that are currently severely constrained by time and money

• PeerPouch
• Metakaolin
• ShutterStem
• writing a book
• reading many many books
• making photos, making music
• more useful apps that people like
• a decentralized internet of things
• helping specify meaningful open standards
• raise an entire pizza on my property (just once)
• …while still having evenings and weekends to spend with the family

I really care about doing all those things because

Let's dissect the first one.

PeerPouch is what I've been calling my (slow and way too careful) work to add magical "it just works" WebRTC support to PouchDB. Why? It'd be cool if app developers could easily use PouchDB's masterless replication over the direct connection WebRTC can provide between two "normal" (non-server) computers. Why? I'd like to help reverse the trend of users sharing all their personal data with money-motivated "too big to fail" corporations. Why? It's better for power to be distributed among smaller semi-autonomous entities than to be centralized. Why? The technologies/societies we design should be robust against individual failures or malignancies. Why? "Woe to him who is alone when he falls and has not another to lift him up!"

So does PeerPouch matter?

Well, I don't believe it actually does. I don't believe humanity will save itself from its own pride, and certainly not via some nerdish network architecture or leaked top secrets or crazy crypto currency; I believe God's already finished with his design to lift us out of our fall. So if you believe PeerPouch is our only hope, write me a check; deep down I keep dabbling at it only because it's enjoyable.

Speaking of enjoyable, I want to harvest everything I need to make pizza off of my own land someday. Why? The oil and salt and flour for the crust are impractical to produce at a small scale, but the result would be worth it. Why? Growing the ingredients will deepen our appreciation for this common food we eat, which is more valuable than the purely practical aspects of having food on the table. Why? A meal should be enjoyable. Why? "Go, drink your wine with a merry heart…" I dunno, you ask to many questions, kid.

So does pizza matter? Certainly a lot less than PeerPouch matters — one can hope?!

Probing within myself for "whats" and "whys" generally tends to be this unproductive. There's a lot I really want to do, with no real reason I can ever find. It could be because these things just look fun when I see others do them well. It could be because I'm addicted to learning, and doing is the best way to learn. It could be because I crave affirmation that I am valuable.

Valuable at a big — perhaps even cosmic — scale!

That's it.

No, seriously. I've built a lot of things, they were fun and I did learn alot and I was still unhappy last night because

well I was basically complaining that because I'm bad at visual design and can't really afford my own taste in it, people seem to overlook all the hard work I've done. If everything I do looks stupid and pointless how will the world ever realize how brilliant and important <del>it is</del><ins>I am</ins>.

And so, I do recommend Adam's article — go read it now if you skipped the first link — and I agree that there is something to just doing.

It's against that "just do it" sense though, especially the "just" (i.e. "only") focus of the doing, that I would gently digress from his advice.

I've got a long list of things I've found I really love doing. And I do them…occasionally, on the side, when I can. I hate halfway I hate settling I hate excuses I hate bursting forth in "blubbering apology to my dreams for not living them". But

I've tried "doing" full time. I try "doing" part time. I'd love to "just do" I'd love to live the dream every week every day every minute. Excuses, however, are real. They are each "absolutely a giant pile of bullshit" but they are all my giant piles of bullshits. Piles which I have been given — alongside my glorious snowflakey passions — and they are real. More real than dreams. If I do not tend these piles, they do not compost. (If they do not compost, they ain't just gonna disappear neither.)

Even after I turn these shit stacks into fertile soil, I have to haul it away just to make room for the next. More soil, more cows, more manure…the sun rises, the sun sets. More of the now-greying snowbank of dreams melts away.

And that's okay. If you, like me, feel like me (Hello, me!) know that that's okay.

That's life. Enjoy it.

Seriously.

Enjoy life. Whenever, however you can.

It may go well, it may go poorly. Sometimes Mother Teresa wins the medal in front of everyone, sometimes the medal gets melted down behind closed doors to help some 1% get 1%er, sometimes they're both in the wrong intersection at the wrong time.

You may shoot for the moon and end up sipping martinis with the stars. You may instead find yourself with another hole in your foot.

Sometimes life is hard, more often than not my dreams remain dreams or at best re-appear as someone else's successes.

But they were always dreams, never commandments. I cannot earn God's favor. Not by übermensching my strange and nerdy passions into success, not by dourly donating my very last beautiful dream to the food bank. I'll keep trying, I'll keep doing, but I've been given only a broad calling and I've been given only one specific guarantee.

"Go, eat your bread in joy, and drink your wine with a merry heart, for God has already approved what you do." — Ecclesiastes 9:7

On Monday I signed a contract to write a book about D3.js for Manning Publications. If all goes according to schedule, it should be out early next year, with draft chapters available in electronic form for subscribers even sooner.

Wow! It's certainly exciting, though I'm still worried over how much work it will be. I do get paid a useful advance at two milestones within the process, but I'll likely need to raise my consulting rates to compensate for my overall lost time.

I didn't say "yes" lightly, but I couldn't really say "no" either.

Why D3? The simplest answer is "that's what Manning asked for". But the longer I sat on their recruitment email, the more I realized that D3 is one of the tools I love and choose and use and even analyze a little every day, but rarely talk about. I'm not a visual artist so my use, especially my charts and graphs and even maps, always seemed a bit too unglamorous for sharing. Meanwhile D3 itself has been steadily and deservedly growing in popularity, and I'd certainly enjoy helping more eye-savvy creators learn to wield its technical side better. So I'm looking forward to sharing more about the library. A lot more.

Why a traditional publisher? Well, I've always been interested in writing a book but not terribly excited about the additional work of paid distribution. Signing a contract with a publisher obligates me to ship (and on time), gives me access to a lot of editorial support and experience, and significantly increases my chances of getting at least some financial return. In one sense, Manning is just a very low-paying new client, but they did offer me a particularly interesting product to work on.

In another sense, though, this is also still an investment. It won't really be "my" product (at least not until its market value has faded), but I will get prominent enough credit — and eventually maybe even a share of their ongoing profit should my work serve them particularly well. Technologies come and go, but D3.js would be difficult to outdo; I could see it staying around for as long as HTML and JavaScript themselves remain popular. Seems like as good an opportunity as any, eh?

Anyway, the book is a mere Table of Contents at this point so I won't persist boring you with further naïveté. Make room on your nightstand. 2014 will be here before you know it.