I'm currently doing some consulting, adding a vectorized query engine to a project written in go. I wrote a thing - columnar kernels in go - about some prep work I did before starting, figuring out how to translate familiar idioms into this foreign (to me) language.
go grouches
So far I'm extremely unimpressed with go.
It disallows using compound types as map keys, requiring javascript-like stringifying workarounds (I already wrestled through some code that should have just been a set of lists of keys). A restriction I could understand in the name of safety, if the rest of the language wasn't casually littered with footguns.
Implicit zero initialization, manual error checking, mixing implicit variable assignments with variable declarations (a, b := ... either declares two new variables, or declares one new variable and assigns to one existing variable).
Nil slices behave like empty slices, except when they don't (I already found a bug in existing code caused by this). Slices are passed by value but contain a reference to an underlying array. Calling append on a slice only does what you expect if the slice happens to point to the end of the array.
Changing a declaration from private to public requires capitalizing the name, causing pointlessly large diffs which conflict with unrelated changes.
Two things are really biting me though:
- The only way to create a sum type is to use an interface, which requires a heap allocation.
- Generic functions, other than language builtins, are implemented by dict passing.
The combination of these two mean that almost any abstraction that I make has a performance cost. Within the data-plane that cost is often unacceptably large. The result is that much of the code I'm writing is boilerplate workarounds for the complete lack of zero-cost abstractions.
Column Sketches: A Scan Accelerator for Rapid and Robust Predicate Evaluation
Goal is to accelarate column scans regardless of data distribution and workload.
Build a histogram of data. Divide into evenly full buckets. Give each bucket a short binary code.
Evaluate predicates against buckets first, and then scan for matching buckets.
Their experiments demonstrate good performance across different query selectivities with 1 byte codes over uniformly distributed 32 bit integers.
Performance remains identical under data skew, while bitweaving degrades.
Builds much faster than bitweaving too. (Note that BtrBlocks builds histograms anyway - might be able to produce the sketch almost for free.)
Why User-Mode Threads Are Good for Performance
Makes a really simple argument: If each task spends 1% time on cpu and 99% stalled, then we need at least 100 threads per core to saturate cpu. In a microservice architecture with high fanout, these numbers get bigger and bigger until the per-thread memory cost is the bottleneck. So the main importance of green threads is not reducing the cost of context-switching (which has been dramatically reduced in recent linux versions) but in reducing the memory cost per-stalled-task so that the fanout layer needs fewer machines.
I don't know enough to decide whether or not I believe that argument, but it's interesting.
Corporate Open Source Anti-Patterns: A Decade Later
I was really struck by the idea that investors pursue gross margin and ignore net margin, and that this is poorly compatible with building a business around open-source software, which by design doesn't capture most of the value it produces.
Managing a Managed Language in a Managed Language
A memory-safe garbage-collected language whose runtime is implemented in itself, via a minimal set of escape hatches.
Celebrity worship is weird and will only get weirder
Celebrity is becoming a requirement for some jobs and most sane people aren't built for it. [...] I'm a full-time author and in this media environment, the only way to reliably sell books is to become some form of celebrity. [...] I literally cannot write novels as a full time job unless I turn myself into a multimedia influencer that posts daily to a large, loyal, highly-engaged audience. [...] The rest of my writing career doesn't rest in my ability to generate story ideas or execute them - plenty of incredible novels sell fewer than a hundred lifetime copies - but in doing that extremely well and also making my personality and lifestyle interesting to the masses. I must become something other than a person, a symbol that can act as a receptacle for strangers' excess emotions, including their random, ravenous hatred.