HYTRADBOI program and tickets
...are here. I'm very happy with how the program turned out.
I have 20 talks uploaded and transcribed. I'm still waiting on recordings for the last 6 talks, but I've seen drafts for most of them so I'm not too stressed. We may also get 1 bonus talk added to the program, if we're lucky.
Also we've sold 174 tickets so far and there are 174 names in the chat. And I only had to answer one support email. So the infrastructure is working smoothly so far.
There is still room for more lightning talks. You should submit a lightning talk! All the cool kids are doing it.
query compilers
I wrote a thing about the missing tier for query compilers.
Fun facts from the comments:
- In SingleStore parameterized queries are planned ignoring the values of the parameters. This allows caching the compiled query but prevents adapting the query plan - for the example in the post SingleStore would pick one plan for both queries. But you can opt out of this behaviour by wrapping each parameter in NOPARAM.
- ClickHouse is a vectorized engine but still jits scalar expressions and order-by/group-by/aggregates, the two big pain points for vectorization. They report speedups for 1.5-3x (presumably only for that operator, not for the whole query) at the cost of 5-15ms of compilation. So there is room for gains from compilation even in a very highly optimized vectorized engine!
I'm kind of pondering writing a query compiler for postgres. The existing jit is pluggable and the main reason people don't use it is the compile times. Compatibility seems tricky though, especially across multiple versions of postgres and against various different extensions which provide their own jittable functions. I'm also unsure what the market for postgres plugins is like, since cloud hosted postgres instances typically don't allow 3rd party plugins.
decorrelation encore
Neumann has a new paper about improving their decorrelation algorithm. It turns out that applying their algorithm one-by-one to nested queries in a bottom-up fashion creates a nasty edge case that creates unnecessary product operations. They avoid that in the fixed version by applying the algorithm top-down in a single pass, combining the context for multiple nested queries as they reach them.
I was struggling to follow the paper at first, and then realized that the reason I'm confused is because the decorrelation algorithm I used in materialize already works like their fixed version. I came up with that approach by extending the partial solution described in sql server in 2001 and when I read Neumanns 2015 paper I assumed it was the same thing, and that the fact that I wrote it top-down was just an inconsequential implementation decision. But I talked to one of the umbra phd students last week and ran their example query in both materialize and the new umbra version, and they produce essentially the same product-free plan now.
So on the one hand, cool story, lucky decision. But on the scarier hand I didn't even realize I was making a decision or that the order mattered. Not sure how to get better at spotting that kind of thing.
And materialize still sometimes creates products during decorrelation in other queries, but that is apparently a separate problem related to a separate optimization that destroys information about column equivalence.
books
How to change your mind. Spends too much time on the authors own psychedlic experiences, and felt too credulous when talking to researchers. But I liked the history of psychedelic research and getting different points of view on why it stalled out.
Meditations for mortals. Anti-productivity-ish musings. Unobjectionable, but doesn't add much to his previous book.
Not the end of the world. I really appreciate the nuance and the focus on effective action. Also surprised at how optimistic the picture is on some fronts eg air pollution, ocean plastic. Definitely recommended.
The uncontrollability of the world. The abstract was promising, but the writing is typical academic philosophy. Lots of references to famous philosophers, no concrete predictions or falsifiable claims.
Brief flashings in the phenomenal world. About the author recovering from shattering her leg in a rafting accident and going on to win an ultramarathon. But also mostly about zen. I wanted to like it but it never veered far enough away from wellness cliche.
misc
Polars has subqueries, although they don't call them that. I wonder if they decorrelate them?
TigerBeetle found a UAF. They called it their first, but I think the manifest race also counts as a UAF.