We're coming towards the end of my second quarter on github sponsors. Thanks to everyone who gave money to a stranger on the internet so that they could type more fun things and less boring things.
Here is what I did:
- Released dida, a library for streaming, incremental, iterative, internally-consistent computation on time-varying collections, based on differential dataflow.
- Everything appears to work, but there are few end-to-end tests.
- There isn't yet any memory management at all.
- There has been no benchmarking, profiling or optimization yet.
- It has javascript bindings, both as a node native plugin and as a wasm module.
- Wrote an in-depth explanation of how dida (and differential dataflow) works.
- Wrote Why query planning for streaming systems is hard. A quick overview of how query planning in streaming systems is very different from in databases, where the current standard approaches are likely to break down and some tentative directions for solutions.
- Wrote Making live repls behave. How to make an interruptible, non-blocking repl with memory limits, so that you can embed it in a web page or native app and execute as the user types, without getting stuck or crashing. Also features an embedded demo.
- Wrote Against SQL. An analysis of the most ingrained flaws of SQL and all the downstream complexity that it causes in the rest of the ecosystem. Also suggestions on how to do better in future database languages.
- Wrote 3 more editions of my sponsors newsletter. I'm aiming for 1 edition per month now, which means they've become much longer.
I also switched my sponsors page to pay-what-you-want as soon as github brought it out of beta, with a suggested amount of $20. I suspect the overall impact on my income will be negative, but I find the idea of tiers and rewards kind of exhausting. I'm just gonna make stuff.
Ideas for Q3:
- Main focus:
- Get dida to the point where it's worth using.
- Start work on interactive animations / debuggers for the explanation of how dida works.
- Distractions:
- Clean up imp and edit the accompanying posts so they're all describing the latest version of the language.
- Can I compile imp into sqlite query plans and use it as a sqlite frontend?
- Can I compile sqlite query plans into dida to get incrementally maintained views for sqlite?
- Staged query compilers in zig. I previously did this in julia and got pretty good results. I think it will work better in zig, and would make a good non-incremental backend for imp/dida.
- Write a proof-of-concept SQL debugger that works by sending each sub-expression to the database individually and showing the values of each as you mouse over the code.
Here is what my sponsorship looks like over time:
And the distribution of sponsorship amounts:
$1 x 1
$2 x 8
$5 x 9
$10 x 7
$20 x 37
$50 x 1
$100 x 1
$128 x 1
$200 x 3
I'm currently earning the equivalent of ~26k CAD per year. That's ~86% of minimum wage, ~81% of the mean phd income at UBC or ~58% (pre-tax) of my 2020 budget.
On the one hand, it's a tiny fraction of what I could earn in the industry. On the other hand, it's trending towards a full phd grant to pursue work that neither industry or academia has any interest in funding (there is no 'academic novelty' in better explanations or simpler implementations). That's worth something.