consulting lessons
I ended my consulting gig not long after starting, when it became obvious that we had very different ideas about how to write high-performance software.
I guess I can add this to my list of things learned about consulting. It's not enough to be clear on what I'm being hired to produce - we also need to make sure we're on the same page as to what producing that output will actually look like. I'll have to think about how to do that effectively.
The disagreement in this case was at least partly caused by my high rates. They wanted me to just focus on assembling the skeleton of the architecture and not waste time on testing/benchmarking/optimization, which they can do themselves later. But I don't think that design is separable from measurement/validation.
"Give us a sketch and we'll fill in the details" makes sense if the details are self-contained, but performance tends to entangle all the details together. The overall design needs to be determined by the details, making the skill is procedural, rather than declarative.
Anyway, I'd be much wealthier if I could just take peoples money and give them what they ask for without caring if I'm delivering something worth the cost.
there are no strings on me
The latest in this series of musings on language design for malleable software:
- The shape of data
- Implementing interactive languages
- Baby's first wasm compiler
- There are no strings on me
I have a whole pile of notes and ideas that have been slowly coalescing in my head over the last year or two. Now it's time to start pinning some decisions in place to constrain the rest of the design space.
I'm planning to spend at least 2024 working on this. Compared to my work on imp, zest has a much clearer purpose and the design problems are more in the 'adjacent possible'. Most of the pieces have already been tried elsewhere in some form, and the remaining few aren't critical.
I'll also return to sustaining that work through github sponsors. It'll likely never be enough to support us entirely without some consulting work on the side, but it certainly slows the bleeding. There's value too to the support and accountability, I think.
buttondown
Tinyletter closed down so I moved the email notifications to buttondown. It's more expensive than the alternatives but seems less likely to go on an incredible journey in the near future. So far it's been pretty pleasant to use - the interface is definitely focused squarely at programmers rather than marketers.
Let me know if you didn't get the email for this log :)
focus goof
I had a bug in focus that I've been lazy about tracking down. Every so often it would just crash. But weirdly it didn't leave a coredump. I also couldn't reproduce the crash in gdb, or even reliably reproduce it at all except to notice that it always happened when searching using ripgrap. So I've just been ignoring it for a while.
I mentioned this to a friend who suggested leaving strace attached to see if it was dying in the middle of a syscall or in userland code. So after a day or two it crashes again and I look at the log from strace and the last syscall executed is... kill. On it's own pid.
The victim was the murderer all along!
Here's the murder weapon:
const pgid = std.os.linux.syscall1(.getpgid, @as(usize, @bitCast(@as(isize, self.child_process.id))));
std.os.kill(-@as(i32, @intCast(pgid)), std.os.SIG.KILL) catch {};
This code is used not just for calling ripgrep but also compilers, unit tests, fuzzers etc. Many of which spawn subprocesses and don't clean them up properly when killed. So to run them I fork a child which immediately calls setsid to start a new process group. And when I'm done with them I kill the whole process group.
But there's a race condition here. If I run the above code before the child process has been scheduled and runs setsid, then pgid is the pid of the parent process and it kills itself, leaving no incriminating coredumps. And since it's a race, running it in gdb can alter the timing enough to prevent reproducing the crash.
The root problem here of course is that linux doesn't have a sane way to supervise processes. So here's my hacky workaround:
var my_pgid = std.os.linux.syscall1(.getpgid, @as(usize, @bitCast(@as(isize, std.os.linux.getpid()))));
var child_pgid = my_pgid;
// Have to wait for child to finish `setsid`
while (my_pgid == child_pgid) {
child_pgid = std.os.linux.syscall1(.getpgid, @as(usize, @bitCast(@as(isize, self.child_process.id))));
}
std.os.kill(-@as(i32, @intCast(child_pgid)), std.os.SIG.KILL) catch {};
jsfuck
Javascript is definitely haunted.
> deno
Deno 1.33.3
exit using ctrl+d, ctrl+c, or close()
REPL is running with all permissions allowed.
To specify permissions, run `deno repl` with allow flags.
> [][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]][([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]((!![]+[])[+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+([][[]]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+!+[]]+(+[![]]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]+(+(!+[]+!+[]+!+[]+[+!+[]]))[(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([]+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]][([][[]]+[])[+!+[]]+(![]+[])[+!+[]]+((+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]+[])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]]](!+[]+!+[]+!+[]+[!+[]+!+[]])+(![]+[])[+!+[]]+(![]+[])[!+[]+!+[]])()(([]+[])[(![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(!![]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]()[+!+[]+[!+[]+!+[]]]+(+(+!+[]+[+[]]+[+!+[]]))[(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([]+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]][([][[]]+[])[+!+[]]+(![]+[])[+!+[]]+((+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]+[])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]]](!+[]+!+[]+[+!+[]])[+!+[]]+(!![]+[])[!+[]+!+[]+!+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(+[![]]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+!+[]]]+(+(!+[]+!+[]+!+[]+[!+[]+!+[]]))[(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([]+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]][([][[]]+[])[+!+[]]+(![]+[])[+!+[]]+((+[])[([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(![]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[])[+!+[]]+([][[]]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]+[])[+!+[]+[+!+[]]]+(!![]+[])[!+[]+!+[]+!+[]]]](!+[]+!+[]+!+[]+[!+[]+!+[]+!+[]])+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]+(![]+[])[!+[]+!+[]]+([][[]]+[])[!+[]+!+[]]+([]+[])[(![]+[])[+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+([][[]]+[])[+!+[]]+(!![]+[])[+[]]+([][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]]+[])[!+[]+!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(![]+[])[!+[]+!+[]]+(!![]+[][(![]+[])[+[]]+(![]+[])[!+[]+!+[]]+(![]+[])[+!+[]]+(!![]+[])[+[]]])[+!+[]+[+[]]]+(!![]+[])[+!+[]]]()[+!+[]+[!+[]+!+[]]])
"hello world"
1ML - Core and Modules United
This has been on my reading list forever. Finally got around to it.
It's going to take a few more readings to really get a grip on it. My rough understanding at the moment is that it's in some ways quite similar to zig. The big and important difference is that types are opaque, so the result of a type-constructing function can't depend on the input type except parametrically. This means that type checking can reason about the result abstractly before specialization, so we can type-check functions rather than just specialization of functions. And have good IDE completions etc.
But a lot of useful code in zig depends non-parameterically on input types! For example, MultiArrayList automates the conversion from array-of-structs to struct-of-arrays.
After some prompting from matklad I don't think this is completely irreconcilable. In:
fn MultiArrayList(comptime T: type) type {
...
}
The ... constructs the type and the return type of the function is just type. But most of the public interface of MultiArrayList(T) is parametric in T (if we allow some associated-type-like helper functions for introspecting the type). It's only the internals that are non-parameteric. So in some hypothetical 1ML-zig we could write:
fn MultiArrayList(comptime T: type) MultiArrayListSignature(T) {
...
}
fn MultiArrayListSignature(comptime T: type) type {
return struct {
fn init(Allocator) Self
fn get(Self, usize) T
fn items(Self, comptime field: @fieldOf(T)) @fieldTypeOf(T, field)
}
}
And then everything in MultiArrayListSignature is parametric in T, so the type checker can reason about it before specialization.
We're probably still going to run into limitations in places (eg can you express the type T where @fieldOf(T) == @fieldOf(U)?). One of the really nice things about the zig/julia approach to types is that it's incredibly simple. You only ever have to reason about concrete types. Whereas with 1ML I'm already unsure exactly what kind of code I'd be allowed to write. So I'm not sure I want to think too far down this path. But I'll keep it in the idea stew.