The kernel dispatch time eats a lot of performance on GPU – CUDA graphs let you chain a bunch of kernels together, and they’re now more accessible from PyTorch:

A blog post from PyTorch veteran and core maintainer Ed Yang based on a talk where he breaks down the fundamentals of PyTorch. From 2019 but still very useful in explaining the tensor mechanics, particularly including striding, which is one of those simple but very applicable concepts!

Thispaper from Tencent last year on the architecture of their recsys, and how it enables a high degree of freshness, via low-latency model updates to deliver fresh and relevant recommendations.

Microsoft published a paper about GPT-4, with the (literal) headline claim that it showed sparks of artificial general intelligence. The paper includes an approach to evaluate the model’s abilities, and many examples.

A longer link-and-rec, but a fascinatingpaper from ByteDance on one of their recommendation systems.

March 20 2023

Saw this referenced a few times, and hadn’t read it before.Short, and worthwhile.

Working on a team that provides a product or a service means deciding, regularly, what to do next.

Over the last couple of weeks I have been trotting out an observation based on an interesting conversation about software engineering teams with Jonny Dimond: mediocre work is the worst kind of work.

There is an old economics joke: two economists are walking down the street. One spots a $20 bill lying on the ground and points it out to the other. “Can’t be real” says the second economist, “or someone would have picked it up already”.