Jim Nielsen’s Blog

Marques Brownlee just released a video essay titled “Dear YouTube”.

I loved it.

He talks about how YouTube is going to start shipping this new feature where makers can essentially publish different “variants” of their videos and see which performs best (an A/B test).

Which immediately begs a lot of questions. Brownlee voices his:

• If I’m watching a video, do I know if it’s part of one of these A/B tests?
• If I’m reading the comments of a video, how do I know I saw the same thing as another commenter?
• How do linked timestamps work?

It’s pretty wild when you think about it. I mean, imagine sending someone a link, “Check out this cool video!” And at that point you’re basically crossing your fingers, “I hope they see the same thing I did…”

Marques says he spoke to some YouTube engineers about these questions and they didn’t really have answers. They were kind of just like, “We don’t know — yolo! This’ll ship soon.”

So his video is a kind of plea to YouTube creators everywhere: Don’t use this feature!

There’s this sort of unspoken rule of community on YouTube, that like we’re all having this same experience together — we’re all watching the same video. That is what makes it a cultural phenomenon is that we all saw the same thing […] so letting people put up two or three different videos of the same thing at the same time kind of fundamentally breaks that, which doesn't feel right.

I agree, 100%.

Brownlee goes on to make a lot of great points in the video that, I think, can be applicable to making software as well. A few that stood out:

• The skill of being a “creator” is not about maximizing views, retention, etc. It’s about finding creative ways to share stories, teach concepts, explore ideas, etc. Views, retention, etc. are all downstream of that.
• Making (and finishing) something over and over and over is a better use of your time than making one thing and then trying all kinds of variants and obsessing over their varied performance.
• When you focus on optimizing, you end up trimming out all the creative (and fun) stuff because those things won’t, by definition, ladder up to metrics of efficiency. (Moby Dick was not the most efficient way to tell that story.)
• Stand by your creative choices. Learn from your wins and failures, then make the next thing. Have some conviction.

I find it interesting how Brownlee shares his opinion that YouTube spends way too much time chasing their competitors and not enough time being YouTube.

Which, when you think about it, is exactly the kind of place a feature like this would stem from: an insecurity in your own creative choices. How YouTube approaches its own insecurities is now trickling down as a feature to its users.

“Let’s let people make lots of variations on things, try all of them, and see what performs best” is exactly the kind of thinking you get in a platform that doesn’t know what it wants to be. So it’s left spending its time 1) doing what others are doing, and 2) following the fickle whims of whatever it can measure.

It’s like this adolescent attitude of, “I don’t know what I want to be, so I’m gonna imitate my peers and let others tell me what they think I should be.”

Anyway, that’s a long way of saying I liked the video. You might too. Hopefully you see the same thing I did, lol.

Reply via: Email · Mastodon · Bluesky

Confession: I got nerd-sniped by Sam Henri Gold’s request for my icon galleries:

I'd like to humbly request artwork-level searching in macosicongallery.com

What follows is a train-of-thought blog post as I play with what an implementation might look like.

I’ve actually long-wanted something like this, e.g. let me search for “coffee” and show me all icons that have some depiction of coffee in them.

Similarly, I’ve wanted some kind of “related” representation for icons. I have this today via existing metadata, e.g. “Show me other icons in the category ‘Productivity’” or “Show me other icons tagged as ‘orange’”. But I’ve wanted a more robust representation of this, so if you were looking at an icon that had a microphone in it, the site would say “Here are other icons that also have microphones in them.” And the relationship would be rich/smart enough to know that “microphone” was meant broadly, i.e. dynamic mics, condenser mics, ribbon mics, etc.

So how would you do this? I could go through every icon one-by-one and classify/tag any attribute of its design that comes to mind, but that would take ages! Seems like a good use case for a vision model.

Trying CLIP

First, I’ll look at Sam’s suggestion: run every image through CLIP.

I’m not familiar with CLIP so I start with a little research: What is it? How would I use it? And most importantly: is it free/open (because I ain’t spending a ton of money to send my thousands of icon PNGs to an AI provider via their API)?

Ok, so CLIP will take an image and spit back an embedding (basically a bunch of numbers representing features of the image). When you do it with multiple images, you can then compare those embeddings to see what the model considers similar (and, if you like, set a threshold for what constitutes a “match”).

After getting a sense of the task in front of me, I work with the LLM to come up with a proof of concept. I don’t need to fit this into my existing site. I just want to make one-off HTML pages where I can feel out, “Can this process create anything useful? What’s the amount of work required?”

• Write a script that runs a sampling of icons through CLIP’s image encoder
  • Read the file locally, e.g. ./ios/256/${icon.id}.png
  • 256x256 pixel icons seem to be enough, as the CLIP model I’m using preprocesses them to ~224px anyway.
• Create a dataset representing the “embeddings” (an array of numbers) for each icon that I get from CLIP, e.g. Array<{ id: String, embedding: Array<number> }>
• Create a dataset representing the top matches between different embeddings, e.g. { [id: String]: [id, id, …] }
• Create a clip.html file has both datasets (plus supplementary icon metadata I already have), render all the sampled icons, and support an onclick for each icon that shows the related[id] icons.

This is enough to create a single HTML file where I can click on an icon and see other icons that look like it.

However, I realize quickly that I’ll need to process my entire icon library to really get a good sense for how well these are matching. So I do that.

[Computer goes brrrr…]

Ok, now when I click on an icon that looks like a camera, I see other icons that look like cameras.

Or if I click on an icon that has a checkmark in it, I see other icons with checkmarks in them — sort-of.

But the results aren’t that great unless an icon is visually distinctive. I share some thoughts with Sam. He has a few other suggestions I follow.

DINOv2, SigLIP2, and More

Sam mentions SigLIP2 so I start with that as a keyword. The LLM recommends DINOv2 so I say, “Let’s try it”.

I give that a try, creating a separate dataset and prototype (e.g. embeddings-dinov2.json and embeddings-dinov2.html) so I can continue to view these different prototypes and compare their outputs.

It’s fine. Different from CLIP. Honestly not much better.

So I figure let’s try another one. I go with SigLIP2. I ask the LLM to create a page where I can compare the results.

Seems like six of one, half dozen of another. One does better on some kinds of icons, worse on others. The LLM recommends that, at this point, I be done shopping models. They’re roughly the same class of tool with different tradeoffs. None are breakthroughs.

So now what?

Try Tagging Icons With Keywords

Sam recommends another approach:

You could also try handing all icons over to a VLM, having it write up a description, and embedding THAT text against what people might search for.

A thoroughly detailed person might’ve done this from the start, e.g. for an icon that’s a checkmark, add the keyword “checkmark” to its metadata.

That would take me forever to go back through all my icons and do — a perfect task for a computer that never tires.

So I give this a try. First I need a free/open VLM. After a little research I decide to try Moondream via Ollama.

I have the machine go through each image and caption it, then pull out “tags” from the caption. For the Clear app icon, I get data like this:

{
  "id": "clear-todos-2021-01-10",
  "caption": "The image features a red and orange gradient background, with a white checkmark in the center. The checkmark is slightly tilted to the right, giving it a dynamic appearance. The background transitions from red at the top to orange at the bottom, creating a sense of depth and movement. The checkmark is the main object in the image, occupying most of the space and drawing attention to itself.",
  "tags": [
    "red",
    "orange",
    "gradient",
    "white",
    "checkmark",
    "slightly",
    "tilted",
    "dynamic",
    "appearance",
    "transitions",
    "creating",
    "sense",
    "depth",
    "movement",
    "object",
    "occupying"
  ]
}

Then the LLM creates a single search.html file where I can test icon matches by searching for tag overlaps (or choosing one of the popular ones).

So, for example, on the search page I can click on “checkmark” and see all the icons with a checkmark.

Or click on “fox” and see all the icons with a fox.

Matches are pretty spot to be honest.

But that’s a different kind of test than what I was doing with CLIP.

• CLIP: click on an icon and see other icons like it.
• Tags: click on a keyword and see other icons with that keyword.

Can I leverage tags for the same kind of “related icons” work that CLIP is doing?

CLIP vs. Tags

I get the LLM to cook up a single-page HTML file where I can compare “click on this icon and find other icons like it” where I’m using embeddings from CLIP vs. matching on keywords.

The results seem to fare much better for CLIP. For example, here I matched on what I think of as a “checkmark icon”.

You can see the approach that matches on tags didn’t work too great. I believe this is because with my simple tag-overlap approach, a distinctive keyword like “checkmark” gets diluted amongst generic tags like “square”, “blue”, and “simple”.

Whereas with CLIP, if you click on an icon with a checkmark, you get other checkmarks (and not other icons that also have related tags like “square”, “blue” and “simple”).

Which all makes sense. Pushing on the implementation here could help, but that’s separate work to do.

So Now What?

I’m not sure.

While doing all of this was an interesting technical exercise, there are a few important considerations I need to think through before implementing anything, such as:

• What kind of functionality do I actually want?
  • A “related icons” feature? Does it match on keywords or embeddings?
  • A “search” feature that matches on keywords?
  • Both?
• How do I build these features into my codebase now, given the thousands of icons that already exist?
• How do I maintain this feature in the future?
  • e.g. every time I add a new icon to my gallery, is a VLM now a dependency of this project?
• Given all the above, what’s the time and money cost?

I’m very picky about adding new dependencies to these icon projects. I like to think that’s why I’ve been able to maintain and continue contributing to them after so many years — because I make it easy on myself (good job, Past Jim).

So, for example, if I make a VLM a dependency of this project such that every time I add a new icon I have to run it through to create the embeddings, that’s a big dependency cost IMO. I’m not sure I want to do that.

That said, Apple now ships foundation models in macOS 27 available through the CLI (go ahead, try typing fm in your Terminal if you’re on Golden Gate). So if my Mac continues to be the primary machine where I add/update metadata for my icon projects, using fm would be a really easy/low-cost way to process each new icon to generate a caption and keywords for matching in search.

But again, I don’t know if I want to do that. I wrote this post to try and work through what I want to do, but I am still undecided.

So I guess the only thing for me to do at this point is hit “Publish” on this post and keep simmering on a decision.

Reply via: Email · Mastodon · Bluesky

So there I am reading Unsung when I come across this quote from Jef Raskin:

An interface is humane if it is responsive to human needs and considerate of human frailties.

This articulation perfectly captures why I recoil at modern “growth tactics” in user interfaces and experiences: they are designed to exploit human frailties, not be considerate of them.

Stated again: a humane interface — or a humane system, even a humane technology — is considerate of human frailties. It acknowledges they exist and works with them, rather than taking advantage of them.

Humane technology does not rely on impulse, addiction, or imposition to survive.

It survives purely on the free choice of its user.

You choose to use it because you have the freedom not to.

Reply via: Email · Mastodon · Bluesky

Whenever a new version of macOS drops, I find myself archiving the new app icons for my macOS icon gallery.

In the past, the work to do this was a bit tedious.

First, I’d find the app in the “Applications” folder (or elsewhere on the Mac), right click and choose “Get Info”:

Right-click the app icon in the Get Info pane and choose “Copy”:

Open the Preview app and go “File -> New from Clipboard”:

This gets me the app icon file with a variety of sizes (the 1024-pixel version is the one I want). From here, I go “File -> Export…”, choose “PNG”, and name the file as the app name, lowercased, hyphenated — e.g. “app-store.png” (I do this for automation purposes not relevant here).

Doing that for every Apple-supplied app in the OS can get tiring. It’s often upwards of 75+ icons.

“This seems like the perfect task for an LLM,” I thought. So I hastily wrote a prompt to see if it could do it.

And it did — sort of, after a little coaching.

On the first try it found an app’s .icns file, grabbed the 256×256 version, upscaled it to 1024×1024, and saved it.

So I scolded it, “No, no. NEVER upscale an image. Always find the biggest one. I can get the 1024 version myself from the ‘Get info’ pane, so surely you can find that size somewhere.”

It came back dutifully, having found the right image, and said it would never upscale again.

Satisfied it could do what I wanted, I worked on creating a list of all the apps whose artwork I wanted to archive. Then I gave that list to the LLM and said, “Do your thing.”

And it worked! It was kind of wonderful. Saved me a ton of time.

76 app icons found, extracted, and saved in much less time than it would’ve taken me to do this “by hand”.

Reply via: Email · Mastodon · Bluesky

I was listening to the episode “Make Your Own Thing” from the Off Protocol podcast where Jim Ray interviews John Gruber.

They cover a variety of topics, including Gruber’s thoughts on the dilution of personal brand by publishing through “chain” platforms like Substack (I quite enjoyed the metaphor of a personal website where you publish your stuff to that of a chef/restaurant, but that’s a separate post).

Around ~1:13:00, Gruber quotes Walt Disney who said, “We don't make movies to make money, we make money to make more movies.” And he goes on to argue that when you’re forced to choose one of these priorities over the other — because you will undoubtedly face decisions where the two sets of values come into direct opposition — the company that prioritizes making movies over making money is fundamentally a different company than the one that chooses making money over making movies.

If you ultimately prioritize making money, then what’s going to be ultimately great about your company is you make a lot of money (for shareholders, for owners, for yourself, for whomever).

Conversely, if you ultimately prioritize making movies, than what’s going to be ultimately great about your company is your movies. Hopefully that makes you a lot of money, but it might not. One day people might say, “Damn, they made great movies. Too bad they’re not around anymore.”

To be fair, the prioritization of those values shifts and changes over time. Sometimes companies prioritize one over the other based on circumstances, e.g. “We’re gonna go bankrupt soon, we have to prioritize making money in these particular areas or we won’t be around long enough to keep making money.”

I’m not saying there’s an easy answer in those situations, I just find the opposition of those priorities intriguing. Ultimately, you do have to choose something over something else. It’s not necessarily a one-time decision, but the outcomes are a function of that prioritization over time.

Gruber believes Apple still generally prioritizes users most, i.e. in most cases, most of the time:

I think it’s true that the second priority for [Apple] is to make as much money as possible, but it’s the second priority. Every time [that] priority kinda creeps ahead […], like the way they’ve shepherded the App Store and the commissions they make from it, is arguably an area where they’re driven more by, “Let’s make as much money as possible because we control the App Store.” Rather than, “Let’s make the App Store as good for users and developers as possible.” And I think that’s the reason we complain about it, of all the things Apple makes [the reason people who enjoy the company’s products still complain the most about] that area of the company is because the priorities are clearly in the other order.

It does feel like so much of tech today is people asking, “What can we make for others that’ll make us a lot of money?”

But that’s a different prioritization than, “Let’s make something great for ourselves and, along the way, let’s see if we can make some money in order to keep doing what we love.”

Reply via: Email · Mastodon · Bluesky

I made a note of it, but Bryan Cantrill wrote a post about the ex-Anthropic employee who pegged the probability of AI killing humanity in the next decade at >10%.

The article is a good one and you should read it, but what I want to capture here is a side story from Cantrill on the Oxide and Friends podcast where he talks about how proud he was of this phrase from his post: “These ghoulish claims strike brazenly at the hearth…”.

He notes that someone online commented on that phrase, saying it sounded like a mix of classic literature and a line from The Simpsons.

Amused, Cantrill says he came up with the line all by himself, so he knew it wasn’t a classic literature reference nor a line from The Simpsons.

But just to be sure, he Google’d it: “These ghoulish claims strike brazenly at the hearth” with “The Simpsons” tagged on to the end.

To his surprise, Google’s AI told him that the line was, in fact, a line from The Simpsons. Here’s Google’s response (as reported by Cantrill):

During the Springfield mayoral debate on Channel 6, Sideshow Bob dramatically delivers the line “these ghoulish claims strike brazenly at the hearth of Springfield” while turning the debate against Mayor Joe Quimby. Instead of denying his criminal history, Bob exploits Quimby’s record and boldly tells the audience they secretly long for a cold-hearted Republican to rule them like a king.

(Here’s my homemade graphic for illustration purposes.)

Cantrill was astonished. Not knowing the episode enough by heart to discern whether this claim was true, he had a little bit of a panic attack:

If I am having what I feel to be original thoughts that are, in fact, just Simpsons lines regurgitating from 30 years ago, that’s troubling […] How am I not an LLM?

So he set out to find the truth. He finds the episode and watches it, but doesn’t hear the line.

But what if the episode he watched was an edited version and the full-length one is out there somewhere? He’s gotta get to the bottom of this.

So he goes back to Google and pushes on the AI’s answer, asking for the full script of the episode. At that point, the LLM folds and says the line is not from The Simpsons, but is actually from a post by — drum roll… — Bryan Cantrill.

Here’s Cantrill, again, reporting Google’s AI response:

What happened here is a classic AI “hallucination”. Because the phrase you shared used highly stylized, melodramatic language, “ghoulish claims”, “strike brazenly at the hearth”, my system misidentified the tone as belonging to Sideshow Bob who is famous for speaking in exactly that kind of grandiloquent Shakespearean style.

I have to say, I found the whole story amusing and relatable.

• You write something on the internet.
• Somebody says, “This sounds like something from the Simpsons.”
• You wonder, “Do I even have original thoughts?”
• AI gas lights you.
• You have to spend your time refuting AI assertions.
• Some teenage part of you gets a kick out of being conflated with Sideshow Bob for your choice of language.

I just wanted to write this all down and save it as a way for future me to remember, “This is what it was like to be online in 2026.”

Reply via: Email · Mastodon · Bluesky

John Gruber makes his picks for the iPhone Hall of Fame:

1. iPhone (original)
2. iPhone 4
3. iPhone 5S
4. iPhone X
5. iPhone 17 Pro

Ok, my turn. Here’s my iPhone Hall of Fame:

1. iPhone (original)
2. iPhone 4
3. iPhone X
4. iPhone 12 Mini
5. iPod Touch

Notes:

• How do you not put the OG iPhone at the top spot? It’s like doing a Mount Rushmore and not putting the first president first. It’s gotta be George.
• Aesthetically, the 4 was the best. Just an absolute beauty to behold (and use, so long as you held it right).
• Aesthetically, the X was a close second to the 4. But in terms of form, X was best. (Shout out to the 3GS, that curved back felt great in the palm.)
• The Minis (12/13) stand as proof that commercial success doesn’t always equate with greatness. Ahead of their time, those ones.

Oh, and you’re probably wondering: “The Touch? That’s not even an iPhone!”

I disagree. The Touch was The iPhone For The Rest of Us™.

When you couldn’t get one of those early iPhones (because of price, because of geographic location, because of ___) the Touch was your gateway drug.

For me, it was price. I was a poor kid in college in 2007, and the iPhone was $$ (and it was on AT&T, which was even more $$).

Eventually, I was able to get an original iPhone on eBay (and jailbreak it for T-Mobile — remember Cydia?). But the Touch was my first taste of iPhone.

That’s my Hall of Fame list. What’s yours?

Reply via: Email · Mastodon · Bluesky

“Artificial” things can get a bad rap next to their counterparts, but they have their place.

For example, artificial light (i.e. not from the sun)? Super useful, especially at night.

And when it comes to skiing, I’ll take artificial snow if I have to. Beats no snow.

Artificial isn’t necessarily inferior in all cases, everywhere, all the time. But it is optimized for different values. For example:

• Artificial grass (i.e. turf) is great for certain contexts, but I don’t want a golf course of artificial grass.
• Artificial flowers are great for certain scenarios, but I don’t want them at my wedding.
• Artificial flavoring is yummy (love me some peach rings), but I’d prefer a real peach pie over one with artificially flavored filling.

Artificial is an alternative. It’s understood as a trade-off, not a substitute.

You trade the real for the artificial when you’re concerned about attributes like cost, convenience, and availability.

Which brings me to “artificial” intelligence: surely a trade-off, not a substitute.

Ain’t nothing like the real thing.

These labs should consider ripping off (cause they’re good at that) Mastercard’s old tagline: “There’s some intelligence money can’t buy; for everything else, there’s Claude.”

Reply via: Email · Mastodon · Bluesky

Poor bottlenecks. Always seen as problematic, antithetical to efficiency.

But bottlenecks aren’t universally bad.

Think about it: a bottle’s neck is designed to constrain the amount of liquid that can flow out. A decrease in bandwidth is its entire purpose! Otherwise an overwhelming amount of liquid flows out and makes a big mess.

We humans have a particular anatomy. We can only consume so much liquid at a time. The neck of a bottle works with that fact.

We could make machines to produce so much wine that we’re drowning in it. But that wouldn’t change the fact that we’re only capable of consuming so much liquid at a time (not to mention digestion, etc.).

When it comes to liquid consumption, the bottle’s neck isn’t the bottleneck — our neck is!

So if you’re having a hard time drinking out of a firehouse, perhaps the question isn’t, “How do I modify my biology to accommodate the bandwidth of the firehose?” But rather, “Why am I trying to drink out of a firehose in the first place?”

Maybe a bottleneck isn’t your problem. In fact, it might just be the solution.

Reply via: Email · Mastodon · Bluesky

I’ve got Netlify Analytics turned on for a few of my websites.

I use it send myself a little daily digest that alerts me when folks link to my writing, be it an individual sending me a few referrals, a newsletter sending me a few hundred, or Hacker News sending me thousands.

But sometimes I want to know more than, “What happened with traffic in the last 24 hours?”

Sometimes I want to know, “What happened with traffic in the last month?” I can log in to Netlify to answer that question.

Sometimes I want to know, “What happened with traffic in the last six months?” Netlify doesn’t have an answer for that because they only store the last thirty days of traffic data.

I’ve known about this limitation for years and always thought, “I really should just start saving that data for myself.”

But, you know, laziness. How bad do I really want it? The answer, it seems, was “I only want it if it’s _easy_”.

Enter Val Town.

Val Town makes this kind of task easy. Each “Val” is it’s own little bucket of scoped computing primitives for just such a task:

• Code (run a script that fetches data)
• Secrets (securely store my Netlify access token)
• Storage (my own little SQLite database for persisting data over time)
• Automation (run this task every 30 days)

All of this might seem like overkill for a single-script task like mine. But I like it. The scoped nature of each of these primitives makes composition easy without the kind of entanglement I’d have to fight to keep isolated if I were co-locating this task with other small tasks.

I don’t want to think about a shared schema for this task and whatever other tasks I might think of in six months. I don’t want to have to reason about evolving an architecture and maintaining a codebase of miscellaneous, unrelated tasks. Vals are perfect for this.

So the way I have this setup is: run the .ts file every 30 days which fetches the last 30 days worth of analytics data from Netlify and stores it in SQLite. (You can see my Val here.)

At that point, I have the data I need to curiously ask any question, e.g.

• What are the top referring sites to my blog?
• What post has received the most page views over time?
• Where does most of my traffic come from?

The answers to these questions get more and more interesting over time, as the long tail of web dynamics play out: search engine traffic, serendipitous rediscovery of old posts in light of new trends, etc.

“But how do you actually view the data you’re storing to get answers to these kinds of questions?”

Great question. Val Town has a UI where I can see the raw data I’m storing (which I chose to store exactly as I got it from Netlify, rather than doing some pre-processing first):

But that’s not very useful. I need to parse that data and turn it into some kind of useful analysis.

And you know what tool is super useful for understanding data? A spreadsheet.

If I were using Excel, I’d probably have to figure out a way to dump the SQL, convert the JSON to some kind of CSV, then import it. And every time I wanted to see the “latest” data, I’d have to run that workflow.

Enter Quadratic.

Val Town has an API where I can fetch the latest data via code, and Quadratic is a spreadsheet that runs code. So seeing “the latest data” from my pipeline in Val Town is as easy as pressing a “re-run code” button in Quadratic.

Quadratic serves as a kind of “front-end” to my data pipeline. Easy to pull the raw data, transform it however I need, and create a “dashboard” I can revisit whenever I want to see the latest insights or ask new questions.

And, since I’m storing data for a couple different sites in Netlify, I have a single cell in my spreadsheet where I can swap out the domain — say from blog.jim-nielsen.com to iosicongallery.com — and the entire dataset and dashboard redraw.

I know, I know. There are lots of products for stuff like this. And there’s probably a million other ways to do what I’m doing, all peculiar in their own way. But this one is mine.

And now I can sleep at night knowing that every 30 days my little Val Town robot is gonna go store the last 30 days of data that Netlify is throwing away.

One person’s garbage is another person’s treasure trove of data.

Reply via: Email · Mastodon · Bluesky