Jim Nielsen’s Blog

In his talk “Why AI Is Breaking Software Security As We Know It” (my notes here), Feross Aboukhadijeh talks about the Axios npm incident and how the maintainer got phished by succumbing to (amongst other things) a faux Microsoft Teams interface:

this is the kind of thing that AI makes easy to do, because it can vibe code that whole fake Microsoft Teams interface pretty trivially

You’ve probably seen these: interfaces designed to look like some other product in order to provide a facade of authenticity and exploit someone.

What struck me in listening to Feross was this idea of how the quality of your interfaces can be a protection mechanism against attackers.

I don’t know if I’ve ever heard someone say that out loud — interface and interaction design as a security control — but I’m saying it.

Now, of course, not everyone will consciously notice the level of polish that world-class professionals imbue in digital interfaces. But some will.

Personally, I’ve always used the quality and care of digital experiences as a heuristic for judging authenticity — and competency to be honest, e.g. “If this UI is so bad, what else will surely be bad?”

Granted, it was a much more dependable heuristic before AI came along. But even now, I can still suss out slop and carelessness which is a skill that continues to be a reliable, protective form of digital literacy (for me).

That’s all to say: a sloppy, careless approach to interface design not only hurts your brand in terms of customer perception, but it can be an attack vector. The easier it is to sloppily reproduce what you sloppily ship, the easier it will be for your product or brand to be leveraged as a vehicle for exploiting your customers.

If everything you make was produced from a single prompt, then everyone else is one prompt away from imitating you. The easier something is to make, the more likely it’ll be in the genre of “easy to exploit”.

One way to protect yourself (it’s not the only one way, security is never a binary “you are / are not secure”) is to do that extra work to make your experiences go above and beyond what you can easily get out of an LLM.

The protection here is having an interface and experience that is hard to replicate with the same level of fidelity that discerning users will notice — things like micro-interactions, loading behavior, UI copy and voice, handling of edge-cases, etc. That’s the stuff that’s hard (and expensive) to fake because it’s hard (and expensive) to notice you need to fake it.

tl;dr — Fidelity to craft is not only valuable from a product standpoint, but it’s also valuable from security standpoint. If attackers are going after low-hanging fruit, your fruit will be harder to reach if it’s up high.

Reply via: Email · Mastodon · Bluesky

Gordon Brander writes:

The thing about computers is that they can do anything you wish, so long as you specify your wish in exacting detail. However, LLMs relieve this constraint. An LLM can extrapolate what you mean (more or less) from just a few words. So computers can do vibes now.

It used to be that computers could only ever do exactly what you told them to do, nothing more.

If they did something unexpected (a.k.a. a bug), it was because you failed to 1) anticipate it could happen, or 2) specify with greater detail that it shouldn’t happen.

But in this new world, computers can extrapolate your intent.

LLMS can act like a fill-in-the-gap machine, which alleviates you from the pedantic demands of coding from years past.

There’s going to be a lot more “Oops, I didn’t mean to do that” in the future.

This should be what separates professional practitioners from hobbyists.

A good practitioner recognizes that leveraging some foresight is part and parcel of the job in order to mitigate unintended consequences.

Meanwhile, a hobbyist or amateur doesn’t even have the capability or experience to understand the assumed intent of an LLM until it’s too late and users have been affected.

Software we use everyday, software that many aspects of our lives increasingly depend on, needs less unintended consequences. Less “Oops, should’ve thought of that.”

Reply via: Email · Mastodon · Bluesky

My whole life I’ve felt like I have great eyesight (and I still do).

But lately I’ve been noticing what feels like…some slippage.

Nothing big. Just this feeling in the morning like, as my brain and body wake up, my vision needs to as well.

The strange novelty of this phenomenon was operating in some small part of my subconscious the other day when I was scrolling the internet and come across this post from Chris Coyier:

you ever get to your mid fourties and have your eyesight drop off a friggin cliff

And I’m all like:

And then even more people weighed in on Chris’ thread, like “Hey me too!”

This is why I love the web. I continually get that feeling of, “Oh hey, it’s not just me!” On both a personal and a professional level.

And for that I’m grateful.

I love being able to connect with people in a way that makes me not feel so alone (or weird) in my experience of life.

In a polarized digital world, it’s good to remember that the web can still draw you to folks rather than push them away.

We’ve got a lot in common.

Reply via: Email · Mastodon · Bluesky

Terry Godier has a post titled “Mea culpa” (which, given his framing, might’ve been better titled “Claude’s Culpa”). I’m not sure how much to even trust anything in his post given the backstory, but this line stood out:

I was careless in relying on AI [...] without doing the work to understand

Let’s face it: carelessness is the grain of AI. It’s what the tool encourages and makes easy by default.

Without constant vigilance and deliberate, active participation to cut against this grain — to maintain an understanding — careless outcomes are the default fruit of AI.

And given how good humans are at being constantly vigilant and deliberate, it seems reasonable to expect more stories like this one.

To analogize, AI is pitched like having your very own fruit tree. “Plant a seed, and soon enough, boom! Yummy, juicy fruit for you! So fast and easy!”

But, like most things, it’s not that easy.

Getting good fruit requires all kinds of effort: planting, fertilizing, watering, pruning, thinning, cutting out disease, etc.

Good fruit requires cultivation and care.

Otherwise you’ll just end up with something that looks like a peach, but when you (or someone else) bites into it, you realize it’s disgusting and barely edible.

Reply via: Email · Mastodon · Bluesky

In the past, search engines and the open web granted access to other people’s knowledge but the task fell to you to synthesize an understanding from it and build a capability to act.

Now with LLMs, it’s possible to build things you don’t fully understand which then require a perpetual license in order to have the capacity to maintain and modify.

LLMs don’t just do things for you, they do things instead of you — and the doing is a huge part of the point. The struggle to understand, to synthesize, to explain, it all has a byproduct: you! Your own capacity to act in the world.

But now you can rent that capacity. Every time you reach for the LLM instead of struggling through it yourself, you’re borrowing against someone else’s capacity that you’re not developing in yourself.

It’s as if I used to be a car mechanic, fixing my own vehicles or looking to other mechanics for advice or a demonstration on approaching my problems. That process produced a working car — the output I wanted — but it also produced a capability within myself to fix the next one that came along.

But now, for expedience and speed, I drop all my cars off the LLM dealer for fixes and maintenance. Fast and efficient — but also more dependent than ever.

(See also: Eric Bailey on evolved antennas.)

Reply via: Email · Mastodon · Bluesky

Every zeitgeist comes with new design idioms unique to its challenges. Many of them disappear as fads change, but others bake themselves into deeper parts of existing software interaction paradigms.

For example, there’s the hamburger menu (≡) which saw a proliferation during the rise of mobile due to the constraints around screen size. It has since spread to many other parts of software interaction design and will likely remain prevalent for a long time as a terse way of indicating “more menu-type content here”.

As another example, before AI what were the connotations of the sparkle emoji ✨? Personally, I don’t know, but now it means AI. (AI = sparkles and rainbow colors — it’s funny when you think about it. They should’ve just thrown unicorns in there for the trifecta. AI = sparkles, rainbows, and unicorns ✨🌈🦄. Apt.)

Some patterns are very specific to the interactions inherent to the nature of AI as a technology. For example: streaming text. This is a pattern made for and refined by chat interfaces, so it may not have tons of utility for reuse across other software interaction paradigms.

Then there are other patterns that’ve been refined by AI interfaces and are starting to spread to other places in software. For example, the “shimmering text” which in AI land implies a kind of “thinking” but is being repurposed to indicate any kind of asynchronous task (thinking, fetching, computing, etc.).

Then there are other influences my subconscious is picking up on. For example, a lot of AI apps use tiny icons. These are most obvious (to me) in desktop Electron apps because they clash with the system-level grain of applications. Take a look at this screenshot, where you have desktop AI apps on the left (Claude, Codex, Cursor) and macOS apps from Apple on the right (Finder, Photos, Mail). You can see how the AI apps all have much smaller, thinner icons than their native counterparts.

Are tiny icons our collective future in interfacing with computers? (Personally, I hope not.)

There are other aesthetics my brain associates with AI, like beige/cream colors, orange accents, and serif typefaces as well as whack-a-mole UI controls (you know, the ones where you click the toggle and the entire UI repaints and you have to move your mouse somewhere else in the UI to click the toggle again? The non-determinism of AI’s grain has seeped into its UI/X).

It all makes me wonder what other aesthetics are being born out of this AI moment and how many will spread, take seed, and become part of common software interaction paradigms for years or decades to come?

Reply via: Email · Mastodon · Bluesky

Jason Grigsby has a great article where he surfaces an opinion from the Safari team about how AI agents shouldn’t get special treatment:

An agent acting on a user’s behalf is, in effect, assistive technology: it should operate a site as the user would, and the site should not single it out for different treatment.

Jason synthesizes different discussions happening at standards levels to argue, in essence, that agents should be required to use existing technologies and solutions (APIs, semantics, etc.) rather than get their own bespoke ones. And where there are gaps in the platform, solutions should be centered around closing those gaps generally for everyone (vs. specifically for agents).

Imagine that! Take the billions being invested in AI and funnel it towards improving and enhancing the existing technology agents already use and profit from. No bespoke solutions just for AI, but generalized solutions everyone can benefit from.

In other words: allow the rising tide of AI investment to lift all boats in the platform because the web is for everyone. As Jason says:

If we’re solving this problem for AI, perhaps we can find a solution that works for end users too.

His suggestion being that maybe we should frame AI needs in the web platform the same way we do other needs in the web’s priority of constituents: user needs come before developer needs, implementor needs, spec writer needs — or even agent needs. (UX over DX over AX.)

Now for the funny part. Here’s Jason:

let’s set aside for the moment the irony that AI is supposed to replace all of our jobs and become a super intelligence and at the same time we also need to add special AI training wheels for it to use the web.

It’s like that person you know who prides themselves on their independence, that they don’t bend to society and culture, and that they don’t need anyone or anything — oh, and by the way, could you spot them twenty bucks?

Reply via: Email · Mastodon · Bluesky

Ed Catmull, co-founder of Pixar and former president of Disney Animation, was on the David Senra podcast and I quite enjoyed the interview.

(If you like the interview, you should read his book.)

Ed talks about what he considered his job to be: get the dynamics right for groups of people working together.

To do this, he would pull people out of meetings, make groups smaller, make them bigger, just constantly work on fine-tuning getting the right people together at the right time with the right feedback (without ego).

Granted he didn’t always do it, but that was his goal. He says:

This is so important to get [these group dynamics right] because this is what our product is based on: getting this group of people to work well together. So paying attention to the dynamics of the room is the job. They’re the ones making the movies. I’m not making the movie. I’m just trying to get them to work well together.

When Steve Jobs came to Ed and essentially said, “We’re gonna bet everything at Pixar on Toy Story, and the same week that Toy Story is released in theaters we’re gonna IPO.” What did Ed think?

I thought it was crazy [laugh] I’ve learned a lot in this process. He was right.

I love this frank exchange and Ed’s ability to go from “I thought he was crazy” to “I learned”.

In a creative endeavor, where so many earlier iterations suck, how do you gauge whether you should keep going? Because to keep going can make you seem a tad crazy, e.g. “This sucks — let’s keep going!”

So how do you know whether you should keep investing in it? Ed’s answer:

What’s your basis for proceeding? For me, the basis was: what’s the spirit of the team? Because we all know it [sucks] but if they’re all working together — they’re laughing, and [doing things] together — then you say, “Let’s keep going. Let’s keep trying to solve it.”

I love this. He didn’t say, “You open a spreadsheet and do a cost analysis.” He said he looks at the spirit of the people working on it.

It really backs up what he said his job is: find and help groups of people work together. His attitude seems to be: a group of people with the right dynamics can solve anything.

Ed talks about Pixar’s willingness to take on hard problems because they believed a hard problem led to better outcomes (because few others were willing to do the hard work):

A hard problem is more likely to lead to an interesting film. If it’s easy, then it’s more derivative.

You know how to write a script, you know what the three-act structure is, you know all these elements of storytelling, and you put together all the pieces and you got a story. Is it a great story? Is it emotional? Sometimes yes, sometimes no […]

It’s fairly easy to come up with something that is mediocre — and it’s cheaper too.

If you take on hard things then you need to spend more time trying to figure out how to be different in what you’re doing. So if you take on a hard problem and you just keep pushing at it, then the fact that it was hard is what is going to make it different.

Stated again for emphasis: “the fact that it was hard is what is going to make it different”.

And it’s the execution that’s hard, not having the idea:

If you’re going to make a movie about a rat that likes to cook, that is not a slam dunk.

A lot of people want to keep their projects secret, but this is one where you could tell everybody: “We’re going to make a movie about a rat that cooks!”

His point being: nobody can steal that idea and make it good. Execution is everything there. And if you look at it and say, “Well that’s too hard. How could you make a film about a rat that likes to cook?” You’re skirting the hard work, which is exactly what is going to make you different.

Don’t skirt the hard thing. If you do something that’s hard, that’s the differentiation.

Lastly: I love Ed’s perspective on mission statements:

The reason [we never had a mission statement] is because a mission statement is an answer, when typically we should always be asking questions, like “What are we doing?”

His point being: if somebody asks “What are we doing?” and you immediately go back to the mission statement and say, “Oh, well we’re doing this” that’s not good. It’s better to always be questioning. “Are we doing the right thing? Are we going in the right direction?” You should always be wrestling with those questions.

Reply via: Email · Mastodon · Bluesky

There are two wolves inside of me, lol.

Some days I want to be a “designer”. Other days I want to be a “developer”.

On the days I find myself wanting to feed the developer, it’s often because making something “work” seems easier (and more impressive) than making something “good”.

Making something function often results in a reaction of “Wow, that’s so cool! It didn’t work before and now it does! And I could’ve never made that, nice job!”

And sometimes it’s like, good job, you made a bear ride a unicycle. Not really what bears are supposed to do — and they’ll probably never be good at it — but it’s novel and functioning!

However, the task of making something good — of arriving at a solution that is obvious — is often met with a kind of ambivalence, like “Nice work…I guess? Seems obvious tbh.”

That’s the work of design: to make something so good, it’s obvious. But there’s often little acclaim for the obvious because, well, it’s so obvious (in hindsight).

This plays out in many different ways.

For example, consider a task like making a web site responsive.

In my experience, it’s often quite easy to get people to say “Hey that’s cool, it looks like a mobile site now! Good job!” Getting to that point is often just a matter of sticking a few media queries in your CSS. And people are impressed because they’re not honing in on the details of how it works, just that it works at all.

“Cool, the site displays on a mobile phone now! We can move on.”

But just because it works doesn’t mean it’s good.

And that extra mile to “it works on mobile and it’s also a good experience” is a ton of work. Is it fast? Is it accessible? Is it intuitive? Does it work across multiple devices? Can it be iterated on quickly? So. Many. Questions.

“Does it work?” is a binary question.

“Is it good?” is a subjective question whose answer lives at the intersection of multi-disciplinary knowledge and taste, which is to say: it’s harder to answer than “Does it work?”

“Let’s do X” often boils down to two stages:

1. Make it work
2. Make it good

To “make it work”, all you gotta do is get it running. Consensus on when to applaud and reward the work is simple because it’s either working or it’s not.

To “make it good” requires all kinds of nuanced work. Consensus on when to applaud and reward this work is often impossible to discern because not everyone agrees on what “good” looks like.

“Make it work” is the first 90% of the work. “Make it good” is the other 90%.

Reply via: Email · Mastodon · Bluesky

I wish I could remember how this podcast came across my radar so I could give credit where it’s due. But alas, I cannot.

It’s episode 217 with Dr. Iain McGilchrist.

At one point, he shares his perspective about how science and reason can only take you so far:

[science and reason] have their limitations. Most scientists and most philosophers are very well aware of that. Some people who are not terribly aware of [this] think that [science and reason] can answer all our questions. The fact that they can answer very many questions and be extraordinary servants doesn’t make them the arbiters of everything that we know.

Replace “science and reason” there with “AI” and you have a pretty good assessment.

I like the word “servant” there too — AI can be a good servant, but it’s a terrible master.

[AI] has no body, so it cannot suffer, it cannot love, it doesn’t know that it’s going to die, none of the things that humans deal with, so it is a very, very poor guide to how we should think [of consciousness]

Ok, enough AI. Back to science and reason.

There’s a lot to be said for the scientific method, but not everything can be understood by analysis:

What most people don’t understand is that most [of the great] discoveries in science and mathematics were not made by following the scientific method at all. They were not made by linear powers of logic or algorithms or anything else. They were made by suddenly seeing a gestalt. In other words, a whole that is a form that cannot be reduced to its parts without a loss.

Einstein was famous for this kind of working. He worked in thought experiments and visualizations. His moments of breakthrough came not as the final steps of a deductive chain of reasoning, but through a sudden restructuring of the whole picture of his understanding — “suddenly seeing a gestalt”. He didn’t have new data or a new experiment. He had insights that provided a new framing to hold the existing pieces together.

Those insights were later translated into equations, and then verified through experimentation by others — to this day, scientists are still conducting experiments whose results prove “Welp, looks like Einstein was right. Again!”

As Dr. McGilchrist says: analysis only gives you information, and information is only useful when it is fed back into a context of a whole.

He illustrates this point by talking about a piece of music. You can take it apart, break it into pieces, practice individual passages, etc. — and that can be useful — but it’s the song as a whole that you’re drawn to and the performance of a song will be futile unless it is re-cohered from individual parts into a whole.

(It’s not the things, but the relationships between them that matter.)

Perhaps there’s room for a bit more humility in our knowledge and creative work:

right at the core of reality is the coming together of opposites. That less can be more. That unknowing can be more than knowing. And that there can be something rich in “non-doing” as Buddhists say.

And these are not the same as idleness and ignorance. Ignorance is what you have before you have knowledge. And unknowing is what you have after knowledge if you’re lucky. And it is the beginning of wisdom.

I love that articulation:

• There’s a difference between ignorance and “not knowing”.
• Not knowing is what you have after you’ve gained knowledge (because you realize how little you know).
• Knowing how little you know is the beginning of wisdom.

Reply via: Email · Mastodon · Bluesky