union.io

Building an AI assistant? Here are my top four non-negotiable UX principles you should follow.

Fri, 06 Mar 2026 00:00:00 -0800

Like almost everyone in my field of software engineering, I’ve seen my company, team and projects pivot toward developing AI-powered products in the past year-and-a-half or so. Everyone’s doing it. I want to set aside my personal feelings on whether this is a good thing generally, since that’s such a big and messy question of which I only have fair-to-middlin’ expertise, and instead focus on something I have much more experience in: Ensuring AI assistants’ user experiences aren’t terrible.

I wrote about making my own proto-LLM way back in 2016, then about the first major LLM coding agent in 2021, and then about interviewing software engineers who use AI coding agents last year, so I know a thing or two about the evolution of AI in the software space. I also designed and led development of an LLM-powered chatbot for Elastic’s support portal, which recently won an industry award for best use of AI for customer support. And in that time, I’ve seen and purposefully declined tons of annoying UX patterns that are being used by the industry at large.

Here are my top four non-negotiables for ensuring you don’t wreck your users’ experiences with an LLM-powered chatbot.

1. Never make a user wait for a response if they didn’t ask for a response.

A single unwanted interaction can ruin a user’s entire perception of a product, and by extension, of a company; for many users, AI is intrinsically an unwanted, useless burden that does nothing except steal data and autocomplete sentences. Those users must be allowed to “turn off” AI where practical, and should absolutely never be made to wait for an AI response before ignoring it.

Making them participate in an AI conversation against their will is not viable, nor respectful, nor a way to convince anyone your tool will help them in the future. Always ensure a user purposefully engages with your chatbot before showing a loading or streaming element that cannot be skipped quickly.

2. Don’t splinter the experience. Talking to an AI bot should only happen through one single UI element.

Often the path of least resistence, when adding a new feature, will be to simply “throw a text box in there somewhere and let them start a conversation about it”. This is the easiest thing for us as developers and maintainers of an AI system to talk about, reason about, etc. It’s clear to us this is just another window into our existing AI toolset. However, this is not clear to a passive, disengaged user. It is not easy for them to reason about who they are conversing with, and what “shape” our AI assistant is taking, if there are multiple unrelated interfaces for talking with it.

There should only be one UI element for conversing with a given AI assistant. If you need an additional interface, or additional space, that single element should animate into that other space. Need a “fullscreen” experience? Animate in and out from the existing chat interface. Need a flyout? Same thing. Product teams will find this is a very easy rule to bend and break. It will be much more difficult to keep saying “no” to new chat inputs. But I feel it’s absolutely vital to the brand, voice and the tone of a strong AI assistant to maintain its singular shape.

3. Reserve the “text message interface” for conversational dialogue. Not every AI-powered interaction is a chat.

AI can be used for a million things, but not every thing it does belongs in a back-and-forth, text message-style interface. Displaying search results without accepting conversational feedback from a user? No need to have a chat interface; just use a search results page. Using AI to generate a block of text to be submitted? Stream the results into the existing form fields; refinement can be done inline, or with future edits overwriting in place.

Are users actually talking to your LLM-powered assistant with a cute name behind the scenes? Maybe. Does the user need to know that? No!

It will often be the easier path to just throw the next new feature into the existing chat window. But, like the previous consideration, product teams must keep a holistic view of their product in the forefront, and make the decision to maintain a strong, thematic voice & tone for an AI assistant, even if that’s more difficult sometimes.

4. When someone opts-out, you must opt them out immediately and, more or less, forever.

Similar to the first consideration, if a user declines to interact with an AI assistant, you should assume that declination is permanent. Or at least, permanent within a very long time range. When a user declines to initiate a conversation, say, when attempting to open a support ticket, that decision should be respected when the next ticket is opened as well. Users are not goldfish: They will remember that they can converse with an AI assistant, even if they declined the first time.

As AI grows and is integrated into more support experiences, I think it’s vital to maintain a discrete, well-structured product, and to always be cautious and err on the side of passivity with regards to users’ time and attention.

In conclusion: Keep your chatbot out of the way, respect your anti-AI users, and, even when it’s the more difficult decision, protect your product’s brand integrity, even when your boss’s boss wants AI in every single textarea.

Interviewing in the Age of AI

Tue, 18 Feb 2025 00:00:00 -0800

One of the earliest and most successful applications of LLM-powered text generation was as a “coding assistant” for writing software. I wrote about an early release of Github’s Copilot waaay back in July 2021 (!), ages before any of the modern “chatbots” were around. It was useful then, and has only become more useful since. These days, when pairing with another engineer at my company, I find it much more common to see a coding assistant installed in their code editor than not. And nearly every new feature I’ve seen developed over the past 18+ months in every project I’m a part of has tell-tale signs of AI or other LLM-powered code generation in it somewhere.

After some reflection on the topic, I’ve come to two conclusions: 1. AI coding assistants are extremely valuable, and 2. they will never go away. These two statements will disappoint a lot of people out there, but I’ve thought about this from every conceivable angle over the past 3+ years and I’ve been unable to change my own mind. This is our reality. Might as well accept it and move forward.

The consequences of this new reality are broad and deep. One place in particular has been completely disrupted: Interviews.

* * *

I’m writing this in 2025. After the endless rounds of tech layoffs started by Musk’s hostile takeover of Twitter in 2022, the industry as a whole has been much more conservative in its growth forecasting, and hiring across the board has been nearly wiped out for much of the past few years. Just now I’m starting to notice headcounts ticking up generally; I’ve gotten my first few unsolicited recruiter cold-calls this year for the first time in forever. And even my little team at work got approved to hire two more engineers. So it was time for me to, finally, interview some software engineers again.

But this presented a challenge I’ve never faced before in my career: Designing a software engineer interview in the age of AI.

There were a few ways my team and I considered approaching the design of a technical interview, which would be conducted via video call:

1. Hardline anti-AI, anti-Googling approach

Option 1 was to ask the interviewee to share their screen, and give them the technical challenge at the start of the interview, and ensure they did not use any AI code completion, chat bot, or search engine for the answer. This was the old-school method for conducting an interview in many traditional settings, and might still be the preferred method by more senior software engineers who were expected to just know syntax and library-specific methods by heart when they were hired ages ago, so that’s what they still expect out of candidates.

2. Total disregard for method; results-only approach

Option 2 was to simply disregard the candidate’s methods, and only concern ourselves with their result. This method, oddly, is also fairly common among software graybeards. “Take-home tests” are typically brutally difficult, but hey, if the candidate can solve it, they deserve the job, right?

3. “Pair-programming”/”whiteboarding” low-code or no-code approach

A third option would be a more abstract way to learn about the interviewee’s abilities, and would somewhat sidestep the other two approaches entirely. Simply talking-it-out with the interviewer, possibly in a group setting that would mirror a real-life problem solving meeting, is one way to gauge someone’s technical thought process without even concerning yourself with how well they can implement their ideas.

* * *

Faced with these options, my team and I agreed on a hybrid design, combining all of the approaches above.

My thinking was this: Not letting an interviewee use a coding assistant is, in a very literal sense, purposely not testing for a skill that is a huge part of modern software engineering. Does the candidate understand how to construct a clear prompt, or write a descriptive function name or code comment for the LLM? Does the candidate understand that AI output must be thoroughly understood before being implemented? Does the candidate have the skill to verify, as opposed to the skill to produce code? No matter your stance on AI in general, these skills are, today, vital parts of the job of software engineer. Explicitly not checking for these skills, I believe, is a bad strategy.

On the other hand, I didn’t want to entirely disregard an engineer’s methods for solving a problem and just give them a “take home” problem. A standard coding assistant (plus asking an advanced chatbot for help with more abstract concepts) really makes it possible for very novice engineers to “solve” very tough problems. And while leaning too heavily on AI to solve problems outside your area of competence might work for a coding interview, there will certainly come a problem in the near future that requires a more thorough understanding of the languages and libraries involved to effectively solve. I mean hey, if we just wanted a coding robot to do the work, we’d just.. use a coding robot.

So here’s how we set up the interview:

— I created a CodeSandbox environment that contained a miniature web app that used the basic libraries we use on the team. In there, I put a couple of basic components together, and added a couple of tricky React lifecycle bugs, and some purposely inefficient architectural decisions.

— When the video interview started, I sent the link to the sandbox to the candidate, and asked them to share their screen

— As they prepare to solve the bugs, I reiterated multiple times:

Please feel free to use any coding assistant, Google, AI, Stack Overflow, etc
We don’t penalize for it. I promise.
We’re evaluating your approach to problem solving, and that includes the tools you would normally use to do that.

— After the technical part of the interview, I blocked out 10 minutes to ask them what they thought of the mini web app, and whether they would architect it any different if they were in charge. This was more like a “whiteboarding” session, to gauge their bigger-picture thinking.

* * *

In the end, I was surprised by the range of skills, competence and proficiency with AI the candidates demonstrated. It was really all over the map; I found it super illuminating to watch them interact with coding tools (every single one of them used at least one coding assistant), and I can truly say that watching them either “just let the assistant auto-complete all the code without checking it” vs. “provide a descriptive and performant prompt to the coding assistant and verifying the output” was an enormously important difference to note! And even if the result was the same, that first engineer would not succeed in the role. The latter engineer would be a huge asset to your team.

And if you didn’t let them use AI in the interview, you’d never know the difference.

Dummy Codes: My dumb little error code idea

Fri, 12 Jul 2024 00:00:00 -0700

I made up a dumb little error code idea last year to help solve a problem every web app has. The problem is this. When a user encounters an error in a complicated part of your app—let’s say a part where three or four smaller parts could have failed—you don’t want to blast them in the face with a huge, technical error.

ERROR: The fetch() call to our internal API endpoint failed at the client-level: The API returned a 501 error when trying to retrieve your user's session data from the /user/session endpoint.

Users hate long technical messages and don’t care at all which part failed. So, usually, we go the extreme opposite direction and don’t give them any data at all.

Oops! Something went wrong. Please try again later.

A nicer user experience, for sure. But that means it’s entirely up to us developers to track down the error when it’s reported to us. Maybe a customer support person will message me, “Hey, this user says they see Oops! Something went wrong. Do we know what happened?” Well, no, we have no idea. Now it’s time to gather more info. Perhaps we’ll need the user’s email address, or the date/time they tried the thing that failed, etc, so we can look up where in the app the error occurred. Then we can track down any errors in our logs and find which line of code threw the exception, so we can see where the failure occurred. This can get messy quick, especially if the user doesn’t remember when exactly they did the thing, or they misremember what they did or their own email address, etc.

But hang on. What if there’s a middle ground here? What if there’s a way for the user to get a BRIEF, USEFUL error message that doesn’t reveal too much about our internal systems, and when they mention it to us it’ll mean something and help us troubleshoot?

That was the idea I had. I tossed around a few ideas for how to simplify the error message (color-coded? a numbering system?), but ultimately they all relied on us keeping a separate error-code-to-real-code translator somewhere, and I didn’t want that, since it would absolutely become outdated and obsolete over time.

So here was my idea. Every place in your app that could throw an error should throw an error all the way to the user. Each throw should send a unique, super short error message… And that error message should be self-referential. The best way to do it, I think, is like this.

If the throw is located in a file called: /backend/api/routes/session/get_session.ts, and it’s the first throw in the file, the message should be:

Error BARSG-1: Something went wrong.

What? What the hell is BARSG-1? Well dear reader, it is simply the first letter of each directory in your codebase, all the way up to the file where the error was thrown. backend/api/routes/session/get_session = BARSG. Then add the “1” at the end because it’s the first throw in the file. Easy.

Now the user will see Error BARSG-1: Something went wrong. and when they complain to someone in support, I now get a Slack message “Hey, this customer got an error BARSG-1. Do we know what happened?”. Yep! We know exactly what happened! Easy to look up, doesn’t require a separate error code translator, and doesn’t even require a codebase search. Just navigate to the file in your codebase and there you have it.

It’s also much easier to find that error in your logs/APM interface if you need to get really granular.

Anyway, this has actually been a very useful little idea since I’ve implemented it, and now I recommend Dummy Codes to everyone who writes web apps. If you try it out, let me know how it goes.

Did the world need another web server? No. Did I? Yes.

Thu, 29 Feb 2024 00:00:00 -0800

When I first started making web apps—way back when they were called “websites”, which consisted of these ancient things called “web pages”, which were files made by people called “web masters” (yeesh)—there was more or less a standard way to serve them. You rented a little space at a hosting provider, and then either they, or you, would run something called Apache. That was the web server. Back then, and this was 20+ years ago, Apache was already an ancient beast, having been the first mega-popular web server software. It was as close to “standard” as you could get.

It is now the year of our lord 2024 AD and, amazingly, Apache is still fairly standard today, although it shares its market dominance with a relative newcomer to the scene, nginx (“““only””” 20 years old now). Somewhat surprisingly, they both share very similar overall user experiences, conceptual architectures and distribution/installation methods. Most hosted web apps can switch between the two with very little effort. How has something as fast-moving and ever-changing as web development clung to the same software pattern for so long?

Well, part of it is that Apache is very good at what it does. It starts up when the server does, it runs as root to open the standard ports and load your secret SSL files, it re-spawns processes when they crash, it has been through a hundred billion trials-by-fire every day… It’s good software.

On the other hand, it’s a pain in the fucking ass to install and configure this monster when all you want to do is serve a couple of blogs and a React app for your friend’s art gallery or whatever.

Why do I have to write all of these boilerplate config files in some far-off /etc/ directory, the location and name of which I have to look up every time and—I think?—use sudo? Why do I have to look up all of these exotic Apache-specific directives to shove in there? Have you seen the Apache config file, by the way? The default one? It’s literally 500+ lines long. What in the fuck. I don’t want to do this. I just want to serve my modern web things in a modern way.

Meet the Modern™ web server

Modern web app development is mostly done in NodeJS these days, which ships with its own way to create an HTTP server with one line of code. The standard web server for developing NodeJS apps is called Express, which can be installed by—gasp!—a non-root user and then run by simply require()-ing and .listen()-ing with it in a JavaScript file. Developers discovering this radically simple server felt something like taking a big gulp of fresh air after being submerged in a swimming pool for about five seconds longer than you should have been. Where has this been all my life?

But people don’t use Express for serving “real” websites. There are a couple of reasons for that.

1. Low port numbers are off-limits

I think the number one reason we don’t use NodeJS servers to serve real domains is that unix-like operating systems don’t let regular users open processes that listen on ports 80 or 443—the default ports your browser connects to when going to a website. That’s because ports lower than 1024 are reserved for root only. Why? I have no fucking idea! If your system has a compromised user who is running arbitrary processes, does it actually matter if they listen on port 80 or 8080? Maybe it matters a tiny bit, but I don’t think it matters that much. It’s already a critical breach! In fact, if a malicoius user runs something on port 80, at least you’ll notice your web server is fucked and you’ll be able to shut it down faster. For me, personally, I absolutely do not give a shit if my own user—the only user I created on my machine!—can open a port with a number lower than 1024. But alas, this bit of security threater ensures most NodeJS-powered servers can only be used during development.

2. Multi-domain hosting sucks

Configuring Express to serve multiple domains is complicated. Well, not really complicated, but definitely not supported out-of-the-box. Throw in the problem of individual SSL certificates for each domain, and, well, you might as well be configuring Apache. Shitty!

3. Perceived immaturity

As the old saying goes, no one ever got fired for picking Apache. It’s safe to pick Apache. Is a NodeJS server ready for primetime? Is it safe? “It doesn’t even run on port 80 by default!” “It can’t load balance or serve 10,000 connections at once or serve NSA top-secret documents!” “It must be a novelty.” Even developers who are running small, mom-n-pop-style servers get dissuaded from picking NodeJS servers because they imagine they will need to do all of these super-advanced, super-mature things that only Apache can do. The truth is, they usually don’t need to do that stuff. And as a point of fact, most of NodeJS servers’ perceived immaturity is just that: perceived. Lots of developers who call NodeJS servers immature are actually fine running NodeJS servers behind an Apache proxy! Guess it’s not that immature, huh?

A union of the old and new

The standard way to bridge the gap between old and new server architectures is typically to run both of them at the same time. Yep, wish I were joking. The standard way is to just run Apache/nginx like normal, then proxy each request to your own NodeJS server running on some higher port number on your same machine. This is stable and it works, but is extreme overkill for what a lot of developers need. I just want to serve a few static sites and React apps on my $5 server! I don’t want to have to install, run and maintain all of this shit!!

So I invented union, a NodeJS web server designed to just serve multiple domains on normal web ports, with SSL support baked in. It runs itself as a daemon in the background, so it’ll restart itself if it crashes. In direct contrast to Apache, it is absurdly quick to install and configure and run:

npm init
npm install @union.io/union
npx union

That’s literally it. Run those three commands wherever your domain folders are located (/var/www/vhosts, for instance) and you’ll have yourself a web server. It reads your folders that look like domain names and serves each of them automatically. I also included instructions with common commands to open up low web ports to normal users, read SSL certificates without root, and start the server up automatically when your machine boots/reboots. Stuff to make your plain NodeJS server act like a real server!

And how do I know it works? You’re reading this post on a union server right now. Beep boop.

The future

I started this project just because I needed it for my own stuff. Maybe some other people will find it useful, maybe not, that’s fine. But if it does start getting used, it’ll need some work in a few areas to be a truly viable Apache/nginx replacement for a critical mass of projects:

Load balancing and multi-threading
Windows and other OS support
Expanded SSL support (including instructions for non-Letsencrypt workflows)

So there’s plenty of room for improvement. But if you find yourself needing to serve multiple basic domains in a standard, NodeJS-friendly way, give union a try and let me know how you like it.

Are you automating too early?

Wed, 31 Jan 2024 00:00:00 -0800

Ijoined a new gym last week. I went to the LA Fit Expo the weekend before, and this new independent gym had set up a booth and given me a free day pass. So I tried them out in person, liked it, and joined. I was what marketing people might call a “textbook conversion”. The gym is so goddamn nice by the way, it’s bonkers.

Anyway, they have you sign in for your day pass on a tablet in the reception area, filling out your name & address, etc, and then when you actually sign up for a membership you do it through their app on your own phone. All lovely. When I signed up on the app, it verified my email address and pre-filled my name & address and all that stuff, as it integrated with the “sign in for your day pass” system, I suppose. Makes sense. But one thing it didn’t fill out was the answer to a question I had already entered on the tablet: “How did you find out about us?”.

I’m sure most people wouldn’t think twice, and would just fill that out again. A small percentage of people would probably think “hm, strange I have to fill this particular question out again”, and move on. But a very small percentage of people are software engineers who work at big data companies, like me, unfortunately, and think “Okay, who dropped the ball here? Why didn’t this data field get normalized and integrated between the two systems? Some kind of data quality issue?”

(I’m not done with this inner monologue by the way. It continues:)

Oh, the dropdown had significantly different options than the dropdown on the tablet in reception. I guess normalization wasn’t obvious so they just left it. Also, because there are so many different expos and events and ever-changing reasons, they’d need to have a standard ‘picklist’ somewhere that both systems referenced, and they’d obviously need to keep that list up-to-date…

(…monologue still going…)

Of course, those lists would need to be versioned, because if you remove an option then you’d lose all previous data for some sign-ups. Oh and maybe you should have some kind of metatagging system, since you might want to know that Event A and Event B were both specific types of events, but users might not care about that distinction, and metatagging would allow—

I’ll cut the monologue there, but it continued for quite a while. This was all happening in my head between bench press sets, by the way. After I was content with my solution for this gym’s data quality issues, I looked around. The gym was brand new. There were only three people working out, myself included. The guy who gave me the tour had recognized me from the Fit Expo. Nice little gym community.

That’s when it dawned on me. This was a little gym community. Why did my sick corporate-tech brain feel so COMPELLED to come up with some enormous, complex, absolutely unecessary data integration & analytics system for a gym with like 25 members? Am I really that far gone?

The gym does not need a bespoke Salesforce + Mulesoft + Elasticsearch tech stack (starting price $250k a year). It barely even needs to integrate its day pass system with its membership sign up app. It only pre-filled your name and address in the app as a convenience. As a fucking convenience to the user! Imagine that!

Systems to automate data used for business intelligence, or sales & marketing analytics, or web/app traffic or ANY of that stuff are only done at corporate scale for the convenience of corporate executives. THEY’RE the ones who want that data stamped and delivered in perfect little JSON bundles because that helps them decide where to allocate resources (and when to fire people, of course) to make themselves more money. That‘s literally it. “The events department had a 3.5% YoY decrease in signups last quarter.. We need to work with Chad to realign our KPIs” is the whole reason this data would ever need to scale to this size. Look at how gross that last sentence is.

I had selected “LA Fit Expo” from the dropdown on the tablet. The guy who gave me the tour said “oh hey, you saw us at the LA Fit Expo, right?”. That is, literally, all the data they could ever need to determine that the Fit Expo was why I signed up. The management team (there are probably like two managers, at most) could have a meeting this month and go around the fucking table and just ask each person how many people they met at the Fit Expo and how many of them signed up at the gym. It doesn’t have to be that complicated yet! We don’t need to automate ourselves to death for every tiny fucking thing!

* * *

And this doesn’t just apply to small, mom-and-pop companies (or SMBs as my sick tech brain reflexively calls them now). I work at an enormous data company and even here we automate way too much out of reflex. I’ll give you an example.

We launched a customer-facing knowledge base system recently, entirely custom built from the ground-up. But one thing we couldn’t help ourselves automating was the feedback mechanism. You know the annoying “was this article helpful?” question you get asked all the time? Thumbs up/thumbs down, leave a comment if you want.

We just had to build this big, bulky data collection system, integrating it a million different ways, throwing the data into BigQuery and Elasticsearch and pinging a Slack channel when there was any feedback, etc, etc, etc. Just a huge undertaking. And now that it’s launched guess how many likes/dislikes we get? One a month, if we’re lucky. No one uses it. We didn’t need to automate at this scale yet.

Do you run a small knowledge base and want to know if an article is good or not? Just check out the tickets/chats where it was sent to a customer and then read what the customer says in response. If the customer says “thanks!” and closes the chat, guess what? That article was helpful! If the customer says “this doesn’t answer my question”, then go read the article and see if it’s poorly-written or inaccurate. Yes, this is some manual work for you, or for the person whose job it is to ensure knowledge base quality, but it’s much less work than automating this huge system and getting zero helpful responses because we aren’t Google, whose least-read knowledge article probably gets 300,000 views a day.

So the next time you’re constructing an enormously complicated data pipeline, ask yourself: Am I doing this because it’s the easiest way to get crucial data I need? Or am I doing this because I’m reflexively imagining a tech executive who looks like a suit model and can’t program “Hello world!” wants a nice graph on their next PowerPoint presentation?

Introducing ASTRA

Sun, 29 Oct 2023 00:00:00 -0700

Almost everyone who makes web apps seems to hate web apps, I’ve noticed. And, more specifically, they hate the complexity that manages to find its way into doing anything, no matter how simple. A modern web app that just says “Hello world!” requires like five thousand times more computing power to render than the entire Apollo space program used to put humans on the moon. The size of the web app itself, counting its dependencies, is probably more than any single consumer computer could store before the 1990s. There has to be a better way.

There are a million “better ways”, of course. Every major front end template out there claims to have finally simplified the process of creating a web app, keeping you the developer from any unnecessary complexity or hoop-jumping with their ruthlessly optimized, strongly-opinionated boilerplates.

But this isn’t really true. They may have kept complexity away from the developer, but it was just by stashing that complexity elsewhere temporarily. create-react-app, the one-time defacto React templating library, essentially just developed a ferociously complicated front end application, packaged all of the configs and scripts up into their own little dependencies—dependlets if you will—and gave you a package.json wrapper around them with which to run two or three commmon build-and-serve commands. The dependlets option proved to be a popular one, probably because it meant your top-level template could just import one single dependlet! Wow, so “““simple”””! Unfortunately, the actual truth is not so simple.

They still configure webpack using voodoo magic. They still have to deal with upgrades to babel and jest and enzyme and whatever the hell else. It’s all still there. They still firebomb your hard drive with 300+ node_modules packages just to serve “Hello world”. And if you want to do something differently than the way they planned (“eject”, in create-react-app language), you better pray there’s a tutorial or accurate Stack Overflow answer, and the walkthrough you’re currently on step 16/22 of wasn’t made obsolete when the React team bumped their dependlet from version 3.4.2 to 3.5.0.

“This sucks”, I thought to myself for the ten billionth time last week, as I wanted to stand up a super quick React app to display a few cooking recipes my wife sent me. “I just want a ‘Hello World’ React app I can build and send to my web server to host as HTML/JS/CSS. I just want a codesandbox I can control.”

Enter bun.sh

The ten billionth complaint proved to be the magic number, since I remembered that bun, a new Node replacement ecosystem, had just released their big 1.0.0 version last month. They made big promises about rewriting all of the Node stuff that was too complex, and using it to do all the basic stuff that Node and JavaScript apps currently needed colossal, inscrutable libraries to accomplish.

I decided to see how far I could ride the bun train. The list of things it purported to be able to replace in my React ecosystem was astounding. It claimed it could replace:

webpack (or any bundler at all)
babel
standalone typescript compilers
jest/enzyme/mocha
expressjs

I’ll cut to the big reveal a little early: I was indeed able to replace every single one of those things with a nearly-no-code bun version.

I finished my recipes app in just a few days, and it was a pleasure to develop. Just a handful of React components compiled into a single .js file, linked from an index.html I could look at in my browser locally. Yes friends, I could just visit a file:/// URL in Chrome to see my web app. No web server, no CSS-loaders or whatever, no weird army of chunk-main-abcdef123467890.js files getting cached who-knows-where and injected who-knows-how. So pleasant. So quaint.

“I should make this a template so I can do it again next time I want to make a quick app”, I thought.

The end result

And so a few days later I finished developing ASTRA, the aggressively simple template for react apps. I thought about naming it something cooler like Astroturf, but decided against it, because it is not a cool library. It’s just, essentially, three bun commands that you can run using npm, and two directories: one to make your app, and one for the build output. It really can’t get much simpler than this.

But within this simplicity is power. When I thought about how to best add routing to my recipes app, I felt something strange, something I hadn’t felt in a while when working in the Node/React ecosystem: excitement. My options were endless! I could use any router I wanted, and not only that, I didn’t have to worry about it being compatible with Babel-TypeScript-React-JSX-ECMAScript2016 or whatever. It was just going to work because I am using nothing except React. Instead of being bound by whatever “super fast! super opinionated!” router Next.js or whoever chose for me, I was liberated to do absolutely anything.

In fact, it occurred to me, I’m not even bound to make my recipes app a single-page-app. I could just, you know, copy index.html into add-recipe.html and then target that file with a different React createRoot render! This multi-page pattern, once the normal way to make web pages by the way, has been entirely abandoned by a generation of web developers, so much so it is seemingly impossible to do with any other front end framework… But how many small-app developers could have saved a ton of agony by just making another .html file? How many small projects should have at least had the freedom to consider that as an option??

ASTRA reintroduces that little spark of possibility.

And all because it’s just aggressively simple.

Postscript

I want to end on a philosophical note, and this is probably all a bunch of privileged navel-gazing, but it’s how I genuinely feel. Often times we find cures for awful things and the cures are so good they seem to eradicate the Awful Thing entirely. And then the generation of people who experienced that Awful Thing die and the next generation of people are left wondering why we have to keep doing this whole Cure thing, because the Cure itself takes some work to maintain and hey, we can just stop doing this slightly difficult Cure thing.

Sometimes the new generation is right, and we really can stop doing the Cure Thing. Sometimes a colony no longer needs to send money to the metropole; the things they were “curing” have disappeared or changed and the colony can now maintain independence. Wipe the slate clean folks, let’s start over.

But other times the new generation doesn’t realize the Awful Thing is still there, and the Cure really is worth the trouble. Like with NATO or the measles vaccine—the cost of the Cure is actually tiny compared to the thing it’s preventing (world war and children dying, respectively). But sometimes it’s harder to see the Awful Thing lurking in the shadows, and just wiping everything off the table and starting clean feels like the easier answer.

I’m hopeful my React template is closer to the former situation, where the overwrought complexity of modern web apps has simply gotten so bad it no longer is preferable to the stuff it was trying to “cure” for most use cases. However, I do want to assert that there is a good chance I’m doing something closer to the latter situation, where I’m throwing 10+ years of good ideas into the trash just because I don’t know how Awful it really is out there. There’s a chance someone using my template will encounter just as much pain in the long-run because of the things I foolheartedly left out than the simplicity I introduced.

But sometimes the only way to know is to wipe away the Cure for a while, and see how it all shakes out.

Good luck out there.

I used ChatGPT to help me write my first iOS app. Here’s how.

Mon, 26 Jun 2023 00:00:00 -0700

The “app ecosystem”, as it exists today, is hopelessly broken.

Around the start of the pandemic I started cooking more, and shortly thereafter decided I wanted a recipe app. This idea immediately made me cringe. Just thinking about having to search “best recipe apps” on Google, getting 200 identical clickbait-and-popups-website results, going to the App Store and getting essentially three relevant results, all of which would inundate you with notifications, try to steal your contact list, block off key functionality to get you to subscribe, and on and on… What a fucking nightmare. Our entire modern app universe seems to be hopelessly, pitifully broken.

A year later my wife brought up essentially the same point when talking about trying to find a period-tracking app. It’s an absolute farce. The two or three major period-trackers in the App Store are patronizing, dark-pattern monstrosities, so manipulative and invasive they would probably be considered abusive if they weren’t all bathed in the same light pink pastel color swatches. “Can’t someone just make an app where you enter your period and then that’s it?” she said.

Finally, this year, I decided it was time to start writing down my friends’ birthdays somewhere. I always forget birthdays. I didn’t want to use any of my Google or Apple calendars. I didn’t want to sync it to my phone contacts. I just wanted to write down birthdays in an app and then maybe once a week get a notification if any birthdays were coming up. That’s it. Thinking about searching “best birthday app” made me want to jump off a cliff. Why is this so difficult? If only I were an iOS developer, I thought. If only I could make my own apps. I would make the simplest, easiest apps of all time. No bullshit, no clutter, nothing. You’d just anonymously sign in via text message so you could keep your data across devices, you wouldn’t have to pay or share your location or see ads. You’d just use an app.

But I’m a web developer, I said to myself, and I know absolutely nothing about app development. It was hopeless.

Until ChatGPT came along.

* * *

I use ChatGPT daily for my job. I mainly write TypeScript/Node apps, with a few other web-friendly languages thrown in there. And ChatGPT has saved me literally hours every single week, writing up utility functions (“how does filter().reduce() syntax go again? Oh yeah, ChatGPT can just write this for me.”) and finding lower-order bugs in code snippets I didn’t want to spend time debugging if I can help it.

I’ve gotten really familiar with using ChatGPT for JavaScript (a language I know well). I’ve seen its shortcomings and have been, generally, quite impressed with its knowledge. So one day, while using ChatGPT to help me with my code, it occurred on me: Could I use this chat robot to teach me how to make an iOS app? I started a new chat:

in this exercise I will be creating a simple application to store my friends’ birthdays, first as a web app, then as an iOS app, and you will be acting as an app developer who assists and consults on each step of the process.

This was the first of 598 messages shared between me and the chat robot over the course of three weeks. It was truly fascinating, equal parts frustrating and enlightening. At times the robot gave insightful, nearly impossible-to-find advice that proved crucial to my app working. Other times it wrote entire functions and classes for me, easy for me to verify their correctness but impossible for me to have written. Other times it confidently gave wrong answers, apologized for giving wrong answers, then confidently gave the same wrong answer over and over again.

Here’s the link to the full chat: https://chat.openai.com/share/dccd5c9a-7898-4cef-8fef-2547d0be84db

Along the way the robot helped me build out a Firebase backend for my app, start an iOS project in XCode, build my first SwiftUI views, find and import a calendar library, make an iOS icon and so much more. Time and time again I’d ask the robot how to do something that I could only describe in web development terms, or just in plain English, and I’d get back a thoughtful and accurate answer within milliseconds. This example question, where I needed to “listen” for a change in a parameter passed to a child class, perfectly encapsulates the type of “hard-to-search-for” question that ChatGPT masterfully answers for me over 200 times in the conversation:

I felt I was truly probing the limit of ChatGPT’s intelligence only a few times. Although I got the sense that I was bumping up some kind of artificial wall; maybe the algorithm’s memory or CPU could only hold or process so much data, and my question required holding or processing more information than it was allotted for our conversation:

Pretty quickly I grew more attuned to the robot’s conversational style and what types of questions it excelled at answering. Eventually I found myself thinking about ChatGPT not as a programmer, nor as a search engine, but something a bit more nuanced: I was using ChatGPT as a hyper-personalized bootcamp course & instructor, teaching me exactly what I needed to know in order to implement a real-world iOS application. A six-week iOS bootcamp would have been much more cumbersome, at times no doubt feeling too abstract or too technical. On the other hand, a running conversation with ChatGPT taught me exactly what I needed to know, using a perfectly personalized, hands-on curriculum, at exactly the pace and direction I needed so I could learn how I learn best.

* * *

People who write about chat robots like ChatGPT seem to polarize into two camps: One camp writes about it as if it’s a true technological revolution, and will make all of us rich by doing our jobs for us. Another camp seems to prefer downplaying its abilities, calling it a forgery machine, little more than a Google search bar that’s developed a few parlor tricks to sound like it can speak English.

In the realm of software & engineering, the truth is closer to the former than the latter. The robot can absolutely synthesize, organize and explain novel ideas at near-human levels, and at super-human speeds. It passes the Turing test, a concept many of my fellow software engineers have conveniently tossed aside as soon as they didn’t like the machine that passed it. ChatGPT is truly intelligent. Generative AI is, in my opinion, the most important invention since the Internet.

But for me its current usefulness seems to really shine in the area between “generalist” and “specialist”. The area where, until now, you needed to learn and memorize lots of routine, repeatable things: language syntax, best practices, usage of common tools. A human acting as a good software generalist can use ChatGPT to implement ideas and answer questions the generalist does not have time to learn, like me with iOS development. A human acting as a good software specialist can use it to implement routine functions and rote documentation that would otherwise take them too long to do themselves, like me with JavaScript development. AI robots dominating this “middle area” will indeed cost some human jobs, but I do think it will push more humans into the “generalist” or “specialist” camps as a result. Roles that I feel are naturally more fulfilling for humans, by the way.

This is not an abstract question anymore. This is real, as proved by my app on the App Store and my daily git commits for my job.

* * *

After about three weeks of work in my free time my first app, Basic Birthdays App, is available for iPad and iPhone. I also open-sourced all the code if you want to read it, or build your own app with it.

My next apps will be a Basic Period-Tracker App and a Basic Recipes App. Although my Basic Suite of apps won’t change the world or save us from the ickiness of modern corporate bloatware-disguised-as-apps, I’m hopeful a couple of people out there will find them useful.

And, hopefully, ChatGPT will empower a few more of us novices to invent more pleasant apps, making the modern “app ecosystem” just a bit nicer. Perhaps we can take just the tiniest little slice of digital real estate back from the corporate monoliths.

The “Next-Minor” Versioning Strategy

Thu, 11 May 2023 00:00:00 -0700

I’ve had some trouble researching “git versioning strategies” in the past. Part of the trouble is searching Google, whose current search algorithms so ruthlessly optimize for “most people search THIS, so that’s what you must’ve meant” and “here is the one-sentence result that our Google Assistant can read to you from your smart microwave” that finding blog posts on abstract versioning-releasing strategies is nearly impossible. But part of the trouble I think is that people don’t generally write too many articles about something as “internal” as software versioning best practices. Instead, those articles are generally found in README files in their private github repos.

So here’s my accounting of my team’s “Next-Minor” versioning strategy, which dictates the structure of our git repositories’ branches, tags and releases. This post might serve no purpose other than to remind me how we did all this stuff the next time I need to set up a new large project.

This strategy is a very common one, in use across all kinds of software products. I don’t know if it has an official inventor or even a commonly-accepted name; if it does, I can’t find it. So I’m going with Next-Minor Versioning Strategy (NMVS). Because, as the strategy goes, your code on main will always be the next minor release.

Terminology

For simplicity’s sake, I’ll refer to the main trunk of your codebase as main (historically sometimes called trunk or master). All versioning schemas will be in the semver style, of major.minor.patch, for example, 4.2.13. Any examples or specific version control terminology will come from git and/or GitHub. But this strategy should work with any other VCS, or could be adapted to work with other versioning schemas.

Summary

In short: main always contains all the newest code, and when main is released, it will be the next minor version. For example, if the “current release” is 2.3.13, the next minor version will be 2.4.0.

This means the current minor, in the above example 2.3.X, is maintained on its own branch—in this case a branch simply called 2.3. That branch is where the current release was cut from, and if there is a next patch release (say, 2.3.14), it would be cut from the 2.3 git branch as well.

In typical development cycles, NMVS dictates that you should always work against main, which means branching from main and merging back into main, and you should port only “bug fixes” to the current minor branch (2.3).

I’m going to attempt to make a visual representation of this branching strategy. Historically, I’ve always hated these version control branching diagrams, and never really understood them, but maybe if I make one myself I’ll like it. Let’s try it out.

Okay, this maybe works. It’s simplified, but covers the major point: the main branch, across the top, is where work happens. And releases from main are the next minor version. After a release, the version of the code on main is incremented (in this example, it’s the version in a package.json file, which gets incremented to the next minor, 2.5.0).

The minor branch across the bottom, 2.3, is where patch releases happen. After each patch release, the version of the code on the minor branch is also incremented. When the next minor is released from main, the old minor branch is effectively dead.

But where did that 2.3 branch come from? And what will replace it? Where does 2.4 come from? Well, I didn’t want to put all of that in the first diagram because things got too complicated. So the first diagram was a bit of a lie, and for that I apologize. But let’s look at what actually happens during a release.

Releasing code

Let’s see what the above diagram looks like when we add what actually happens for a new minor release from main:

New minor branches are branched from main when the time comes for a new minor release, in the above example it was the 2.4 branch that was created.

All releases technically happen only from minor branches. (Even the release of a new major version, say 3.0, is technically the patch release of 3.0.0.)

How-To: Patch release

So let’s say we want to do a patch release of version 2.3.13 in this example. The steps are:

(1) Checkout the minor branch from which to cut the release. (Make sure it’s up to date by fetching/pulling if necessary.)

git checkout 2.3

(2) Use git tag to tag the release.

git tag 2.3.13
git push origin 2.3.13

(3) Head back to the minor branch and increment its version in the code.

git checkout 2.3
[edit package.json files, etc, to update version to 2.3.14]

That’s it. The version management part is done. Now all you need to do is actually build/release the code you tagged with 2.3.13.

Typically you would check out that tag:

git checkout 2.3.13

Then follow release steps specific to your code base. In many cases this will involve building images or making executables… That sort of thing.

At this point I also like using GitHub’s Releases interface to create an official release for 2.3.13, so it can be a documented artifact. See GitHub’s docs for how to create a release in your repository.

How-To: New minor release

If you want to release a new minor, which comes from the code on main, there’s really only one unique step: You need to create a branch off of main! First, ensure you are on main and have pulled in the latest code. Then, create the new minor branch. In this example, the existing minor branch was 2.3, so we will create 2.4 from main:

git checkout -b 2.4
git push origin 2.4

The version of the code in main should already be 2.4.0, which means that the code in main needs to be incremented to 2.5.0 now:

git checkout main
[edit package.json files, etc, to update version to 2.5.0]

And you’re done. Now, you’ll follow the steps above for a patch release from the new minor branch you just created (2.4). Your release will be 2.4.0.

If this is making sense to you so far, but you feel like perhaps something’s missing, you’re right. We haven’t covered how to actually get changes INTO a minor branch. Up to this point, all of the coding and merging has happened only on main.

Once again I have lied to you. The above diagram was still too simplified; backporting is the key feature that allows for patch releases, but I didn’t put it in the previous diagrams because, once again, they got too complicated. Please accept my apologies as we cover backporting.

Backporting

All code is merged into main, but only some code is backported to the current minor branch. Exactly what code does and does not get backported is up to you, but I like the rule that only bug fixes get backported.

As a result, all patch releases off the current minor branch will contain only relatively small bug fixes. And the next minor, from main, will contain all the new features.

Backporting is the process of selectively copying specific commits from one branch to another. In the context of NMVS, backporting involves cherry-picking commits from main to the “current minor” branch.

You can perform a backport manually after merging code into main using git. The primary command for copying individual commits is git cherry-pick. The command takes the commit hash or reference as an argument and applies the changes made in that commit to the current branch.

For example, to backport a single commit with the hash abcdefg from main to the current minor branch 2.3, you would run the following commands:

git checkout 2.3
git cherry-pick abcdefg

It’s possible at this point you’ll encounter a code conflict, and git will prompt you to resolve it. But in my experience this is surprisingly rare, as only bug fixes were ever backported to the current minor branch after already applying them to main, so things generally keep in sync.

But, if you do manual backporting, you’ll probably need to brush up on some full documentation for the cherry-pick command.

To simplify the backport process, you may want to consider a backporting helper library. I recommend backport if you’re working with a node project. It also has extensive documentation on how to automate backporting with GitHub actions, that way PRs labeled as “bugfix”, for example, can be automatically backported when the PR to main is merged. It’s a huge timesaver, highly recommend.

Releasing a Major Version

Eventually the time might come when you want to release a major version; if we keep our examples above going, this would be version 3.0.0. What then? All you have to do is increment the version on main to the next major, instead of the next minor.

And what if you want to release the next major version now, not after the next release? Well, folks, it’s so straightforward I can hardly stand it.

Yep, just increment the version of the code on main to the next major version, then proceed as normal to a release.

Conclusion

NMVS is currently in use all over the place, in software projects of all sizes, and works at scale well enough to support even the largest code bases.

I find it keeps releases smaller and more predictable, doesn’t burden developers with having to worry about managing tons of branches or having to know which release their code needs to target; you always work from, and to, main.

NMVS does have its drawbacks. I don’t find it particularly intuitive. And, although it makes developing simpler, most of that complexity has simply been stashed away in the backporting and releasing tasks. So it absolutely requires good documentation and at least one experienced developer to help keep things straight during the first few releases.

I do think, overall, NMVS is worth it. And if you think so too, maybe it’s the right system for your next project. Which means we’re finally ready for the last, final, way-too-detailed diagram of ultimate complexity. This is NMVS. I’m sorry:

Note to Self: How to Use a VPN to Stream Blacked-Out Shows on Your iPad or Other Mobile Device

Sun, 19 Feb 2023 00:00:00 -0800

This is not a typical blog post of mine. It’s just a note to self. See, I realized my blog is actually the best format I have to save a long-form note with code snippets, so I’m just going to use it to remember how I set up a private VPN for my wife to watch This Country on the BBC iPlayer, in case I ever need to do it again.

If you find yourself here because you also want to stream content from other countries or blacked-out regions without subscribing to some VPN service, feel free to follow along. If not, please ignore this post and have yourself a nice day.

The back story, if you care

My wife said she wanted to watch a show called This Country on BBC. The first two seasons of the show are freely available in the US on their website, but for some reason the third season is blacked out for people outside the UK. She also wanted to watch it on an iPhone or iPad, and I didn’t want to pay some random, shady VPN service that might be blocked from accessing the BBC website anyway. So those were the requirements.

Since I’m a software engineer I figured it wouldn’t be too hard to set up a personal VPN for a few bucks and then turn it off when my wife finished watching the show. And, I was right. It only took four steps. Here’s the executive summary on how I did it:

Spin up a server in the UK, using a pay-by-the-hour service like Digital Ocean
Follow instructions for how to set up a Squid proxy on that server
Get a little inventive with your configuration to circumvent some basic security checks
Connect your device to that server, create a BBC account, and stream away

This took me a few hours total to figure out and implement. But now that I know how, it would only take me a few minutes to do it again. The total cost was something like $0.70. And this should work with almost any streaming service that checks your location via your IP address, like MLB or NBA or NFL blackouts.

Please note this worked for me in mid-February 2023. The farther you are in the future from that date, the more likely the instructions are to be slightly out of date, or for the streaming sites to have wisened up and blocked more kinds of traffic; so, in short, the less likely this trick is to work.

But anyway, here are the instructions that worked for me. Good luck.

You’ll need three things for this: (1) A **Server**, which will be the thing you pay a hosting provider for that will run the VPN. (2) A **Home Computer**, which is how you’ll test things out and access the server, and (3) A **Device** to watch the content on (obviously!). Your **Device** could also be your **Home Computer**.

1. Spin up a server in the appropriate region

The first thing you need is a physical server in a place that is allowed to watch the content you want. You’ll be receiving all the streaming content to that server, then forwarding it on to your device. That’s the basic premise here. So I recommend you spin up a cheap server on some provider like Digital Ocean that lets you choose where that server is located:

Digital Ocean lets you create servers (called “Droplets”) in a bunch of different regions; I chose London. Within a few seconds my server, which was $0.009 an hour and came with 1000 GB of free data transfer, was ready. And you can delete the server anytime you want and they only charge you for the hours you had it running. Pretty neat.

For this exercise I chose the Ubuntu 22.04 Linux server, so all of the following instructions will be for that kind of machine. You will need to access this machine as the root user, which you can do from your terminal or from their web interface.

Once you’re in your new server as root, let’s get a few key pieces of information together, which will help us later. First, run this command from your server:

# curl ifconfig.me/all

You’ll see output like this:

# curl ifconfig.me/all
ip_addr: YOUR.SERVER.IP.ADDRESS
remote_host: unavailable
user_agent: curl/7.81.0
port: 51400
language:
referer:
connection:
keep_alive:
method: GET
encoding:
mime: */*
charset:
via: 1.1 google

The important part is the ip_addr line. This is your server’s IP address, so write that down or just keep it in mind. Now, run the same command on your Home Computer. The curl command is built in to the Terminal app on Mac and Linux machines, and I think it’s available on Windows machines in the Command Prompt app. The resulting ip_addr is your home IP address—the one the streaming site is blocking—so keep that in mind too.

2. Create a Squid proxy

Okay so maybe it’s not technically called a “VPN”, I’m not sure, but a proxy does what we want it to do: It relays all of your web traffic so the streaming site thinks you’re somewhere else. And Squid is a free, open source proxy we can install quickly and easily. It also seems to handle streaming content just fine. So let’s install that on our server:

# apt update
# apt install squid

Next we need to edit a few lines in the Squid config to allow our home devices to talk to the server, but ONLY our home devices. You’ll need your home IP address from above. You’ll also need to edit a file on the server using a command line editor like VIM or nano.

# vim /etc/squid/squid.conf

You’ll want to find the lines, around line #1550 or so, that look like this:

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
include /etc/squid/conf.d/*

# Example rule allowing access from your local networks.

# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
#http_access allow localnet
http_access allow localhost

# And finally deny all other access to this proxy
http_access deny all

.. and edit them to add your own home IP address to an acl named localnet, then allow access to that localnet.

# INSERT YOUR OWN RULE(S) HERE TO ALLOW ACCESS FROM YOUR CLIENTS
#
include /etc/squid/conf.d/*

# Example rule allowing access from your local networks.
acl localnet src YOUR.HOME.IP.ADDRESS

# Adapt localnet in the ACL section to list your (internal) IP networks
# from where browsing should be allowed
http_access allow localnet
http_access allow localhost

When that’s done, you’ll restart Squid:

# systemctl restart squid.service

Then, allow traffic through Squid’s default port:

# ufw allow 3128

(You can just run these as root by the way. It’s fine.)

If you want more info on how all this works you can read the full instructions here, but they contain a couple small mistakes and also walk you through setting up password security. I just took the important parts and simplified them to work for the root user, so, up to you.

Now you should have your VPN running and it should allow your computer talk to it. Let’s check from your Home Computer.

$ curl -v -x http://YOUR.SERVER.IP.ADDRESS:3128 ifconfig.me/all

This curl command sends your HTTP request through your Server. Check your output… You should see your server’s IP address in the ip_addr. Congratulations, you have set up a working proxy.

3. (A) Configure Squid to be sneakier

For some services, like MLB At-Bat, you don’t need to do any other config changes, and you can skip right to step 4 below and watch your content. But for BBC and probably a ton of other streaming sites you’ll need to get slightly craftier. Notice this line in the output of the curl command you just ran:

via: 1.1 ubuntu-s-1vcpu-1gb-lon1-01 (squid/4.10), 1.1 google

Yes, your default Squid server is sending along proxy information with its request. Basically, it’s a snitch. Some streaming sites know this and then block you. So let’s tell Squid not to do that. You’ll need to edit your Squid config on your Server again (remember, it’s at /etc/squid/squid.conf):

First, find the via option, which is around line #5520 and looks like this:

#  TAG: via     on|off
#       If set (default), Squid will include a Via header in requests and
#       replies as required by RFC2616.
#Default:
# via on

… Uncomment it, and turn it off.

#  TAG: via     on|off
#       If set (default), Squid will include a Via header in requests and
#       replies as required by RFC2616.
#Default:
via off

Lastly, find the forwarded_for option, around line #8646:

#       If set to "delete", Squid will delete the entire
#       X-Forwarded-For header.
#
#       If set to "truncate", Squid will remove all existing
#       X-Forwarded-For entries, and place the client IP as the sole entry.
#Default:
# forwarded_for on

…Uncomment, and set that to delete:

#       If set to "delete", Squid will delete the entire
#       X-Forwarded-For header.
#
#       If set to "truncate", Squid will remove all existing
#       X-Forwarded-For entries, and place the client IP as the sole entry.
#Default:
forwarded_for delete

Now save and restart Squid again:

# systemctl restart squid.service

Once that’s done, run the curl command from your Home Computer again to confirm the via line no longer includes proxy info:

ip_addr: YOUR.SERVER.IP.ADDRESS
remote_host: unavailable
user_agent: curl/7.54.0
port: 42080
language:
referer:
connection:
keep_alive:
method: GET
encoding:
mime: */*
charset:
via: 1.1 google

All good!

3. (B) (Optional) Acquire and configure a reserved IP address

At this point I recommend skipping to step 4 and trying to access the content. There’s a good chance it’ll work. However, I found I was still, somehow, being blocked by the BBC. My best guess is my Digital Ocean server’s IP address is on a list somewhere. A list called These Might Be VPNs So Don’t Let Them Watch Things. So I spent the extra few minutes and configured a “reserved IP address” for my server… It’s free and provided me an individual IP address that is less likely to be on those kinds of lists.

If you try step 4 and it is still blocking you, come back here and try the reserved IP trick. Here’s how you do it.

First, in your Digital Ocean panel, enable the reserved IP:

It should be ready in a few seconds. Then, logged in to your Server, configure it to use this new IP address as the default address for outgoing traffic. The official documentation for that is here, but I just used the following two commands:

First, find your server’s “anchor IP” which can be found by running this special command from your server (it does some Digital Ocean magic in the background I guess):

# curl -s http://169.254.169.254/metadata/v1/interfaces/public/0/anchor_ipv4/gateway

This should return an IP address. Then, take that IP address and run this command:

# sh -c "ip route del 0/0; ip route add default via THAT.ANCHOR.IP.ADDRESS dev eth0"

And that’s it. Run the above curl command one more time from your computer, this time using your reserved IP address:

# curl -v -x http://YOUR.RESERVED.IP.ADDRESS:3128 ifconfig.me/all

And you should see your reserved IP as the ip_addr and no via headers that give away your home IP address. You’re done!

4. Connect your device to the server and go

The last step is to connect your mobile device, or whichever Device, to your server. I’m going to give instructions for how to connect iPads/iPhones, but all devices can use the VPN you set up, you can just Google how to connect that device to a proxy.

First, on your Device go to your network settings for your wifi network:

Then, at the bottom, go to HTTP Proxy > Configure Proxy:

Select “manual”, then enter your Server’s IP address (or reserved IP address, if you made one in step 3), and port 3128:

And that’s it. Now visit a website like whatismyipaddress.com from your Device to confirm: The Internet thinks you’re wherever your server is.

Now head to the BBC website and sign up for an account. You might want to do this all in a private browsing tab, or clear your cookies, since maybe the last time you visited they tagged you as Not Allowed To Watch Things. When you sign up it’ll ask you for a UK postal code (you can Google one, or just use NW1 0AU). Then, lie about having a “TV Licence”, whatever that is, and there you go. Your content is now free to stream.

Troubleshooting

If your server stops sending your traffic

If the curl commands above stop working from your Home Computer, it’s possible your home IP address changed. That’ll happen once in a while. Find your new IP address and go into the Squid config to update the allowed IP address to your new one. Restart Squid and see if that’s the issue. If not, you’ll have to Google around, it could be a lot of things, sorry to say.

If the BBC still says you’re not allowed to watch stuff

It’s possible your IP address is on a list somewhere. Maybe try a new one by repeating step 3(b) above. Or, try an entirely new provider, if you can.

Final thoughts

Living in Hollywood I’ve met plenty of producers and actors and grips and audio engineers and PAs and union guys who unload the trucks and everyone in between, but I’ve never met anyone who works in entertainment who enjoys or supports withholding their product from potential viewers just because they’re in another location. So I’m fairly certain regional blackouts are another exploitative tool used by executives who don’t need to work for a living so they can further enrich themselves rather than creating things. So I don’t have any ethical hold-ups in circumventing these regional blackouts, and I hope you don’t either.

Happy streaming ✊

Three small things you should be doing right now that will make your web app way, way better

Wed, 01 Feb 2023 00:00:00 -0800

Developing and maintaining a successful web app sometimes requires decisions that are really big and highly complex. Like routing, linting, testing, CI/CD strategies, etc… Those things could each be their own six-part blog post, and you’d still barely scratch the surface. Other times these decisions are tiny, clever tricks you can start using right away, costing you almost nothing. I like those kinds of things better. And when I find one, I like to write a few paragraphs about it to remember to keep using it, and hey, I might as well share a few of them with you now.

So here are the top three small things I think you should be using today to make your web application more stable, more inclusive and all around way better.

1. Make all of your API calls fail randomly in dev

I love this one. So brutal in its simplicity, but has such an enormous positive effect on your end UX. I’ve used it for years now after a coworker showed it to me and I’ll never go without it again. The trick is this: Every time you hook up a fetch from the browser, or a GET from a server, or anything that talks to any external data source at all, make it fail like 10% of the time while you’re running it in development mode. It doesn’t have to be fancy, it can be as simple as this:

const oneInTenChance = Math.floor(Math.random() * 11) === 10;
if (oneInTenChance) throw Error('Please try again later');

Now stick that in your client-side component (or action, or what have you):

try {
  const oneInTenChance = Math.floor(Math.random() * 11) === 10;
  if (oneInTenChance) throw Error('Please try again later');

  const resp = await fetch(`/foo/bar`, { // [...etc]

  }
} catch (e) {
  console.log("Error:", e);
}

… or better yet, stick it directly into your API server’s routes:

app.get('/api/some/route', (req, res) => {
  const oneInTenChance = Math.floor(Math.random() * 11) === 10;
  if (oneInTenChance) throw Error('Please try again later');

  res.send(
    `foo = { "bar" }`
  );
});

(Just don’t forget to remove this when you run in production mode, lol)

This has the wonderful effect of making you mad as you develop the client software, and you yourself become the user, frustrated with a random API error here or there. You simply can’t ignore it. Gracefully handling errors has got to be one of the top five things that separate a great app from a fly-by-night operation, and your users absolutely can tell the difference. Start pissing yourself off today so you don’t piss off your users tomorrow.

2. Use non-traditional mock data

And by “non-traditional” data I mean “not White-Euro-American, culturally Western” data. Using John Doe/123 Fake Street as all of your mock data sucks. Sorry folks. This isn’t some leftist communist globalist manifesto thing, this is just practical web development. You should make a habit of using entirely different mock data sets than ones you’re most comfortable with, starting today. I’ll show you why.

The design…

…vs reality

App designs and implementations using “John Smith”-style mock data will be unnecessarily brittle and usually too constrictive, not allowing enough space for longer text nodes or different data shapes. The fixes are almost always trivial—ensuring an element flexes or a text node truncates at a certain length, etc—but if your designers and developers and product owners all use the same kinds of names and addresses, you may never know you needed to implement these tiny fixes until it’s too late.

This also extends to non-visual development as well. All too often you’ll see a unit test that looks like this:

// Test our buildEncodedURLParams() function

const mockData = {
  name: "John Doe",
  email: "john@example.com",
  phone: "123-456-7890"
}

const urlPath = buildEncodedURLParams(mockData);

expect(
  urlPath.length.toEqual(
    mockData.name.length +
    mockData.email.length +
    mockData.phone.length +
    3 // encode the "@" symbol!
  )
);

That test is way too brittle. Without even knowing what buildEncodedURLParams() does under the hood, you simply can’t expect the test as written above to capture any edge cases, and essentially anything that deals with non-culturally-Western data! To better generalize a unit test, which in many ways is exactly the purpose of a unit test, we should try thinking in data units that are outside our own common professional experiences, precisely because the ones within our common experiences will be tested naturally during development! In other words, a far better mock data set would be this:

// Test our buildEncodedURLParams() function

const mockData = {
  name: "张伟",
  email: "zhāngwei@xn--s7y.co",
  phone: "021-15545638, ext. 034"
}

// [...]

That test right there makes for a more solid, scalable web app. And it’s free to implement. To start generating better data right now just use something like ChatGPT or you can Google for a name generator that lets you select different cultures/locales/nationalities when making fake personal data. Make this a habit and soon you’ll cringe every time you see old John Doe making your tests and UIs brittle.

3. Tokenize all of your copy

This one took me a while to fully commit to, but now that it’s a habit, I never want to go back. Here’s the gist. All of the words on your web app need to be tokenized. So, not this:

but rather, this:

// tokens.js file
const tokens = {
  sendEmail: "Send email"
};

// component file
import {tokens} from 'tokens';

“Oh my god that’s way too much work, our website has like a million words!”, “Never happening, I can’t be bothered to do this.”, “Good luck getting all our developers on board! They’ll never do this”, yeah yeah, I get it. I was there too. I said each one of those things. But guess what? It’s not actually that bad. It’s only a few seconds of extra work per token, and when new developers contribute to the codebase, they all learn very quickly to tokenize their copy.

For some context the Elastic Support Portal—a big web app I led the development on—is fully tokenized and the grand total number of tokens is currently… 366. Not that bad.

And what does that enable us to do? Internationalization. Or, i18n for short. When we want to roll out a new version of the app in, say, Greek, all we have to do is add translations of the tokens that are already there:

const tokens = {
  english: {
    sendEmail: "Send email"
  },
  greek: {
    sendEmail: "Στειλε το"
  },
}

By plugging those tokens into any i18n library, what you get is almost frighteningly powerful. Now, paying an interpreter to translate a few hundred JSON values then do a quick scan of the app for completeness is almost a negligible cost compared to the end result: A native app experience in any language or location. And that’s not even getting into how easy it will be to audit all the language on your website if the need arises; delivering a copy of all the text used in your company’s app to the legal department will now take roughly 10 seconds.

“Well, what about currencies? Or date formats? Or pluralization??”. Valid concerns! And each of those are addressed by one of a number of i18n libraries out there.. but they all start with tokens. The work you do now will not be wasted; it’ll all port over easily to whichever fully-baked i18n solution you choose.

There are a million ways to go from here, like the popular i18next, but it all starts with one thing: { tokens.startTokenizingNowExclamationPoint }