Take A Break, Script Something

Lately I've been putting in a fair amount of work improving Adventurer's Codex, and we have some very exciting updates coming soon (no spoilers 🤫). A lot of the changes required modifying our existing single-page app or API backend, but one involved the creation of a totally new repository and a new set of public-facing pages. It's those pages that I want to talk about.

The problem was fairly simple: given a set of data that changes very infrequently, create a set of web pages that display the details of each record in the dataset. Simple right?

Now, there's two ways I could have built out this functionality:

  1. Dynamic pages that serve the content using templates served by Django
  2. Static pages built once and served by Nginx
Additional Context

I wanted this functionality to be separate from our main project, so the first solution would involve setting up a new Django app and database as well as building a system to import the data from JSON files.

I elected to do the second solution because of it's simplicity and because the data changes so rarely. When all was said and done, I had three scripts — two bash scripts, and one Python script — and a set of Jinja templates. Running the main build script would download the dataset if it didn't already exist, parse the data, and build the pages. The whole process takes about 10 seconds to download and generate over 1200 pages. After it was done, I even set up a Docker container to build and serve the pages with Nginx. In total the project is 295 lines of code (1k if you include the HTML templates), and basically never needs to be updated again; if the data does ever change, we could simply re-run the script or rebuild the contaner.

Scripting is a Much-Needed Break

This is one thing I love about scripting as opposed to application development. In app development, you construct a codebase and then you need to live with it long term, adding new functionality and deprecating old functionality. Scripts, on the other hand, are low risk, high reward: you write them once to solve a specific problem and then rarely touch them again. I use a lot of simple scripts on a daily basis, some of which I wrote nearly a decade ago and haven't touched since.

Scripts are programming candy whereas app development is the real meat and potatoes. In a script you can take shortcuts, be a bit messy, and forgo worrying about the complexities of large software. Once the script works, there's not much else to do: just ship it.

Whenever I get the chance to take a break from developing large apps and just do some quick scripting, I leap at the opportunity.

The Road To Glass & Stone

I don't think I've ever talked about this here before, but I play in a band. And that band, The Fourth Section, just released its first EP, 'Glass & Stone'; and it's available now.

It's been almost two years since we formed, just before lockdown in March of 2020, so technically this is our pandemic EP. I'm super happy that we've finally gotten a release out there and available for everyone.

Glass & Stone

I've released music on Bandcamp before, but this is the first time that I'd ever gone through the process of getting music onto the big platforms. It's both a time-consuming process, and still incredible that a bunch of indie artists, with no label backing, can simply fill out some paperwork and then distribute music world wide. We often lose sight of it, but the internet is really magical sometimes.

The EP was recorded, mixed, and mastered at Cacho Studio in Tijuana, and they did a great job. My thanks to everyone at the studio for their hard work.

A Virtual and Untrodden Road

Since we formed right before the pandemic lockdowns hit the U.S., we couldn't practice at a studio in-person at first. For nearly a year we rehearsed virtually or acoustically—as well as masked and social distanced—at a local park. The three of us used Jamkazam for our virtual practices and it works well enough for that purpose, but it's no substitute for actual, plugged-in rehearsal at a practice studio.

We're back to in-person rehearsal now, and it feels great. Lots of new stuff is in the works and hopefully there will be more to show in the coming months.

Having gone through the process of writing, composing, and distributing music once, we now have a pretty good idea of what it takes to make an EP (or an album) happen, and we're really stoked to do this whole process again (and soon). As with most things, the first try is usually the hardest. There's always the initial, one-time setup that has to be done, and there's just so much you don't know the first time. That is all past now. We know what it takes, and the setup is done. Releasing more stuff is a lot easier now.

And that is what we plan to do.

Nine9s Gets An Upgrade ⬆️

I'm super excited to announce a new suite of features for Nine9s: my uptime monitoring service. Now, I don't want to exaggerate too much, but I honestly thing that as of today, Nine9s is one of the most powerful uptime monitoring services out there; if you haven't already, you should try it out.

Monitor Anything

Uptime monitoring, in most cases is a pretty simple task: if you have a web service, then Nine9s makes a request to it, and verifies that it returns a successful HTTP status code. Most services stop there, or they expand on the idea and allow for checks to be more complex (e.g. verifying that elements exist on the page or that users can log in).

The problem with this approach is that lots of services exist on the web, but do things other than simply serve web content. You could have a service that sends emails or that runs batch jobs. These services can't be monitored using simple HTTP requests. Even still, sometimes you don't want your services to be publicly accessable, in which case uptime monitoring services can't reach out to them at all.

That's where Nine9s steps in to fill the gap! As of today, servers can now periodically ping Nine9s to report that they're still up and running. Nine9s will watch out for these reports and let the user know if the service doesn't report in within a given amount of time.

Better still, Nine9s can evaluate the data sent by the server and compare it against some criteria. Imagine you have your server report its current disk usage to Nine9s. If the disk ever fills up past say 95%, then Nine9s will alert you immediately.

Example Criteria on Nine9s

When you set up a new Measure—which is what Nine9s calls these new features—Nine9s will generate a couple of sample cron jobs you can quickly add to your servers to get started! There's even a quickstart guide to writing additional jobs!

Sample Cron Scripts

Once you understand the power of these new Measures, you'll be able to monitor nearly anything your servers are doing and make sure they're performing as expected.

Nine9s is macOS Native!

This release also marks the launch of the official Nine9s app for macOS! It's my first ever public macOS app and I'm super excited to get it out there into the world. The app shares all of its code with the iOS and iPad versions and is built on Catalyst, so it should feel mostly native.

The app has all of the features you'd imagine: the ability to monitor your Endpoints and Measures, support for push notifications, and more.

The app is also my first ever native app released outside of the App Store!

I've been working on this update for a few months now and I'm stoked to get it into the hands of all you out there! If you do end up downloading the app, let me know especially if you have feature recommendations! I'd also love to know how you all are using Measures in the wild.

The Internet's Original Sin

A piece I wrote was published in the San Diego Union Tribune today. The piece is a short dive into a possible alternate history of the web and the internet as it could have been.

In a bucolic way, the piece compares the development of the internet and the inevitable consequences of a few seemingly insignificant policies adopted by ISPs in the early days. The piece draws a through-line from those policies straight to the emergence of the centralized platforms we know today.

In short: ISPs killed the internet and made platforms like Facebook and Amazon inevitable.

The piece compares America's idealized conception of the westward expansion on the frontier to the development of the internet in the last few decades.

There are no homesteads on the internet frontier. This is true on both the large and the small scales. Whether you start a website yourself, or use Facebook, you’re still effectively renting space from someone else. On the internet, we are all simply users of other people’s computers, tenants renting homes we can never own.

The title for the print version is much more attention grabbing.

However, the development of the internet was fundamentally different from westward expansion in two key ways. First off, there was no one already living on the internet frontier—which isn't relevant to the piece directly, but it's an important distinction. The expansion onto the internet frontier was bloodless.

But secondly, ISPs made the internet a renter's frontier. Their decisions to not assign static IP addresses to homes or user accounts, and their decisions to provide non-symmetrical connections and block ports 80 and 443 made the internet fundamentally different than it was intended. On the internet today, truly owning your online presence is only possible for a very small subset of individuals and companies; the landlords of the internet.

No one would accept a world in which you could never own the car you drive, or the house you live in. Some people prefer to rent or lease, but cars and houses can still actually be bought. On the internet there is no path to ownership. Each of these stumbling blocks ensures that ordinary people can never truly own a piece of the internet frontier.

If computers are the land of the internet, then our personal information is the crop. Instead of being given our own small farm to start our online lives anew, we’re forced to be digital sharecroppers. In exchange for a portion of our privacy, we are allowed the chance to cultivate lands we do not own. On the internet, we are not citizens. We’re just users. Is this freedom?

Check out the full piece for yourself →

Last Year I Started Reading A Physical Newspaper

I subscribe to a lot of local and national news outlets, and as most people do these days, I read the news on a screen, because well, I'm a 30 year old millennial and it's 2021 and not 1980. But last September, out of a mix of both idle curiosity, anemoia, and a strong urge to put an end to pandemic-induced doom-scrolling, I signed up to receive a printed copy of the Sunday paper.

I've been happily reading a Sunday paper ever since.

Every Sunday Most Sundays, a paper appears before I wake up, I stumble, half asleep, out to get it, I make coffee, and then I read the news. It's a quaint experience, and it's a habit I've really come to enjoy and look forward to. The whole process takes anywhere from 30-90 minutes, then it's done, and I go about my day. On Sundays I do my absolute best not to read the news on my phone, as I do every other day. Once I've read the paper, I'm done reading the news.

Newspapers on a surface

A stack of papers I had lying around. Photo credit: me.

Part of the reason I originally signed up for a physical paper was to force myself to read it. As I said, I subscribe to a lot of new sites, and I find a lot of articles that I want to read, but I almost never actually read them all.

With so many stories to read, I found myself naturally gravitating towards the most exciting or most outrageous news. National news is almost always much more polarizing, engaging, and exciting than state and local news, but the paradox is that state and local news is often far more important and more likely to directly influence you in your day-to-day life.

What's exciting, urgent, and engaging is rarely what's important. I noted this in a previous post about my policy blog, Democracy & Progress.

A few years back, while listening to the Ezra Klein Show, Ezra lamented that we as a society didn't spend more time focusing on local and state politics—where our time and energy is often better spent. Collectively, we don't focus on state and local politics, and yet it's only there where a lot of policy solutions can be done. That conversation stuck with me...
California, Democracy, and Progress

I found that having a physical paper show up at my door made me more likely to read it. Simply throwing it away felt wasteful.1 That slight guilt motivates me to keep to my own goals.

In the last year, my knowledge of local & state politics has grown immensely, and I've really enjoyed the process of learning more about my state and city. I also know a lot more about both San Diego and California and what progress is and isn't being made.

Lots of people both inside and outside of the media lambast the news industry for being overly negative, and while it certainly is, national outlets are catering to a much more diverse and varied reader-base. On top of that, almost any political news story in a country as large and sufficiently polarized as the U.S. is bound to upset around 50% of readers. A political news story out of Michigan, Utah, or Texas (to pick three states at random) will likely upset Californians and vice versa. State and local news is more likely to comport with your views—assuming you somewhat agree with your community at-large.

I often discuss the news with friends and I usually get asked some version of the question, "How can you read the news so much? Isn't it just depressing?"

It can be, sure. But California has done quite a few things lately that I really like, and there's a lot of potential here for us to tackle big problems. If you're constantly focused on national news, it can feel like nothing ever gets done (because nationally nothing ever gets done), but locally that's just not true. Huge things are happening in California, and it feels good to know what they are.

I've also learned a lot about things I didn't think I would be interested in. The internet has trained us to seek filter bubbles and echo chambers, and to retreat into our known interests. A newspaper, because of its physical constraints, cannot be all things to all people. There are often pages of articles about topics I wouldn't say I care about, and yet I find myself reading them. On Sundays I find myself learning about up-and-coming bands, concerts, local events, and investing tips whether I wanted to or not, and it's been a really positive experience; not universally positive, but still very positive. Echo chambers aren't great, and physical papers are a good escape from them.

I'm not sure I had much of a high-minded conclusion to this retrospective, but I can say that I've enjoyed the anachronistic ritual of reading a physical paper, and I expect to continue doing so for a while.

1. They all get recycled. Don't at me.

Automated Podcasts With Automator & Overcast

I've mentioned before that I use Siri as an editing tool. I write a piece, lightly edit it, and then have Siri read it back to me. This helps me catch unintended grammatical errors and clumsy sentences. Building on that principle, Pine.blog and Hewell both ship with a feature that use iOS's AVSpeechSynthesizer API to read articles or location information aloud.

That said, I often find articles that I want to read, but after a long day staring at a computer screen, I don't want to actually read them. Lots of sites these days provide spoken audio for their articles—which is great—but the vast majority don't.

That's where Automator comes in.

Save Spoken Text to File

This Automator service simply runs a bash script that takes the contents of the selected text as input, feeds it to the built-in macOS say command, and outputs it to a file on the Desktop named using the contents in my clipboard.

Check out the full script
cd ~/Desktop;
# A hack to get stdin into say through Automator. For some
# reason simply saying -f didn't work for me.
while read line; do echo "$line" done < "${1:-/dev/stdin}" |
    say -o .spoken_text -f -

TITLE="$(pbpaste -Prefer txt)"
if [ -z "$TITLE" ]; then
    TITLE="Spoken Text"
fi
# Sanitize the article title. Writers love colons which macOS hates
TITLE="$(echo "$TITLE" | sed -e 's/[^A-Za-z0-9._-]/_/g')"

# Conver the audio and be quiet about it
/usr/local/bin/ffmpeg -i .spoken_text.aiff -loglevel -8 -y "$TITLE.aac"
rm .spoken_text.aiff

The script also uses FFmpeg to convert the audio to an AAC file so that I can then upload it to Overcast, my preferred podcast player.

By default, macOS will include Automater services in the right-click menu, but I've also bound the script to Cmd+Ctl+Shift+S (which is similar to my existing Cmd+Ctl+S shortcut for reading the selected text aloud).

The macOS Services Menu

Now, I can discover new articles to read, perform a quick set of keystrokes, upload the audio to Overcast, and then go for a walk while I catch up on the day's interesting news!1

I've provided the Automator service as a zip archive below if anyone wants to play with it.

⬇️ Save Spoken Text to File.workflow


1. There are a few quirks to this workflow still. Websites are filled with non-article content, so to avoid selecting those, I typically following the following steps:

  1. Turn on reader mode (Cmd+Shift+R)
  2. Copy the title of the article to the clipboard (Cmd+C)
  3. Select the article text (Cmd+A)
  4. Run my Automator service (Cmd+Ctl+Shift+S)
  5. Upload the new AAC file to Overcast

I admit, it's a little cumbersome, but it does work really well.

Retrospective On A Year Spent Writing

Around this time last year, I started writing a lot. Since then, I've published a book, started a policy blog, and started writing for a local paper. All in all, I think it's safe to say that I've written about as many words in the past 12-15 months as over the preceding 10 years.

A crude calculation of the word-count of this blog shows that I've written approximately 93,719 words. That's a lot, but considering that Going Indie is over 62,000 words and my published articles total almost 7,000, that 93,000 looks a lot less impressive.

$ find archive/ -name "*.md" | xargs -I {} cat {} | wc -w
   93719

I've learned a lot about myself and my writing in the past year. I've learned how to pitch articles, how to build up the courage to submit them, and how to research and write about complex topics. I've also gotten better at defining my assumed audience.

I'd always intended for this blog to be a place for me to write about whatever I wanted, and while I have written about a number of topics here, over time I've gravitated towards discussions of software, the tech industry, and personal matters. Last year, during the depths of the pandemic, I wanted to expand and write about public policy, but this blog never felt like the right place to do that. Hence why I started Democracy & Progress, and why I continue to write for the local paper.

Writing for both D&P and the paper has helped me focus my energy on working to better inform people and convince them to take an interest in a given subject. It's also helped me better understand the in-depth nuances of topics I previously thought I knew something about. Nowadays, I write not only as a way to inform and convince others, but as an exercise to educate myself. They say you don't fully understand something until you try to teach it, and that truism has held strong for me this past year.

Writing this much has also been a catalyst that pushes me to write even more.

I've really enjoyed this deeper commitment to writing, and while it remains just a hobby, it's an incredibly fulfilling and enjoying one that I hope to continue as long as I can.

Grove, A New Tree-Planting Wellness Game 🎉

Today marks the release of my newest app: Grove! Here's a brief description of the app from the product page:

Grove Logo

Grove is an augmented-reality game where you plant virtual trees in the real world! Collect the various kinds of trees, learn about them, and earn achievements all while getting outdoors and enjoying your virtual garden.

Grove is part game, part educational app, and part wellness app! In Grove you care for your trees and tend to your garden, and in turn, you stay fit and healthy.

Grove Product Page

If you're at all interested in the app, please do give it a try, and let me know what you think. I can't wait to hear your feedback (and see some of your trees)!

What is Grove?

At its core, Grove is part game, part AR-wellness app with some educational bits sprinkled in. The player plants virtual trees in real-life locations and builds out their virtual grove. Each tree is unique and randomly generated. Trees can be of several collectable types, each with their own unique artwork and animations. Players tend their grove by regularly watering, fertilizing, and harvesting from their trees and well-tended trees grow big and strong.

Each tree has a unique name, fun facts, and secret stats that determine the bonuses it gives when harvesting. Harvested resources can be sold at the market for coin that in-turn can be used to expand the player's grove and help tend their trees.

Grove is also a social app. Players can invite their friends to play with them and visit each other's trees. Lonely trees drop fewer seeds, but trees with friendly visitors are happier and more productive.

The app also includes some optional in-app purchases that can provide additional boosts, or unlock a secret Developer Diary and custom avatars to show off to friends.

As players tend and grow their grove, they earn achievements for their progress and rewards that help them advance further.

Where's the Wellness?

In Grove, as in real life, trees need space to grow; they can't be too close together. In order to plant trees, they need to be spaced apart and trees can only be watered, tended, and harvested from when the player is nearby. In essence, think of Grove as an app that encourages uses to go on a daily walk to tend their grove. Trees need to be spaced at least 30 meters (~100 ft) apart so there's plenty of walking to do when you've built up a full size grove. The app also awards bonuses and rewards for completing daily step goals.

And then there's Climate Change

Yup, you read that right. Climate Change is a gameplay mechanic.

Trees in the real world naturally absorb carbon dioxide from the air and turn it into wood, leaves, and branches. One technical name for processes like this is Carbon Capture and Sequestration, and tree planting is one technique that can be used to mitigate the effects of Climate Change in our world today.

In Grove, your trees capture carbon too! (virtual carbon that is) As your trees grow they capture carbon at the rate of real trees using data collected by the European Environment Agency. This helps players get familiar with this crucial emerging technology and get a feel for just how much tree planting can do to help the environment. Also, there's achievements for capturing lots of carbon.

A New Challenger Appears!

I've been working on Grove for the past six months and it's been a blast to build. I've never built a game before and while Grove is technically more of a wellness and education app than a game, there are certainly game-like components.

Grove is also the first iOS app I've built that heavily relies on custom assets. Usually I try to stick to drawing simple things in code or simply structuring the app to focus more on textual content, but for Grove that approach simply would not do. It needed to be cute, and it needed to be beautiful. I'd like to thank Grove's designer Victor Teles for everything he's done to give Grove a unique and adorable feel.

If you'd like to learn more about how (and why) I built Grove, give the app a try and unlock the Developer Diary. I've written a deep-dive there that goes into exactly why and how Grove came to be.

I'm sure I'll be going into more detail on various aspects of Grove in due time, and especially on my podcast: Indie Dev Life, so be sure to stay tuned for updates.

As always, thanks to all of my beta testers and to everyone who contributed to Grove. This launch would not be possible without you.

Check out Grove →

Unbounded Possibility Is Bad For Productivity

Being productive is hard; especially if you're working by yourself or working remotely. When you're working alone you have a lot of freedom, but that also means you have a lot of slack. No one is holding you to a schedule or deadline, and nothing is stopping you from procrastinating or getting distracted.

Even when you're focused, it can be hard to decide what to focus on, since there's often no required order in which things be done. From an objective perspective, whether I choose to call my bank today or tomorrow makes absolutely no difference. The same is true with what features I choose to implement on any given day. As long as the features get done, the order and the exact date they're completed isn't really important. Some features must be done before others for technical reasons, but others are completely unrelated and can be developed in any order. But this ambiguity is precisely the problem.

If you could work on anything at any time, what should you work on right now?

I'm going to generalize here: I don't think humans deal with unbounded possibility very well. We long for some sort of structure—or at least I do. When presented with the choice of doing any feature I want, I'm left unfocused and forced to decide—moment by moment—what features to build, which not only wastes time, but increases decision fatigue.

Lights in the Infinite Dark

Speaking with a friend earlier over the weekend, we stumbled on a maxim that I think sums up the solution pretty well:

Planning is the art of bringing order to chaos.

I've found that arbitrary deadlines, like arbitrary goals keep me motivated and focused. Without some sort of deadline or goal, I feel adrift and it's difficult to force myself to work on anything for a significant period of time. So I create artificial deadlines and goals, sometimes completely arbitrarily. Often times, I'll just pick a date on the calendar based on nothing but gut intuition, and then I change it later if necessary.

Without goals and deadlines Infinite Focus Infinite possibility gives no guidance.
With goals and deadlines Focus with Direction Adding goals gives you a direction.

By setting completely arbitrary deadlines and goals, I'm able to narrow down the unbounded, infinite possibility that is creating software into a simple series of steps. This isn't a new idea; tons of people do this. I just find it interesting to think of deadlines this way.

Whether your planning process involves ultra-precise scheduling, or just a notes file with some rough deadlines in it, having any sort of plan at all gives focus to your efforts and it guides you through the haze of infinite possibility.

Even if your deadlines are completely arbitrary and can be changed at will, having them is the most important thing.

Imports Are Endorsements

When you import someone's code, are you endorsing them?

At first glance, the answer might seem simple: of course not! And while it's pretty obvious that imports are not universal endorsements of the code's author, they aren't entirely void of meaning either. Endorsements aren't an indivisible quanta—some fundamental particle that cannot be divided—they exist on a spectrum.

Supply chains are tricky things

Importing code written by someone else is always a risky endeavor. Most often external dependencies work and work well, but they also expose your software to additional risk. The fact that you are willing to depend on someone else's code implies some kind of inherent bond of trust. It implies a relationship between the developer (or organization) and the code author. Importantly, it also implies that the developer finds the author's code valuable in some way.

Dependencies are part of a software's digital supply chain—along with any other provider we use to power our software. And in today's world, where alternative dependencies abound, many people understand that the various links in the supply chain aren't simply bound together out of mutual necessity. They choose to depend on each other, and so there are shared values and responsibilities that are common to all in the chain.

Using an example out of the news, Apple doesn't manufacture many of the components in its devices, yet when it's partner Foxconn is found to be abusing workers, we place some of that blame on Apple for choosing to work with Foxconn given their past behavior. Similarly, Google and Microsoft do not generate their own power, yet they've made efforts to rid their supply chains of fossil fuels, and the public has—rightly—heaped praise on them for these actions. From fashion to technology we understand that companies are somewhat responsible for choosing ethical and responsible supply chain partners. Why should developers be any different?

Our decisions matter

I think most people would agree with the decision not to use software written by an outspoken white-supremacist, but even that extreme example implies that there is some threshold where the author's views would impact the technical decision to use a given toolset. The literature, music, and film worlds are well-accustomed to this debate. Authors leave a mark on their work. How big that mark is remains a subject of debate, but there's no debate that the author has at least some impact.

Obviously big tech companies and organizations don't suffer because one company decides not to use their stuff—ideas require collective, industry-wide action to produce results.

The point is that our decisions to use Facebook's frameworks, Google's toolsets, Apple's platforms, or Amazon's services must be informed by their creators' behaviors and policies. Sometimes these decisions will be good for business, and sometimes not. Other times they might be incredibly beneficial or utterly unremarkable. Regardless of their effects, these decisions matter.

Some readers might bemoan this idea, claiming that I'm making software political, but everything is political in some form. Software doesn't exist in a vacuum and there are real consequences to our choices that echo beyond the apps and websites we build.

Whether we like it or not, the role of engineers is to manipulate the real world to achieve some end, and how we do that work has just as much import as what end we achieve.

I, for one, am driven to do what I can to mitigate the effects of Climate Change, so I host all of my new services in data centers powered by renewable energy and I'm working on migrating my existing services there as well. My hosting platform is a part of my digital supply chain, and I bear some responsibility for the emissions my services produce. The downside is that those servers are in Europe now, so my ping times suffer a bit, but to me that tradeoff was worth making.

Destinations matter, but the road to the destination matters too. Developers achieve our ends through importing other people's code, and those imports matter. Choose yours well.