BiteofanApple Archive About Apps Microblog Photos Links
by Brian Schrader

A Sneak Peek: Pine for iOS

Posted on Fri, 15 Jun 2018 at 12:42 PM

It's been a few busy weeks since I first launched Pine, a new kind of Feed Reader. And while it's not quite ready for prime-time yet, here's a sneak peek at the official Pine app for iOS.

The app will be free and require a Pine subscription to use. My favorite features though are the Dark Mode and Dark App Icon support.

The first version of the app is pretty bare bones with only only a few screens: your timeline, likes listing, and full article view. And while you'll of course be able to share and like posts, Version 1.0 will not have any sort of search or follow/unfollow support (though that's coming soon afterwards).

Deciding what to cut and what to keep is always hard, but I've once you've set up Pine in the browser, this version of the app should have enough features for a nice day-to-day experience.

I've been using the app for a couple of days now and it's really shaping up into something special. Hopefully it will be ready for launch soon.

Announcing Pine: Microblogging for a New Generation 🎉

Posted on Wed, 23 May 2018 at 11:30 AM

Pine Logo

Today I'm announcing Pine, a new kind of RSS Reader and Microblogging client. I've been developing Pine, in addition to a bunch of other projects for a while now, and it's finally time to show it off to the world.

In a classical sense, Pine is a Feed Reader, but it's really more of a successor to my old Microblogger project from way back. Not only can users follow the people and sites they want, but you can also use Pine to post to your own Wordpress site and find new sites and people using the built in search. In a lot of ways Pine and Micro.blog are similar with a few key differences and, from what I can tell, both Manton and I started working on them at around the same time. Pine is my contribution to a new ecosystem of open social networking tools and services. Anyone on Micro.blog can follow people on Pine and vice-versa.

Pine is also my first venture into building paid software. It's $5 per month and at that price I don't have to sell ads or do anything shady to earn money.

I've been reluctant to post anything about Pine until it officially launched, but now that it has, I've got a few interesting design decisions I'd like to talk about. Building a feed reader has always been on my to-do list, and it turns out that there's a lot more to it than just parsing feeds.

I've got a lot of great new features planned, including an iOS app which should be coming soon. If you're interested, give Pine a shot, and let me know what you think!

Pine →

The People vs. Democracy

Posted on Sat, 19 May 2018 at 01:46 PM

I just finished Yascha Mounk's excellent book The People vs. Democracy. In it Yascha talks about how the two main parts of a Liberal Democracy are now at odds with each other, not working together.

  • A democracy is a set of binding electoral institutions that effectively translate popular views into public policy.
  • Liberal institutions effectively protect the rule of law and guarantee individual rights such as freedom of speech, worship, press, and association to all citizens (including ethnic and religious minorities).
  • A liberal democracy is simply a political system that is both liberal and democratic — one that both protects individual rights and translates popular views into public policy.

...Democracies can be illiberal...[especially] where most people favor subordinating independent institutions...or curtailing the rights of minorities they dislike. Conversely, liberal régimes can be undemocratic, despite having regular competitive elections.

I've been thinking a lot about that distinction lately. Democracy is kind of a holy word in our society, a thing that cannot be bad, but as Yascha points out, the original democracy in Athens was known for executing writers and philosophers, and not extending suffrage past its male elites. It's the combination of Enlightenment era liberalism and democracy that gives us the systems we see around us today. Yascha's main point is that this combination seems to be falling apart all over the world as systems either become less democratic, or less liberal, or both at the same time.

Yascha also dives into the possible causes of this split, and how to fix it; from social media to housing prices. It's a fascinating read, and it really changed the way I think about today's political climate for the better. I haven't torn through a book that quickly in a long time.

Safari's WebSocket Stream View

Posted on Sun, 04 Mar 2018 at 10:16 PM

I've mentioned before that Adventurer's Codex uses XMPP over WebSockets for it's live-chat and party features, and we're planning to add lots more of these kinds of features in the future. That being said, as a developer, I've had a pretty difficult time building features that rely on WebSockets because it's kind of annoying to see exactly what is being sent and received over the connection. I know that Chrome has some features to help with developing WebSocket applications, but I've never had a good experience with them, and I use Safari as my default browser for both casual use and for web development.

That said, I stumbled across a new WebSocket Stream View in Safari's Dev Tools and, just like when I discovered that Git supports notes, I got really excited. Clicking on an active WebSocket connection now brings up a live timeline of every message sent and received by the browser over that connection. It even shows the ping/pong frames that the browser occasionally sends to verify that the connection is still open.

safari websocket viewer

Safari's New WebSocket Stream View

I'm hoping that this new view will make it a lot easier to debug problems because I can now see exactly what's being sent over the connection. I'm not sure when this was added to the Safari Dev Tools, but it's definitely very welcome.

Git's Hidden Feature: Notes

Posted on Sun, 04 Mar 2018 at 08:37 PM

When I'm programming I tend to create a lot of experimental branches to test my ideas before actually implementing them, and a lot of times I'll leave these branches in some half-state or with uncommitted changes. Then when I come back I've forgotten what I was doing and why. After running into exactly this problem earlier today, I started wondring if there was something built into Git that would give me a place to write notes to myself without committing.

Well, apparently there is: Git has built in notes!

According to the docs you can use the command git notes which:

Adds, removes, or reads notes attached to objects, without touching the objects themselves.

It's always exciting for me to discover a cool new tool, and even more so when I learn more about one I already use.

Git Notes Documentation →

Real Life Science Fiction

Posted on Tue, 06 Feb 2018 at 05:59 PM

Earlier today SpaceX launched their first Falcon-Heavy Rocket with Elon's personal Tesla onboard. All in all the launch went really well, and congrats to everyone at SpaceX for making it possible 🎉. There was a lot of amazing technological development and hard-core engineering that had to happen to make all of it possible, and it was amazing to watch, but one part in particular felt surreal.

Falcon Heavy is an absolutely huge rocket. According to SpaceX:

Only the Saturn V moon rocket, last flown in 1973, delivered more payload to orbit.

For me though, the real crowning moment, the minute everything stopped being real and starting being Science Fiction was when the first stage boosters came back down, and landed in perfect sync; it felt like complete magic.

Most days we're trapped in the real world, but things like today's landing are so awesome that, for a second, it can feel like we're living in a Fantasy or Sci-Fi universe. While I don't really approve of SpaceX's corporate practices, they do truly amazing and fantastic work, and the people there are top notch. I can't wait for the next launch.

The Web as a Social Network

Posted on Fri, 02 Feb 2018 at 12:12 PM

Brent Simmons on Micro.blog

Twitter and Facebook are convenient, sure, but so are fossil fuels, and the cost was similarly unknown for a long time. But now we have some idea just how bad these things are for the world....

[With Micro.Blog] your posts are just a normal, everyday part of the open web. At this writing, mine appear on micro.inessential.com — but it’s on my to-do list to have those appear on my main blog (this blog) instead. (Probably won’t happen until after I ship the app I’m currently working on.)

And this is how it used to be, and how it never should have stopped being: my blog is me on the web. I own my blog: I own me.

Here's where I'd normally write something like this:

Brent is right. Blogs and microblogs can and are the future of social networks. They allow you to keep control of your content, and protect you from huge, often nefarious corporate whims...

But any reader of this site will already know my feelings on the Open Web and Social Networking. So this time I've decided to make this post a retrospective on what I've said before, and how my opinions, and the web has changed over the last few years.

The Open Web Series Collection

My first post about the Open Web was from July 2014, when this blog was still in it's infancy. Since then I've basically written about this subject whenever it's come up in the circles I travel in. At first I was just concerned with preserving my site and my thoughts for the future, but over time I morphed into the Open Web loving nerd you see before you today.

How much history is lost by those who live it not writing it down. With the internet, we have the ability (assuming storage is cheap) to preserve everything we ever say, think, write, or post. That's really powerful, and possible accountability and privacy issues aside, its probably the most important use of an invention in the history of man.

I've retweeted this tweet multiple times, mostly because it's hilarious, but every time I want to do so, even though I have it saved in Instapaper and favorited on Twitter, its easier to google the text of the tweet (as close as I can remember) and let Google do the work.

After a while I got it into my head that RSS could be used for a new (old?) kind of social network, and I've been rolling with that idea ever since.

What would it mean to have a social network (like Dispora) that is decentralized, and independent of corporate ties? Could such a thing exist?

Then I was (am still) mad at Twitter for a while, which just stoked the Open Web flames.

Could it be that the time for new social networks is past us? I hope not. RSS succeeded as an open standard, and so did email (though email had an early start). The podcasting industry is growing, powered by RSS... RSS use floundered after Google Reader shutdown, but RSS is still with us. That's important.

Create systems that are ambivalent about the open or closed web. If I create a tool that's good at posting content to Facebook and Twitter, it should also post to RSS feeds...

I tried making a standard once, but putting something on GitHub in a README doesn't make it a standard.

A few weeks ago I began drafting a new standard for open, platform independent communication service. You can think of it as Twitter meets RSS...

One of the critiques of RSS feeds in a world dominated by Facebook and Twitter is that RSS just isn't fast enough. You can't hope to achieve what Twitter calls "in-the-moment updates" and "watch events unfold" if your client is polling each web site's RSS feed once an hour for new microblog posts...

After a while I started noticing that a lot of creators and bloggers I followed were moving back to open, platform-less distribution methods (like CGP Grey and his RSS feeds and email lists).

[Social Media Platforms] have to do this to make money. It just sucks that how they make money is by compromising the people that made the service what it is (cough Twitter cough).

I was still mad at Twitter.

Twitter is a complementary medium to blogging, but it's not a replacement...

By knocking down a few walls and moving some furniture around, blogging is preparing for a comeback, and we'll all be better off for it.

More recently though I've gotten more realistic and still more optimistic about the future of social networks and openness. Facebook and Google's Algorithms still loom high and mighty, but I started to see a resistence forming, and then Manton Reece released Micro.blog

I've long heard people say that we've lost video, music, and messaging to the walled worlds of YouTube, iTunes, and WhatsApp/Facebook/Twitter respectively, and while I believe that is the current state of the web, I don't believe it's the end-state.

If we're going to allow... algorithms to be such a huge part of our lives, which I don't believe is necessarily a bad thing, then they should at least be subject to some sort oversight.

I've written lots of stuff about services like Manton's, and the Open Web in general. It's really important that we preserve not just the way the Web used to work, but as Manton says: the way it should work.

The future looks bright for the open web and social networking, but the fight is ongoing.

Primitive Tech is Here for the Long Haul

Posted on Fri, 24 Nov 2017 at 02:46 PM

John's most recent Primitive Technology blog post has me pretty excited. The new video is great, as his videos tend to be, but the post mentions something a bit more exciting.

I bought a new property to shoot primitive technology videos on. The new area is dense tropical rainforest with a permanent creek. Starting completely from scratch, my first project was to build a simple dome hut and make a fire.

This is just me speculating here, but he wouldn't be buying a whole new plot of land to shoot videos on if he wasn't planning on making more videos long term. While I never had any reason to believe he'd be stopping anytime soon, such a big purchase is great news for him, his channel, and fans like me.

Primitive Technology Blog →

The FCC Moves to Dismantle Net Neutrality

Posted on Thu, 23 Nov 2017 at 09:18 PM

Jonathan Shieber for TechCrunch →

Federal Communications Commission Chairman Ajit Pai today made good on his long-standing pledge to tackle regulations established in the last administration designed to protect the distribution of internet content.

On Tuesday, Pai distributed to the other commissioners at the FCC a draft of his suggested rule changes under the auspices of the “Restoring Internet Freedom Order.”

The move sets up a December 14 vote at the FCC that could have broad ramifications for the entire internet. Under the rules established by the Obama administration, internet providers are required to provide open access to their networks for all digital content.

I'm really sad to see the FCC going forward with their plan to dismantle the Net Neutrality protections. What's worse is that Ajit Pai seems to know full well what this will mean for the web, and seemingly no amount of public comment against his proposal can disuade him.

It seems like the only recourse we really have now is either to chip away at Ajit Pai's resolve with public comments before the December vote, or wait for some act of Congress to reverse the decision: something I can't even imagine them doing.

Help Protect the Internet by Contacting Your Representative →

MyGeneRank: Behind the Scenes of the Newest ResearchKit App

Posted on Wed, 25 Oct 2017 at 10:11 AM

I'm super excited to announce that MyGeneRank, an app that I've been working on at my jobby-job at the Scripps Translational Science Institute for a year and a half, is now available on the App Store, and the source code is available on GitHub!

I've wanted to talk about this project for a while, I've written many unpublished posts about how it works, and now the time is finally right. If you're looking for the scientific or research parts, I'm going to leave that to the paper we published. I want to talk more about my experiences and what I've learned in building the system.

As a quick overview: MyGeneRank is a ResearchKit based research study app aimed at providing users with their Genetic Risk for diseases, and measuring their reactions to this information. The first disease being Coronary Artery Disease. I'm definitely not a doctor, statistician, or biologist, and everyone else on the team handled all of the scientific work, but I am a Software Developer and I worked on the vast majority of the API, Computation Engine, Website, and iOS App development, DevOps, and System Administration as the sole developer. I learned a hell of a lot during the last year and a half and looking back, I'm not sure how I even got this far. Since the source code is available, at least for the API, (iOS source is coming) so now you too can see my mistakes and pass judgement! 🎉

From a Technical Perspective

Vaguely, MyGeneRank's backend has three main parts: a database (Postgres), a Django REST API (which is open source), and what we've called a Computation Engine. All of it runs in-house, maintained by yours truly. The API and database are pretty self-explanatory, and the "engine" is really a Celery cluster and Redis queue which runs, among other things, a series of Python wrapped command-line tools and custom R scripts to calculate a person's genetic risk given their 23andMe genotype data. While the computation stuff is a sort of special case (what with the CLI tools and R scripts), the API's design goal was to be as industry-standard practice as possible. It's 90% covered with tests, leverages Travis-CI, uses DRF and Celery for the vast majority of its work, and everything runs in Docker containers on CentOS.

If this stack sounds familiar to readers of this site, then you're catching on. In my post about Adventurer's Codex's stack I spelled out basically the same setup. In truth the stack for AC was heavily influenced by MyGeneRank. I took everything I learned building MyGeneRank and ported it to Adventurer's Codex a year later. That's how developers work, we do things once, then copy-pasta it everywhere.

Scientific Computing at Scale: Performance and Throughput

MyGeneRank has very demanding computational needs. Currently, we have 178 cores and almost a terabyte of RAM powering the app and its backend. Turns out, calculating a person's genetic risk, even using genotyping data and not NGS, requires a lot of computational power. Scaling this kind of intense scientific computation for public use was one of the most challenging (and enjoyable) parts of the project. But even now I don't really have many concrete answers to the problem other than the twin suggestions: add more cores and make your work as functional and therefore parallel as possible.

Into the Weeds for a Bit

The calculations needed to return a given user's genetic risk score can be broken into roughly 110 individual tasks and is mostly trivial to parallelize. What takes ~110 minutes of CPU time per user can be done in 3.5-4 minutes of wall time on our current system, but as any web developer knows, even that kind of processing time is hard to scale. The first couple tasks are run in series and then two chunks of tasks are run in parallel. The first chunk contains a single task which calculates the user's genetic ancestry, and the other chunk has 52 two-part tasks. This means that at any one time, 53 tasks per user are running during the bulk of the computation. These first 52 tasks take ~1.5-1.8 minutes each depending on which chromosome they're processing (some are bigger than others) and then the second batch takes about the same amount of time per chunk. The genetic ancestry takes ~3.2 minutes. Once all of these tasks are complete there's a final step that calculates the actual risk score, which is fairly instantaneous. What this means is that the time from start to finish for a given user's score is parallelize-able up to ~54 cores, then it's core speed that matters, which is harder to improve. The extra cores we have allows us to calculate more scores at once, but even with our huge core count, we can only calculate ~3-4 users' scores at a time. The good news is that all of the steps are really good at keeping memory use low; CPUs are the bottleneck here.

Improving API and Website level performance is much more straightforward than doing the same for the backend. Like most sites, MyGeneRank sits behind an Nginx Reverse Proxy with some out of the box microcaching for popular pages.

At time of writing, I'm not sure what the load will be like when we finally announce the study publicly, but I've spent a lot of time worrying about and trying to ensure that the site can handle the loads we hope for. There's been a lot of interesting news and blog posts over the years about what kind of download numbers an app, and especially a research app can expect, and I wanted to build MyGeneRank with those kinds of numbers in mind. Once the project has hit its first month I'm going to do a retrospective on how it all went and we'll see if my performance enhancements were enough.

Lessons Learned

There's a lot of little things that I've learned in building MyGeneRank (and later Adventurer's Codex). When we started, I'd worked on a few toy iOS apps and a few corporate web projects, but MyGeneRank turned out to be of a completely different scale.

Before MyGeneRank, I'd never used Django, or Django REST. I'd heard of them, and had a friend who used them, but aside from a few toy projects in Flask, my web experience was in front-ends or Java/Spring (and I guess PHP). My work at that time was mostly in writing analysis pipelines in Python and since it's my preferred language, I wanted to use it for MyGeneRank. To this day, the structure of the API project is a little wonky and apps aren't where they should be; both are cruft from those early days. I try not to worry about it too much since I was learning as I went, and this kind of legacy cruft is impossible to avoid unless you knew everything at the start, and we most assuredly didn't. I can say that Django/Django REST has shown me just how boring building websites really is because it does most of it for you automatically and supports anything you'd ever really need right out of the box; you should definitely use it.

The modern web is really complex and there's a reason that it takes so many skilled developers to build large systems. Server setup, administration, DevOps, reporting, and application development are all sub-disciplines unto themselves (which is why they're separate jobs at most places). And I've found that jumping in and out of these different worlds can result in a sort of Programmer's Jet Lag as your body adjusts to the new environment after spending days in a completely different one.

On the native side, Apple's frameworks can be fun to use, and their OS frameworks, documentation, and user guides are world-class, but their tooling can also be frustrating and slow at times. The iOS app is written entirely in Swift and that has had some major effects on the development. Swift's tooling is still very new and the language has changed drastically since it came out. Having worked in both, I can say that, while I do enjoy Swift, nothing has made me appreciate the maturity of Python more.

Overall, my advice for building these kinds of systems is the same as when I wrote a similar post about Adventurer's Codex:

...ask people who've done it before... The internet is great, but it's actually pretty difficult to find out how to design modern web systems from scratch with just a vague notion and Google.

MyGeneRank, to me, represents my passing from a junior to a senior developer in a lot of ways. By no means have I learned all there is to know, but having now built two large web projects, and being the sole developer for one of them, I feel like a different person from the one who started the project a year and a half ago. I'd love to know what you all think of the source, and if you find a bug, file an issue please.

Archive

RSS

Creative Commons License