BiteofanApple Archive About Code Twitter
by Brian Schrader

Hiking the Pacific Crest Trail

Posted on Sun, 19 Jul 2015

I've always wanted to hike the Pacific Crest Trail. Last week, a part of that dream came true. Last week, two friends and I hiked a section of the PCT, from Klamath Falls, OR to Crater Lake, OR. It was a trip long awaited, and it meant a lot to all three of us. Sean, I'd known from Scouting, and Nathaniel, I'd known from college. The three of us had talked about the possibility of doing a trip like this back in November of 2014, the weekend of Thanksgiving, and last week it finally happened.

What follows is an account of the journey

Day 1: Once everyone had arrived, Sean on the train, and Nathaniel and I on a plane, and no one from a car, we set off, with smiles on our faces, on a shuttle from Klamath Falls, OR to our starting point off Hwy 140. We'd spent months planning to get here, and we were all thrilled to get started.

The Gang's All here!

After being dropped off by the shuttle, we started hiking.

The Trailhead

At the trailhead I found a walking stick that someone had left there. The tip was covered in charcoal and the handle was already worn; someone had used and loved that stick, and now I was its next holder. It served me well.

Setting out

Needless to say, the trail was gorgeous.

Hiking Day 1!

About 3 miles into the hike we came across the first sign that we were actually on the right track. As we took a short break in the clearing, a hiker came up the trail from the south. He was about our age, on vacation from Germany, and had been hiking the trail since April. He told us that he'd been averaging 30 miles a day and that he planned to be done by July's end. He didn't stop long before setting back off again up the trail. We never saw him again.

A sign!

After 10.5 miles and at the end of a long day, we set up camp, and went to sleep. We were all exhausted.

Day 1 Camp

Day 2 was filled with amazing vistas and high ridges. Early that morning, as we were breaking camp, another hiker approached from the south. She asked us if we'd seen a blonde, German guy. "We saw him yesterday," we said.

"What, yesterday?! I hiked with him for 2 days, but he said I was too slow and he went on without me," she replied. This was the first of many times that another hiker would ask about "the German guy". Apparently, he was a legend.

The trail forested 1!

As we climbed up the side of a ridge, we took a break and admired the scenery. As we climbed, the cover of trees broke and gave way to some really beautiful vistas.

Resting on a ridge

Surprisingly, we had cell service up here in the middle of the Oregon wilderness

Ridge view

A bit further, we found a spectacular campsite at the top of the mountain.

Day 2: Camp

It had been a long, mosquito filled day. We cooked ourselves dinner, and, from our high perch, we watched the sun go down, mantling the mountains in the reddish-purple haze of evening light.

Day 3 would prove to be the day of our highest highs, and lowest lows (literally and figuratively). We would reach the tallest peaks, feel the best we had in days, and end on the worst note of the entire trip.

The day started off on an off note as we entered the what remained of the forest after a recent fire.

Burned out forest

Traversing such a breathtaking and scarred landscape was something none of us had ever done before, and it was an experience I recommend, if only for its wonder. The utter silence is, at the same time, both captivating, and deeply unsettling as no sound of birds, or buzzing of mosquitos broke the ever present silence.

After the journey through the silent, dead swaths of burned out forest, we found ourselves on the top of a hill. A perfect retrospective of our progress thus far.

Looking back

That mountain in the back, Mt. McLoughlin, was where we started 2 days prior.

After a short break, we found ourselves traversing some rocky ridges on the way to our highest high.

Rocky ridge

At this point, we'd finally ascended Devil's Peak, 7,329ft above sea level, our highest high.

Devil's Peak

As a bonus, the peak came stock with a most spectacular view to the west.

view from the top

The way back down would greet us with 5 water stops in the span of 2 miles, which we heartily enjoyed. Today had been our best day by far... at least until now. We had one mile left to go.

During that last mile, Sean slipped and fell in a stagnant, mosquito filled pond, and I received a lovely gift from the microbes in the river water. All of that, coupled with that last mile being the most mosquito filled section of the trail, so far, had beaten us down when we finally settled and set up camp for the night.

Unfortunately, as we all knew, the next day would be a hard one: it was our first of two whole days without the ability to fill our water stores. Whatever we had now was all we'd have to last 2 days. We'd known about this section of the trail before setting out, but this 20 mile section was devoid of water until we reached the campground at the end, Mazama. After a short discussion, we elected to try to cover the entire 20 miles in one day, double our so-far daily milage. The prospect of being left high and dry, and with another night between ourselves and a burger at the Mazama cafe, was too much of an incentive.

Day 4: I only have one picture of day 4; it was a long one. We did it though, all 20 miles.

The trail took us over more ridges, through more burned, and scarred terrain, and onto some lovely vistas, all of which are left to the imagination of the reader. We didn't have much time to stop and take pictures, we were on a tight schedule.

I do have this picture though, taken just after the 3 of us arrived at Mazama after 12 hours of hiking, and covering 20 miles.

At Mazama

We didn't last long that night after this. We showered, ate microwave burritos from the convenience store, and went to bed.

Day 5: We literally did nothing all day. Exhausted from the day prior, and a day (actually 2) early in the schedule, we took some time to relax, play card games, and eat the burger we earned the day prior. Once again, we encountered a hiker who, as they set up camp, asked us if we'd seen the German guy. When we said that we'd last seen him 3 days ago, he was impressed but not too surprised.

"He was really moving," was all he could say.

Day 6 and 7: For the next two days we bounced between Mazama and our final destination because of logistical reasons. I'm combining these days because, really we spent them in the same place, Crater Lake.

Crater Lake

There were other happenings during the day like how it rained on and off, or how we were left behind by the trolley and had to hike the 5 miles back to our camp site that evening, but really the majesty of Crater Lake blew all that out of the water.

Crater Lake, again

Crater Lake, again, again

At the end of the day, we spent time at the lodge, and at the meadow next door overlooking Crater Lake and all its immense glory, but as these things usually go, it was soon time to get back on the shuttle and head home.

Before that, I needed to leave something behind. It was time to leave my walking stick in a place akin to where I'd found it, at a trail marker on the PCT.

My hiking stick

On the shuttle ride home I found myself in a somber mood. All of that awe-inspiring scenery, the grand wilderness, the epic feeling of exploration, and the grandeur of Crater Lake were all behind us now. We'd had an amazing time together, but it had come to an end, and now here we were, flying home to our respective lives, all right where we left off.

My hiking stick

Until next time.

Afterword

This was my first excursion down a small part of the Pacific Crest trail, and I don't know when my next will be, but it can't come soon enough.

Well, maybe it can wait until after my bug bites heal. Yeah, that would be good.

WWDC and Open Source Swift

Posted on Tue, 09 Jun 2015

I'm pretty happy with yesterday's announcement (excluding the "One more thing..." part). OS X got a very understated release, but that's because, I suspect, that it seems to be mostly bug fixes, and that's great news. I'm mostly excited about the bug fixes and performance enhancements, but there were some solid updates to Safari, and Maps as well (transit isn't available for San Diego yet, but hopefully it will be soon).

On the iOS front, the iPad got all the love, and that's long overdue. The "new" (Surface Pro style) multitasking is nice, and the popovers look cool (Netflix and Twitter at the same time!). I'm also a fan of the new battery improvements and the Spotlight search API (Finally).

To me though, the biggest news of the whole conference was the announcement that Swift will be Open Source1. For Apple, this means that Swift will be placed alongside Go and Rust in the new, hot systems programming language category. Chris Latner expressed his desire for Swift to be open source last year at WWDC, and I'm glad he was able to convince the higher ups. Making Swift open source allows it to be a real alternative (and competitor) to Rust and Go, and I can't wait to see what people do with it. Swift interoperability with Python is also something I look forward to hearing about since Python already interfaces with Rust quite nicely (examples). Using Swift, Rust, or Go as your low level language instead of C has a lot of advantages, and its great to see Apple keep pace with the outside world. More competition and choice in the systems programming language world is always good, and there will shortly be 3 great options to choose from.

1. Yes I'm aware that Apple said, "later this year." Yes, I am also aware that FaceTime was supposed to be an Open Standard. I contrast with this: ResearchKit is on GitHub; this is a new Apple.

Software 'Engineering'

Posted on Mon, 25 May 2015

Ben Adida:

...most people have a pretty good idea of the trust they're placing in their doctor, while they have almost no idea that every time they install an app, enter some personal data, or share a private thought in a private electronic conversation, they're trusting a set of software engineers who have very little in the form of ethical guidelines.

Where's our Hippocratic Oath, our "First, Do No Harm?"

I've talked about this before. Software Engineering is unlike any other field that calls itself 'Engineering'. Unlike licensed engineers in other fields, there's no code that binds Software Engineers to a strict set of ethics, and no state certification to revoke when Software Engineers are found malpracticing. This kind of lax attitude toward development has indeed helped the industry boom the way it has, but success hides problems.

It's shocking really, coming from an education in Aerospace Engineering, the lack of precautions that a lot of software engineers take when designing their systems. In school, from the beginning, we had concepts like "factors of safety", "margins of error", and "graceful failure" drilled into us. Included in our final project —creating a design for a supersonic business jet— was the requirement that it be able to complete its entire mission with One Engine Inoperable (OEI). If one of the two engines burst into flames at takeoff, the plane still had to be able to fly. It's these kinds of regulations (FAA in this case) that enable commercial airlines to have the amazing safety record they do; people's lives are at stake. A professor one told my class, "These calculations are important. If your numbers aren't right, someone could die."

Personal data is extremely valuable, and precious to the person it belongs to. Although their life may not be at stake, their finances, their livelihood, and their personal affairs might be. Software has reached a level of pervasiveness (and arguably did so years ago) that an engineer's decision (or lack thereof) can affect millions of people. If any other branch of Engineering tried to design and build a something that would affect that many people (their data or their wellbeing) you'd bet they'd be licensed.

the responsibility we have as software engineers →

Python multiprocessing and unittest

Posted on Tue, 28 Apr 2015

I've been having an issue with unit testing Microblogger when my tests need to use Python's multiprocessing module. I've been looking at this code for days now and I can't seem to find the bug. I'm hoping that by writing down my thoughts here, I can think through the problem.

Basically, the test is trying to verify that a User object can be created with information from a remote XML feed. The test gives the User module a URL and tells it to fetch all information at that resource.

    def test_cache_user(self):
        user = User(remote_url='http://microblog.brianschrader.com/feed')
        user.cache_user()
        self.assertEqual(user._status, dl.CACHED)
        self.assertEqual(user.username, 'sonicrocketman')
 

The cache_user method starts up a crawler to go out and parse the contents of the URL provided.

    def cache_users(users):
        ...
        from crawler.crawler import OnDemandCrawler
        remote_links = [user._feed_url for user in users]
        user_dicts = OnDemandCrawler().get_user_info(remote_links)
        ...

Everything is ok still. Inside that OnDemandCrawler().get_user_info() method, the OnDemandCrawler crawls the URL given and then calls self.on_finish(). This is when things get funky.

    def on_finish(self):
        self.stop(now=True)

The stop command tells the crawler to shut down, the now keyword just tells it to force stop the crawling process and don't wait to cleanly exit.

If we look at the source to the microblogcrawler (v1.4.1) we see that stop does the following:

    def stop(self, now=False):
        ...
        if now:
            # Try to close the crawler and if it fails,
            # then ignore the error. This is a known issue
            # with Python multiprocessing.
            try:
                self._stop_crawling = True
                self._pool.close()
                self._pool.join()
            except:
                pass
        ...

The curious part is that self._stop_crawling = True part. In the tests for the microblogcrawler both forcing the crawler to stop and normally stopping it work fine. The issue arises when trying to stop them in a unit test. For some reason the crawler doesn't stop.

Here's a sample crawler and the output it produces when run as a unit test:

    class SomeCrawler(FeedCrawler):
        def on_start(self):
            print 'Starting up...' + str(self._stop_crawling)
        def on_finish(self):
            print 'Finishing up...' + str(self._stop_crawling)
            self.stop()
            print 'Should be done now...' + str(self._stop_crawling)

>>> python -m crawler_test
>>> Starting up...False        # Correct
>>> Finishing up...False       # Correct
>>> Should be done now...True  # Correct
>>> Starting up...False        # lolwut?

For some reason the crawler isn't receiving the signal to stop. Looking at it from my Activity Monitor it appears to stop (the 4 worker threads are closed), but then the crawler creates 4 new worker threads and does it all over again.

The last step of this process is inside the crawler itself. The crawling process is controlled by the self._stop_crawling attribute:

    def _do_crawl(self):
        ...
        # Start crawling.
        while not self._stop_crawling:
            # Do work...
            ...
            self.on_finish()

From this code, if the _stop_crawling attribute is set to True, then the crawler should finish the round it's on and close down, but the value of the attribute doesn't seem to be sticking when it's assigned in the stop method above.

If anyone has any ideas as to what the issue could be, I'd love to hear them. I'm pretty much out of ideas now. As I said before, the tests in the microblog crawler (which are not unit tests) work fine. The issue only comes up when running a test suite through unittest itself.

Microblog Crawler v1.4(.1)

Posted on Sat, 25 Apr 2015

Version 1.4.1 of my MicroblogCrawler is out on PyPi! Technically v1.4 was out last week but it had a fairly large bug that needed fixing. 1.4.1 has patched it and it's ready for prime time.

v1.4.1 is full of enhancements, a few of which are listed here:

  • Calling stop now actually stops the crawler. This bug was due to a nasty bug in Python's multiprocessing module (9400). The crawler now alerts you when such a problem arises by outputting it through the on_error callback.
  • Fixed a bug that would cause feeds to throw errors if no pubdate element was found. Elements are not parsed but are discarded, and on_error is called.
  • Fixed a major bug when attempting to stop the crawler immediately.

The full version notes are available here.

The major enhancement in this version (besides the graceful exiting) was the addition of a workaround for a bug in Python's multiprocessing module. The bug has to do with what happens to exceptions raised in child processes. When they are raised, they are pickled and sent back to the parent process. The problem arises when an exception is not pickleable. The child process hangs and never exits. The interesting thing is that the bug was first reported in 2010 and affects all versions of Python since 2010 (i.e. 2.7, 3.2, 3.3, 3.4). This bug has been baffling me since I started converting the crawler to be multiprocessed, and its nice to finally have a workaround.

If anyone out there is using MicroblogCrawler, I'd love to hear from you, and pull requests are very welcome!

PEP 484 - Type Hints for Python

Posted on Wed, 08 Apr 2015

Guido van Rossum:

I'm for any improvements that will help my favorite language run smoother, with fewer errors, and maybe faster someday*.

This PEP aims to provide a standard syntax for type annotations, opening up Python code to easier static analysis and refactoring, potential runtime type checking, and performance optimizations utilizing type information. Of these goals, static analysis is the most important. This includes support for off-line type checkers such as mypy, as well as providing a standard notation that can be used by IDEs for code completion and refactoring.

There's been a big push for better static analysis in Python for the last few years, and there've been attempts like this before (see Cython) and having a language level standard for Type Hints would bring the benefits to all the various Python implementations.

# An example of the proposed type hint syntax.
def greeting(name: str) -> str:
    return 'Hello ' + name

I admit, the new syntax looks very Rust/Swift-like, and that's probably by design. One thing that worries me, and which isn't obvious from that code sample is that Python Type Hints will (must) include generics and blocks (i.e. lambdas, closures, etc). When those get into the mix, the Type Hint system starts to look a little messy.

from typing import Mapping, Set

def notify_by_email(employees: Set[Employee], overrides: Mapping[str, str]) -> None: ...

Even though that code isn't particularly pretty, the Type Hints can help the static analyzer find errors that could potentially be very hard to track down. As I said, I'm completely in favor of this addition to the Python syntax.

As a final note, for those of you worried that Python might be changing to a statically typed language, fear not.

It should also be emphasized that Python will remain a dynamically typed language, and the authors have no desire to ever make type hints mandatory, even by convention.

PEP 484 - Type Hints →


* According to the PEP, the goal of Type Hints will not be performance based, but they do go on to say, "Using type hints for performance optimizations is left as an exercise for the reader," which keeps me hopeful that PyPy or maybe even CPython could use them for that purpose as an added benefit.

Archive

Subscribe to the RSS Feed. Check out my code on GitHub
Creative Commons License
BiteofanApple is licensed under a Creative Commons Attribution 4.0 International License.