03. February 2017

Scripting in Racket

A few years ago I was struck by Adrian Laiacano’s piece “Keeping tabs on your data analysis workflow.” Laiacano offers a compelling argument for doing one’s best to codify ad-hoc data analyses into composable tools. This led me on a very big kick to write more generalized tools instead of one-offs.

In my case, the obvious language choice is Python. I love it, and rarely get to write it for my day job (it’s a Scala team). But working with your go-to language can be a mixed bag. When you use one language a lot for the same type of task, you tend to think and plan in that language exclusively, where problems reveal themselves through approaches that feel most natural to that language. Nothing wrong with that, but it makes it harder for me to learn new languages if I think in one favorite language at the expense of others.

At any rate, I was looking at an old Scheme book and wondered if I could use Racket for scripting. I thought I would try it with simple query: what’s my team working on now in Pivotal?

I started off with the command line. How can I get args from the command line into a basic runner function?

(define (usage)
  (printf "Arguments needed: <token> <project-id>\n"))

(define (runner args)

(define (main)
  (let ([cli-args (current-command-line-arguments)])
        (if (> 2 (vector-length cli-args))
            (runner cli-args))))

How do we call a web service and get back a string? The net/url package gives us a basic GET call. We can then copy-port the response into a string:

(require net/url

(define (fetch url header)
  (define in (get-pure-port url header))
  (define out (open-output-string))
  (copy-port in out)
  (get-output-string out))

(define (api-call url header)
  (fetch url header))

To get the current Pivotal iteration for a given team/project, you pass an API token in the header, and a project-id on the URL. I also want the readable names of the first owner of each story, so I’ll need to grab our org’s membership from the memberships endpoint.




(define (pivotal-url project-id)
  (define url-string
    (string-append PIVOTAL-URI "/" project-id ITERATION-PATH))
  (string->url url-string))

(define (members-url project-id)
  (define url-string
    (string-append PIVOTAL-URI "/" project-id MEMBERSHIPS-PATH))
  (string->url url-string))

(define (pivotal-header token)
  (list (string-append "X-TrackerToken:" token)))

As you would expect, the API returns a lot of JSON.

The Racket JSON library (specifically string->jsexpr) helps convert the JSON to a list of very small structs:

(require json)

(struct story
  (name owned-by-id kind current-state owner-name) #:transparent)

(define (iteration-stories iteration)
  (hash-ref (car (string->jsexpr iteration)) 'stories))

(define (story-node->story node users)
     (hash-ref node 'name)
     (hash-ref node 'owned_by_id)
     (hash-ref node 'kind)
     (hash-ref node 'current_state)
     (hash-ref users (hash-ref node 'owned_by_id))))

(define (extract-stories iteration memberships)
  (let ([users (extract-users (string->jsexpr  memberships))])
    (map (λ [s]
           (story-node->story s users))
         (iteration-stories iteration))))

(define (extract-users nodes)
  (let ([m (make-hash)])
    (for ([n nodes])
      (let* ([person (hash-ref n 'person)]
             [id (hash-ref person 'id)]
             [name (hash-ref person 'name)])
        (hash-set! m id name)))

This code extracts stories from the iteration JSON. extract-users pulls user nodes from the memberships API call, and sets the id/name pairs into a hash that matches human-readable names with each story’s first owner id. When you pass that hash to the story-node->story function, each story struct has the human-readable name.

All that is left is binning the stories by name and printing them out. This is just a command-line script, so we don’t get too fancy here:

(define (grouped stories)
  (group-by (λ (s)
              (story-owner-name s))

(define (print-grouped grouped)
  (let ([formatter
         (λ (story)
           (~a (story-current-state story)
               " - " (story-owner-name story)
               " - " (story-name story)
               #:max-width 90 #:limit-marker "..."))])
    (for ([g grouped])
      (for ([s g])
        (printf "~a\n" (formatter s))))))

group-by does just what it says; then some rather ugly formatting (from racket/format) prints out a line that doesn’t take up too much space.

When you run this you get a birds-eye view of what’s completed, and what’s underway, for this iteration. You can read the whole script in this gist.

So what did I learn? I learned that I write Racket like a Python developer. Functions doing text manipulation, filtering, and taking fairly few chances.

I’m still not great at naming, abstractions, or writing anything extensible in Racket. The other funny thing is that Racket’s real superpower — creating new languages — does not come into play here. It’s just a script. So I wonder if this really fits the task.

On the other hand, it was very fun to write, because Racket asks you to think differently about your problem. Yes, I could have written this with thirty lines of Python, but what would I have learned?

01. November 2016

Managing Difficult People

People are difficult in different ways, and not all of them are catalogued here. These types are all rough approximations. I’ve experienced them all:

I have my own way of doing things

  • Ted likes to come in late, every day. He asks that we never schedule meetings before 10 because it’s really hard for him to come that early, which effectively means the team’s schedule is determined by one person * Alice doesn’t really want to work on projects that aren’t “interesting.” She’ll say “project X isn’t interesting to me” and will get out of working on X for that reason. Sometimes she’ll bargain with you, or with co-workers, in order to work on the stuff she finds “interesting”
  • Bob asks: “Is it ok if I don’t do a code review? I feel like it is really hard for me to be exposed to that kind of critical review.”
  • Carol says “well I don’t really think that [working on technology X] is why I was hired”

Tortured genius

Rather pervasive in our line of work. In their own assessment, they’re quite good: good code, good brain for programming trivia, good on the fundamentals of Some Aspect of Engineering. Tend to not champion others or even recognize skill in others. Critical, opinionated, but lacking in allies. No one sees their point of view, and they don’t know why, because it’s more or less correct. (I had one person say “hey, Sandi Metz and Uncle Bob said it, so it’s true.”)

You (my team lead) are an idiot

Person just doesn’t respect you. They think you’re a joke. Maybe you were once peers and now you manage them; maybe you don’t know anything about Some Technology and therefore don’t deserve respect; maybe it’s personal animus. It doesn’t matter because whatever you do will be greeted with scorn bordering on amused disbelief.

Not really listening

The person for whom you have to say a thing over and over. And over. “Please don’t commit println debugging statements to master.” Later that day: a println debugging statement is in master. Are they not listening?

Incredibly difficult people

Not super-common, but incredibly difficult people (and their cousins, narcissists) are out there. Beware! They often appear to be high-functioning, accomplished people and very good at creating alliances. Good at reading people, particularly people’s vulnerabilities, and great at exploiting them. So what should you watch out for? Here are some behaviors:

  • Starts strong and committed, but something happens and their effectiveness fades
  • Builds fast friendships and alliances, but they tend to turn sour with matching speed
  • Prone to inappropriately extreme emotional reactions
  • Unable to accept responsibility; causes run from “it was like that when I got here” to “other people are making my life hell” to “I got sabotaged” to “my point of view is not supported”
  • Hoards work
  • Claims that without their efforts, you would be screwed

The bumbler

Never seems to get things right. Commits huge bugs to master; comes up with crazy ideas that are not needed; seems to be at the center of crises; doesn’t grok the main things team is trying to get done. Can be overly-diligent about something irrelevant because they are afraid to do something else.

When you explain that things need to improve, doesn’t get the hint that you are seriously saying things NEED to improve. Often ends up working on tasks so low-risk that their bumbling-ness is shunted off. Examples: build tools that are not helpful, performance analysis that no one reads.


21. September 2016


In 2008, Nick Paumgarten wrote a New Yorker piece about Art Garfunkel, describing Garfunkel’s habit of cataloging the thousand and twenty-three books he’d read since 1968. When I read it, I was pretty envious. To have read that much seemed impossibly awesome, a testament to hard work and discipline. When did he find the time to read all that stuff?

When I combat envy, my typical response is aspirational. Reading about Garfunkel, I started to catalog my reading, aspiring to some external measure of Garfunkel-ness. Which, now that I look back is weird. But envy is weird.

The Bible is full of injunctions against envy. It counts as one of the seven deadly sins, and is attributed to the reason why Cain killed Abel. And envy can perpetuate itself; Adam Philips, writing about Cinderella’s envious sisters, claimed that “making oneself enviable [is] the last refuge of the envious.”

My envious mind thinks “by becoming a better X, I’ll lose the envy I feel for the good Xs I know.” For me that X takes on different values: better husband, better programmer, better skateboarder, musician, father, cook, manager … the list of potential improvements is pretty unrealistic.

Envy is a failure, not of achievement, but of vision. The root of “envy” is the Latin invidere, “to look against, to look at in a hostile manner.” Envy blinds you. Your attention trains only on what you lack, and lack necessarily slots you as others’ inferior.

Of course, envy can also drive a search for for bigger things. Before I started programming, I didn’t understand code and wanted to learn more. I saw a stack of Java books on a programmer’s desk and asked where he bought them, and how he knew what he wanted to study. I envied his understanding of his craft and wanted to know more, and ended up buying my first programming book.

But envy usually blinds you. I had an extrovertive colleague, and his facility in talking to everybody really bugged me. He was someone who actually drew energy from talking; he did it with everybody. (The idea of talking to everybody makes me tired.) I wondered — and wrestled — with the fact that this person could so easily do what I couldn’t, and wondered why his career was working so well while mine languished.

A friend pointed out that I was actually quite friendly, and could expand my social world simply by trying each week to go to lunch with one co-worker I liked. I could stretch myself socially without becoming someone else. I tried it, and it really worked. But it was only when I opened my eyes to skills I already had that I made the first steps.

It may be that envy catches us in part because it makes us the victim and not the actor. Breaking free of envy forces you to do more than complain or rebel against circumstance, and sometimes it’s easier to complain. “The revolutionary wants to change the world”, Sarte wrote. “The rebel is careful to preserve the abuses from which he suffers so that he can go on rebelling against them.”