Home | XOR Media
Backstory
There are times when you need to middle truncate a string. In many cases
it’s for UX/human purposes, though in some situations it’s the best way
to generate unique string for a length-limited field. This is the case I
ran in to recently in trying to automate submission of IAP to both
Google Play and the App Store which require short unique names for each
SKU.
Which of the following lists is sorted in the most “natural” fashion?
A:
Elementary Season 1 Episode 1
Elementary Season 1 Episode 10
Elementary Season 1 Episode 11
Elementary Season 1 Episode 12
Elementary Season 1 Episode 13
Elementary Season 1 Episode 2
Elementary Season 1 Episode 3
Elementary Season 1 Episode 4
Elementary Season 1 Episode 5
Elementary Season 1 Episode 6
Elementary Season 1 Episode 7
Elementary Season 1 Episode 8
Elementary Season 1 Episode 9
Async-What
The secret to building high performance sites which depend on external
web services is asynchronous HTTP. The trick to asynchronous HTTP (or
anything with the exception of UI) is to avoid callback
hell. Enter futures/promise
objects. When used
correctly they make doing substantial asynchronous IO, relatively
straightforward.
In the past couple days there’s been
several
articles
about how Google’s interview process has failed. The headlines are
overblown, inaccurate really, but to be expected giving the goal of
getting clicks. In reality Peter Norvig was talking about
counterintuitive
findings.
They were surprised to find that some of their best employees were not
unanimously voted in. That makes a lot of sense given my personal
experience with the Google interview process and my opinions of what
matters in making software engineering hires.
Background
There are two main sources of latency in the backend of web
applications: rendering (HTML templating or data serialization) and IO
(database or external service calls.) Today we’ll look at the latter and
more specifically focus getting rid of extraneous database round trips.
The fastest query possible is one that doesn’t happen.
It’s a fairly common use case to have a latitude and longitude and want
to find the closest objects to a given point. While there are
heavyweight solutions: MySQL Spatial
Extensions,
PostGIS, they can be more trouble than they’re
worth especially if you’re making use of an ORM.
I recently gave a presentation on
graphite and
statsd to the East Bay Django
meetup
It’s more about the importance of stats in general when running
services, web or otherwise. The
slides
are available and embedded here. There’s also a related demo
project that turned out to be
much more interesting than the slides themselves.
A recent Django security
release
added a new configuration option,
ALLOWED_HOSTS.
It’s optional in 1.3 and 1.4, but required in 1.5 setups. The relevant
detail is that when enabled in production Django will throw 500 errors
if the Host header doesn’t match one of the values in ALLOWED_HOSTS.
LinkedIn does a pretty good job of email communication. I probably read
50% of the emails they send me and I’ve remained subscribed to them for
quite a while now.
This is part two of a two-part post on the general categories of
software engineers. To pick things up at the beginning, check out the
first.
You can’t hire solely rock
stars and hiring
the wrong person is worse than leaving the position vacant, but what
makes someone right? In the first of a series of posts on building and
manging teams, I’d like to talk about the types of software engineering
positions. Who you should hire depends on your exact needs.
In what is probably my biggest WTF with Django to date, it doesn’t
validate your models before saving them to the database. All of the
necessary code exists and when a dev sets up her models she usually adds
the relevant validations using EmailField, URLField, blank, null,
unique, …, but unless you explicitly add code the constraints won’t be
enforced (adequately.) Some things will be caught with IntegrityErrors,
but not everything and not consistently.
A new year, a new theme. Since switching XOR Media from it’s hand-coded,
mostly placeholder, last May I’d been planning to build a custom theme
for it and over the holiday break here in the US I had a chance to
tackle it. I started by looking around for inspiration. I was looking
for something clean and simple with a horizontal top-nav, but that was
all I started with. From there I skimmed over the wordpress’s theme
directory and visited a lot of the sites that I read, but normally don’t
visit thanks to Google Reader.
If you’re in the startup world you’ll run across the startup curve,
rather frequently. The canonical
post on the
subject was done by Fred Wilson, but the original
concept came from a
talk
given by Paul Graham
About the only thing a rock star developer and their namesakes have in
common are they’re both exceedingly rare.
I’ve run across several articles recently that discount the value of
rock star developers and after reading them I’m left with the impression
that the authors wouldn’t know one if their life depended upon it. The
article “6 home truths about rock star
developers”
by Andrew Oliver begins by claiming that you can’t afford to hire all
senior developers. I thought this was about rock stars, what’s seniority
got to do with it. The fact that you’ve been doing something a long time
has absolutely nothing to do with being exceptional at it or else we’d
have a lot more 65 year old musicians debuting on the top 40 charts.
We hate ads. You and I do, but apparently some people don’t or else we
couldn’t make any money with their use. When I released Muni
Alerts
with ad support about 5 months after originally publishing the App I
decided to give users the option of turning off ads, as in a check-box
in the App’s settings no payment required. It’s worked out just fine and
I think more people should do it.
It’s been a busy few weeks for me at work, we
launched our
app,
were featured in the iPad app store home page, and have been called out
in other app store sections since. It’s been a wild ride, capped off by
a trip to Comic-Con where I spent 5-6 hours
a day on my feet giving my spiel about our app and company. Our booth
was pretty sweet, definitely the best in the area we were in, and it was
capped off with a really cool iPad video
wall
we built. The wall was a great conversation piece and drew a lot of
attention.
If you’ve spent some time with Django
chances are you’ve run across its
signals system.
It is most commonly used to provide hooks in to the model setup and CRUD
processes, but there are many other uses of the facility throughout the
code including auth, db, and the request life-cycle. What it doesn’t
offer out of the box is a signal that’s called during startup after your
models have been loaded.
As part of a project I’m currently working on we wanted to have a brief
splash video that plays the first time our app starts up for branding
purposes. The video also kills a little time as some initial processing
is happening.
I ran across this gist on Hacker
News which has a quick rundown of the
latencies of various operations. It’s nothing earth shattering, but if
you compare the intra-data center round-trip (0.5ms) to reading 1MB of
data from disk (20ms) you can pretty quickly see why systems like
memcached work so well even after adding the overhead of reading from
memory (0.25ms) and some time for the sending and receiving of the
request. Summed up the memcached response is often back in a few ms,
nearly an order of magnitude faster than reading from local disk.
Python decorators are
extremely useful when used with care, and using them is really
straightforward. Coding them up on the other hand can be complex and
requires,
reading
along with trial-and-error. There are some
helpful modules out there worth
taking a look at, but for this use-case we won’t be making use of them.
If you’re storing or transmitting time information in any way shape or
form (web-services, databases, queues, NoSQL, caches, log files, or just
plain txt files) the safest and thus best thing to do is to use UTC
(Coordinated Universal
Time)
everywhere.
What we’re after
jQuery’s
extend function is really useful
and if you’ve ever written a plug-in for the library chances are you’ve
made use of it. I’ve run across a use for this functionality in python
and it also makes an interesting interview question (regardless of
language.)
If you want a quick and easy way to extract a clip from a larger video
it doesn’t get much simplier than using
avconv (the new ffmpeg.) All you need
to know is the start time and duration of the section you’d like to
extract. Once you have those you can plug them in to the following
command.
If you’re looking to serve both HTTP and HTTPS out of EC2 from python,
using gunicorn and
ELB you’ll need to add a
bit of configuration to get gunicorn to
correctly detect which scheme the original request was made with and
thus allow Django (or whatever
framework you happen to be using) to correctly generate urls with the
matching scheme.
If you run in to a situation where you’d like to determine if a commit
has been pushed to a remote git branch there’s a
quick and easy command-line way to do so.
I’ve run in to a couple situations where I’ve wanted to iterate over a
list in pairs lately and decided to create a simple iterator to clean up
the code a bit.