Sunday, January 10, 2016

Visualing High Dimensions as DNA Strands

For a community project, I needed to research which U.S. cities were most similar to mine. The U.S. census has some wonderful data that covers 1,579 statistical areas, using the Office of Management & Budget's definition.

With this data, I selected the relevant attributes and then calculated the root mean squared error of the scaled distances from the target city as a (dis)similarity metric. After sorting the results, I could identify the ten most similar cities.

But there was a catch... I was to share my findings in a presentation and raw statistics aren't always conducive to making engaging slide shows. Displaying two dimensions is straightforward, maybe even three or four are doable. My analysis had forty-plus dimensions. My approach was to think of each attribute as a point in a DNA strand. When you are finished, each little twist and turn represents another data point. Up or down ticks don’t matter, they just show the shape of the city in data-vision. FYI, all images were created using the ggplot2 package in R.

Start with a graph of each (scaled) point plotted.

Next, remove the grid lines.

Connect the dots.

Display all of the other cities' strands.

Subset to the ten cities most similar to yours.
In your presentation software, just set the animation or action to show each image in succession. I could tell that the audience was able to comprehend the message of how I discovered the ten most similar cities to ours. Thought I would share this little technique in case you face a similar challenge. By the way, if anyone has seen this concept before I would love to know how it has been used elsewhere.
Visualized as a layered animation

4 comments:

  1. this is a parallel coordinate plot, https://en.wikipedia.org/wiki/Parallel_coordinates

    ReplyDelete
    Replies
    1. Right. See also Alfred Inselberg's page (http://www.math.tau.ac.il/~aiisreal/).

      Delete
  2. Thanks for the links. I was hoping (and sure) that someone here would know.

    ReplyDelete
  3. I was going to say the same ... it is what Parallel Coordinate Plot is all about ... but never the less, good attempt

    ReplyDelete