Joypy

Not one to miss a fad in data visualisation, I noticed joyplots getting a lot of attention over at reddit’s dataisbeautiful subreddit and have given a go at producing some myself – I’m hoping to integrate them into a talk I’m giving this Wednesday as part of the RSPSoc‘s annual conference, and am hoping they make enough sense to include.

I’m tinkering with the joypy library, a set of scripts whose sole purpose is to produce these types of plots, built ontop of the excellent (and frequently used by myself) seaborn plotting library.

For now, I need to get of the fad wagon and keep on writing!

1_Overstrand_Quality_100.0.png

A sample joyplot I’ve produced.

Advertisements

Gamify it

I’ve been planning and chinking away at writing up the last three years of work into a coherent thesis in the last 6 months or so. It’s very interesting to look back at the reams of planning documents, literature reviews and interim results documents I’ve produced over this time!

Knowing what and how much to write on each topic is a bit of a dark art however; the initial targets I’ve set are very loose, but I think important to form some sort of structure to grow the report into. As a bit of a tongue-in-cheek joke I produced some ‘progress bar’ style bar charts, one for each chapter planned for the final report and have been updating day on day. The satisfaction gained from seeing them creep up has actually been surprisingly effective in getting me into a writing mode each day!

I’ve gone with a traffic light colour palette, the top bar indicates how many words I planned to write, the second the word count to date and the bottom the upper limit I’ve set myself. I know obsessing over word count is a massive waste of time, and I don’t worry about them too much at all, but couldn’t pass up an opportunity for some opportunistic data visualization!

Progress.png

Standard summary report I’ve been producing

A slippy map for Sentinel bot

Over the weekend I decided to expand on what was in sentinel bot‘s portfolio by having an automatically updating slippy map, which plots where the point for which sentinel bot has found an image is in the world, as long with the basic metadata of date acquired and lat/lon. I was trying to the leaflet’s marker-clusterer to work but to no avail, couldn’t quite get the knack of it! If anyone has experience with it I’d love to hear from you. I continued with just the pins nonetheless!

One really cool github project I used was this, which allows you to cycle through basemap providers for leaflet and provides you with the javascript code for inserting into your own map. I chose Heidelberg university’s surfer roads for no reason in particular, but may change this in the future. I think I’ll be returning to that github for future slippy maps!

In any case, the product is not perfect, but gives an interesting view of what the bots activities have been for the week it’s been active. I’m not trying to reinvent Earth Explorer, so will probably spend no more time on this, but it was an enjoyable pursuit!

Check the map here.

Data visualisation

Haven’t posted in the last while, so thought I’d make a quick post about some of my favorite data visualizations I’ve come across lately. The more I read about these the more it makes me want to improve the own graphics I produce, so if you’re looking for inspiration look no further! In no particular order:

Markov Chains

Basic as the visuals are, it really gives a good feel for what finite state problems look like. Can modify with your own code too!

Markov Chains

Baye’s rule/Conditional probability

From the same blog. Bayesian stats can be a bit daunting. Let this visualization of balls dropping through a filter calm you down as you need. Interactive to boot!

Conditional Probabilty

Fourier analysis

Just beautiful graphics putting simply what so many hours of reading couldn’t. Probably my favorite in the list due to the depth it covers!

Fourier analysis

Pathfinding

Not something I’m overly familiar with but have bookmarked because of how nice the graphics are to look at. Search is such a basic concept which is such a necessity to modern computing, I love the simplicity with which it’s presented.

Pathfinding

Blend4web curiosity app

Some might call it gimmicky, but I think the ability to be able to scroll through the cameras while the robot moves is just such a cool feature.

Curiosity

Potree

I can’t believe this is freeware. It’s amongst the best tools on the internet for point cloud viewing and the design is brilliant!

Potree

Seaborn

From the DIY category – seaborn is a front end plotting library for making graphs in python. It produces some beautifully crafted graphics! I love the joint plots.

Seaborn joint plot

Bokeh

Actually a pretty standard library it seems, I can’t believe how long it took me to find. I’m preparing some interactive graphics for upcoming conferences and bokeh makes it so simple to do! I particularly like the Lorenz example!

Bokeh Lorenz

Stamen mapping skins

Some very attractive base layers for using in your mapping needs. I think I’ll have to give making a base layer a go at some stage, but for now I can appreciate the possibilities…

Stamen

100,000 stars

Last on our list, one from the astronomers. An in browser interactive environment for exploring our stellar neighborhood!

100,000 stars

 

 

 

Leaflet maps, beautiful!

I decided to try and make some sort of map to help visualize the goings on of the movement of refugees through Europe, but got distracted in the technical aspects of putting together a web map and appreciating the effort that some open source developers have gone through to make really beautiful tiles for openlayers-based maps.

As such, I’ve put together a chloropleth map showing where the 130,000 extra refugees Europe is being asked to accept would go based purely on GDP. In essence, it’s a map of EU countries GDPs but may give a bit of context as to the practicality of certain countries offering asylum. It’s got some basic javascript components (Roll over for info, with highlighting), I’ve disabled the zoom as it wasn’t very relevant considering how sparsely the dataset is populated.

You can see it in my webspace here, a printscreen is attached! Here‘s a second similar one, showing number of asylum applications per country as of December 2014.

refug

50 posts!

For my 50th post I thought I’d just present another couple of geovisualisations based off of the UK national LiDAR inventory. The first is London, and is relatively complete. The point spacing at the most zoomed level is 7m as this was the best I could get without crashing my computer! I still think it’s quite interesting to see, and I hope to add some more functionality to it as time goes on. It’s viewable here.

Secondly I wanted to present a more topographically diverse region, so I searched for the area around Snowdonia, which had no data listed. Next, I searched around Ben Nevis, which had no data either! I then searched around Scafell Pike where there was a tile with data present, though you’ll see that it’s patchy, but somewhat interesting nonetheless! See it here.

A national 3D point cloud

I was searching various forks of Potree‘s web based point cloud viewer on Github, and happen to have stumbled across a fork with a python bindings for processing huge datasets. The example given is 640billion points, and has a live search feature that is pretty damn cool. The customisable color bar is also something I’m pretty excited about, as these ideas can be used for more than just height modelling – I’m looking forward to seeing some thematic models with this in mind, with perhaps an active legend which changes with whatever level of zoom (octree) you’re on. The Netherlands, however, is pretty flat, so I may try and adapt these ideas to the UK inventory over the weekend if I have time – either way I suggest you give the webviewer a look as it’s a technical marvel if nothing else!

I’ll include 1 screen cap showing the detail at the lowest level, you can make out individual houses and trees, really impressed by it all!

From the giant dataset at the lowest level, a building set amongst trees

From the giant dataset at the lowest level, a building set amongst trees

http://ahn2.pointclouds.nl/