Django greyscales

Access the application here.

I’ve been learning lots about the django web framework recently as I was hoping to take some of the ideas developed in my PhD and make them into public applications that people can apply to their research. One example of something which could be easily distributed as a web application is the code which serves to generate greyscale image blocks from RGB colour images, a theme touched on in my poster at EGU 2016.

Moving from a suggested improvement (as per the poster) using a complicated non-linear transformation to actually applying it to the general SfM workflow is no mean feat. For this contribution I’ve decided to utilise django along with the methods I use (all written in python, the base language of the framework) to make a minimum working example on a public web server (heroku) which takes an RGB image as a user input and returns the same image with a number of greyscaling algorithms (many discussed in Verhoeven, 2015) as an output. These processed files could then be redownloaded and used in a bundle adjustment to test differences of each greyscale image set. While not set up to do bulk processing, the functionality can easily be extended.

web_out

Landing page of the application, not a lot to look at I’ll admit 😉

To make things more intelligible, I’ve uploaded the application to github so people can see it’s inner workings, and potentially clean up any mistakes which might be present within the code. Many of the base methods were collated by Verhoeven in a Matlab script, which I spent some time translating to the equivalent python code. These methods are seen in the support script im_proc.py.

Many of these aim to maximize the objective information within one channel, and are quite similar in design so it can be quite a difficult game of spot the difference. Also, the scale can often get inverted, which shouldn’t really matter to photogrammetric algorithms processes, but does give an interesting effect. Lastly, the second PC gives some really interesting results, and I’ve spent lots of time poring over them. I’ve certainly learned a lot about PCA over the course of the last few years.

web_out.png

Sample result set from the application

You can access the web version here. All photos are resized so they’re <1,000 pixels in the longest dimension, though this can easily be modified, and the results are served up in a grid as per the screengrab. Photos are deleted after upload. There’s pretty much no styling applied, but it’s functional at least! If it crashes I blame the server.

The result is a cheap and cheerful web application which will hopefully introduce people to the visual differences present within greyscaling algorithms if they are investigating image pre-processing. I’ll be looking to make more simple web applications to support current research I’m working on in the near future, as I think public engagement is a key feature which has been lacking from my PhD thus far.

I’ll include a few more examples below for the curious.

 

This slideshow requires JavaScript.

Sentinel bot source

I’ve been sick the last few days, which hasn’t helped in staying focused so I decided to do a few menial tasks, such as cleaning up my references, and some a little bit more involved but not really that demanding, such as adding documentation to the twitter bot I wrote.

While it’s still a bit messy, I think it’s due time I started putting up some code online, particularly because I love doing it so much. When you code for yourself, however, you don’t have to face the wrath of the computer scientists telling you what you’re doing wrong! It’s actually similar in feeling to editing writing, the more you do it the better you get.

As such, I’ve been using Pycharm lately which has forced me to start using PEP8 styling and I have to say it’s been a blessing. There are so many more reasons than I ever thought for using a very high level IDE and I’ll never go back to hacky notepad++ scripts, love it as I may.

In any case, I hope to have some time someday to add functionality – for example have people tweet coordinates + a date @sentinel_bot and have it respond with a decent image close to the request. This kind of very basic engagement for people who mightn’t be bothered going to Earth Explorer or are dissatisfied with Google Earth’s mosaicing or lack of coverage over a certain time period.

The Sentinel missions offer a great deal of opportunity for scientists in the future, and I’ll be trying my best to think of more ways to engage the community as a result.

Find the source code here, please be gentle, it was for fun 🙂

dainlptxkaajaaw

WhatsApp Images

One thing I’ve noticed since sharing images across a range of formats/websites, is that image compression algorithms on various platforms vary noticeably. This is most evident, from my experience, with WhatsApp, where images tend to be resized without even an anti-aliasing filter. The results are images with huge amounts of speckle in them when they are not resized before uploading.

Obviously the target market for WhatsApp and its user base isn’t people using high end cameras to share their images on the application, but it still seems like a couple of functions could fix a lot of the visual problems that I see, which would save me having to do it locally.

It seems astounding to me that such a big company wouldn’t put more time into sensible image compression/resizing, or perhaps they have and I am catching exceptions. The blocky artifacts I’ve written about being associated with the algorithm on this blog before are evident. Even with the third example included, where the image was resized to 20% of it’s sized before compression applied produces a much better result qualitatively, even with the smaller pixel count upon redownload of the latter.

Whilst whatever algorithm they are using is likely directed towards smartphone camera users it still seems like an oversight by the developers. Hopefully WordPress doesn’t apply a similar type of compression when I post this now!

MP map

Just a quick entry detailing an interactive map showing MPs’ constituencies and party membership created at the request of a friend. It uses leaflet.js and geojson to draw the map, meaning it’s standalone html code which can be easily moved and modified.

mp_map.png

It’s based largely on the chloropleth example included in the leaflet documentation and was pretty interesting to make!

You can see it at my website here.

Leafiness

I thought it might be fun to try something different, and delve back into the world of satellite remote sensing (outside of Sentinel_bot, which isn’t a scientific tool). It’s been a while since I’ve tried anything like this, and my skills have definitely degraded somewhat, but I decided to fire up GrassGIS and give it a go with some publicly available data.

I set myself a simple task of trying to guess how ‘leafy’ streets are within an urban for urban environment from Landsat images. Part of the rationale was that whilst we could count trees using object detectors, this requires high resolution images. While I might do a blog on this at a later date, it was outside the scope of what I wanted to achieve here which is at a very coarse scale. I will be using a high resolution aerial image for ground truthing!

For the data, I found an urban area on USGS Earth Explorer with both high resolution orthoimagery and a reasonably cloud free image which were within 10 days of one another in acquisition. This turned out to be reasonably difficult to find, with the aerial imagery being the main limiting factor, but I found a suitable area in Cleveland, Ohio.

The aerial imagery is a 30 cm resolution having been acquired using a Williams ZI Digital Mapping Camera, and was orthorectified prior to download. For the satellite data, a Landsat 5 Thematic Mapper raster was acquired covering the area of interest, with a resolution of 30 m in the bands we are interested in.

This experiment sought to use the much researched NDVI, a simple index used for recovering an estimate of vegetation presence and health.

Initially, I loaded both datasets into QGIS to get an idea of the resolution differences

jezzer.png

Aerial image overlain on Landsat 5 TM data (green channel)

So a decent start, looks like our data is valid in some capacity and should be an interesting mini-experiment to run! The ground truth data is resolute enough to let us know how the NDVI is doing, and will be used farther downstream.

 

Onto GrassGIS, which I’ve always known has great features for processing satellite imagery, though I’ve never used. It’s also largely built on python, which is my coding language of choice, so I feel very comfortable troubleshooting the many errors fired at me!

The bands were loaded, DN -> reflectance conversion done (automatically, using GrassGIS routines) and a subsequent NDVI raster derived.

ndvi2.png

Aerial image overlain on NDVI values. Lighter pixels denote a higher presence of vegetation

Cool! We’ve got our NDVI band, and can ground truth it against the aerial photo as planned.

ndvi1

Lighter values were seen around areas containing vegetation

Last on the list is grabbing a vector file with street data for the area of interest so we can limit the analysis to just pixels beside or on streets. I downloaded the data from here and did a quick clip to the area of interest.

roads1.png

Vector road network (in yellow) for our aerial image. Some new roads appear to have been built.

I then generated a buffer from the road network vector file, and generated a raster mask from this so only data within 20 m of a road would be included in analyses. The result is a first stab at our leafy streets index!

map1.jpg

Visual inspection suggests it’s working reasonably well when compared with the reference aerial image, a few cropped examples are shown below.

This slideshow requires JavaScript.

Lastly, we can use this this data to scale things up, and make a map of the wider area in Cleveland. This would be simple to do for anywhere with decent road data.

map3.jpgThis might be useful for sending people on the scenic route, particularly in unfamiliar locations. Another idea might be to use it in a property search, or see if there’s a correlation with real estate prices. Right now I’ve run out of time for this post, but might return to the theme at a later date!

 

Image compression

Probably my final post on image gradients, I thought I’d include one last mini-experiment on the effect of image compression on image gradient histograms, a la my previous posts NRIQA! No Reference Image Quality Assessment and Blur detection. Using the same script, I generated three image histograms of the same image, though with different RAW -> JPEG/TIF conversions, before analysis in 8 bits using the script.

JPEG ‘quality’ settings are to with the level of compression in the files vs. the originals. While a full review of what JPEG compression is is beyond the scope of this post, at a high level this blog post is very good at presenting the compression artifacts associated with JPEG images.

The RAW file in this case is a .NEF taken from a Nikon D700.

The three images generated were:

  1. Default RAW -> TIF conversion using imagemagick (built on dcraw). This is converted to 8 bits using OpenCV within the script. [Size = 70 Mb]
  2. Default RAW -> JPEG conversion using imagemagick. The default ‘quality’ parameter is 92. The image is visually indistinguishable from the much larger TIF. [Size =3 Mb]
  3. RAW -> JPEG conversion using a ‘quality’ setting of 25. The image is visually degraded and blocky. [Size = 600 Kb]
_dsc8652_tif_tog

TIF image

_dsc8652_jpg_tog

Default JPEG (‘quality’ = 92)

_dsc8652_jpg25_tog

JPEG with ‘quality’ of 25

In a general sense, the script tells us that there is more high frequency (abrupt changes in pixel value) in the Y direction within this image. The comparison between the TIF and default JPEG values shows almost no difference. Within JPEG compression using quality values greater than 90, there is no chroma downsampling, so the differences between the TIF and JPEG images are likely not due to RGB -> gray differences.

The JPEG at quality 25 shows clear signs of quantization – the blocky artifacts are visibily smoothing the image gradients. This pushes the neighbouring pixel changes towards the center of the histogram range, evidence of the fact.

It’s interesting that no signs of degradation are visible within the first two images and it’s actually quite difficult to see where the differences are. For one last test, I subtracted one from the other and did a contrast stretch to see where the differences are occuring. The subtleties of the JPEG compression are revealed – at a pixel level the differences range from -16 to +15 in DN – the larger differences seem reserved for grassy areas.

diff.png

Difference image between default TIF and JPEG images (TIF – JPEG)

Will these subtle changes affect how computer vision algorithms treat these images? Or how will it affect image matching? Can we envision a scenario where these would matter (if we were calculating absolute units such as Radiance, for example)?

Questions which need addressing, in this author’s opinion!

Blur detection

I thought I’d supplement the recent blog post I did on No-Reference Image Quality Assessment with the script I used for generating the gradient histograms included with the sample images.

I imagine this would be useful as a start for generating a blur detection algorithm, but for the purposes of this blog post I’ll just direct you to the script on github here. The script takes one argument, the image name (example: ‘python Image_gradients.py 1.jpg’).  Sample input-output is below.

fusion_mertens

Input image

Image_gradients.png

Plot generated