Predictions, predictions, predictions

I’ve just listened to the latest episode of Alastair and Andrew‘s podcast, scene from above, and the discussion section based around near-future predictions for the Earth Observation (EO) industry, as well as some of the discussion in the news section, was extremely interesting. I’m fully onboard the hype train for machine learning booming in EO, with Andrew seemingly somewhat skeptical.

Before I go into why I think that’s the case, I’ll mention Alastair speaks about a Voyager documentary, the Farthest (I’ve actually just noticed a big Irish producer, crossing the line was involved in production, wahay!). It sounds absolutely incredible, and will go on my watch list, but Alastair’s comments reminded me of an xkcd comic alluding to the fact that the edge of the solar system is difficult to define! I actually really enjoyed listening to their thoughts on Voyager in general, and would love to hear more discussion around the history of EO as well as wider planetary missions – every time I read and think about Corona, for example, I can’t help but be amazed.

far

Voyager spacecraft (NASA)

 

One of the main predictions made within the main section of the podcast is that analysis ready data (ARD) will see wider use and release by data providers. We have seen a move towards sentinel 2 ARD and planet have recently released their atmospherically corrected surface reflectance product, I would hope this is an indication that this is quite well developed already!

planet.png

A figure from Planet’s surface reflectance white paper (source)

On the machine learning (ML) front, I attended a google earth engine workshop at the beginning of this year, and having had fruitful discussions with the host on the project’s directions, I think the iron is hot for ML and the hype justified. In particular, the host spoke about the team preparing tensor flow integration into the platform in time for AGU next year. Having been lucky enough to participate in (albeit not at a competitive level) the planet kaggle competition for classifying image excerpts into one or more classes last year, I have a decent idea of just why there has been a frenzy of research surrounding convolutional neural networks (CNNs) in the computer vision community, and I’m surprised that they haven’t appeared more in EO research.

While Andrew notes that supervised and unsupervised classification has been around and used for decades, the difference between those and deep-learned information is like night and day in my opinion. The competition, past the task presented, gave me a look into how neural networks are transforming image analysis, and how recurrent CNNs on massive scales could be leveraged in an environmental context for things like linking phenological mapping to data which might provide reasons as to why a change is happening with spatial context. Object-based analysis is unparalleled for applications like this, and CNNs are now so easy to use and much better at handling massive data sets than previous methods. Computer scientists are poised to integrate more and more with the EO community as higher resolution data becomes available, and so I feel like when high temporal and spatial resolution open data becomes available multi-disciplinary research will really kick off. Infact, I put together a starter ipython notebook for bird identification, showing just how easy it is using a pre-trained CNN for this application, albeit not with EO data.

birds.png

Example plot from ipython notebook

This leads to a prediction of my own – as more imaging scientists move into EO, Unmanned aerial vehicle (UAV) and satellite data will need to be better integrated. Currently, there are a raft of problems linking data collected from consumer level cameras onboard UAVs to satellite data, not least of which is radiometric normalization. The demand for higher resolution data from the deep learning end of the community will lead to new standards being introduced for how UAV data is collected and metadata stored (shameless plug). EO platforms will begin to integrate publicly collected UAV data and satellite researchers will begin to collaborate with computer scientists using nearer earth images. We will then see satellites being used as an early warning systems and UAV missions automatically launched off the back of satellite derived information in a range of new applications.

This isn’t a particularly insightful prediction, but one which continuously hasn’t really been addressed. I’m always surprised as to how infrequently satellite and UAV data are used in tandem, but I’m hoping this will change!

That’s all for now, look for my Google Earth Engine blog coming next week, I was blown away by the product and definitely need to do a separate post on it 🙂

Advertisements

Scene from above

I’ve been severely neglecting my blog on account of focusing on writing up my PhD project as well as being sick (don’t underestimate the pain of getting your tonsils out as an adult!).

I wanted to write up a decent post for my 100th entry, but have subsequently realised it’s lead to me posting nothing for the last couple of months! I have a plan for a good entry coming up, though will need to find the time to put it together.

In the meantime, I picked up that Alistair Graham (geoger), who gave a talk at the conference I ran this year, and Andrew Cutts, who I have never met, though I remember worked through the straightforward openCV GUI demo from his website which I thought was great, have started a podcast, scene from above.

Science communication is tricky at the best of times, so I’m excited they’re giving this style of delivery a crack. The demo episode discusses Sentinel 5p and the larger scope of the sentinel project, remap’s webapp and cloud computing more generally, and the launch of a Moroccan satellite.

I think the discussion of the webapp was my favorite part. I appreciated Alistair’s humility in admitting that maybe he was approaching interaction with data from a point of view that was somewhat outdated, as he seems (as am I!) skeptical of the benefits of a sleek interface. Admittedly the app isn’t designed with me or others in the RS community in mind, but I can’t see it being used much in it’s current iteration.

Thinking of my ornithologist friends currently in PhDs/postdocs who would be the target audience for an app like this, they would almost definitely look at it for an hour or two with interest, and never think to use it again. Having consistently tried to get them interested in RS and accurate mapping, the tools need to be unbelievably simple to get people to consider using them seeing as so much of other scientists time is dedicated to learning specialist knowledge and general computing skills. It’s one of the many challenges of interdisciplinary work in science!

I’m looking forward to the next episode of the podcast, and hope a forum opens up for discussion online as I think I’d have something to contribute, and would love to hear other people’s opinions on these ideas!

Keep an eye out for a longer update soon 🙂

RSPSoc Annual Conference

I had a great time at the RSPSoc conference yesterday, and very much enjoyed catching up with the some of the people I made friends with at Wavelength this year – this is a short entry to just make available the slides of both Mike (supervisor) and myself, who’s primary focus was on image quality in photogrammetric work. Unfortunately I think I filled my slides a little too much and probably could have put in about half the content, but somehow couldn’t stop adding plots from the beautiful seaborn library, lesson learned!

Link to Mike slides

Link to my slides

Looking forward to writing a blog on RAW – JPEG conversions very soon, check the undemosaiced sneak preview below 😉

imtest.png

 

Joypy

Not one to miss a fad in data visualisation, I noticed joyplots getting a lot of attention over at reddit’s dataisbeautiful subreddit and have given a go at producing some myself – I’m hoping to integrate them into a talk I’m giving this Wednesday as part of the RSPSoc‘s annual conference, and am hoping they make enough sense to include.

I’m tinkering with the joypy library, a set of scripts whose sole purpose is to produce these types of plots, built ontop of the excellent (and frequently used by myself) seaborn plotting library.

For now, I need to get of the fad wagon and keep on writing!

1_Overstrand_Quality_100.0.png

A sample joyplot I’ve produced.

Sentinel_bot – now with NIR vision

A quick blog post as I’m very much in the throes of writing! I took a few minutes today to introduce false colour (Near Infrared – Red – Green) images into @sentinel_bot’s programming, so now there’s a 20% chance that an image it produces will be false colour. In the near future I think I’ll introduce other band combinations (such as PCA band combos for mineral contrast enhancement), but for now I’m going to let it sit and appreciate some of what it comes up with, such as the image below.

Source : https://github.com/JamesOConnor/Sentinel_bot

Twitter : www.twitter.com/sentinel_bot

dgs6xtyxyaadcvj

NIR – R – G image over Argentina

Gamify it

I’ve been planning and chinking away at writing up the last three years of work into a coherent thesis in the last 6 months or so. It’s very interesting to look back at the reams of planning documents, literature reviews and interim results documents I’ve produced over this time!

Knowing what and how much to write on each topic is a bit of a dark art however; the initial targets I’ve set are very loose, but I think important to form some sort of structure to grow the report into. As a bit of a tongue-in-cheek joke I produced some ‘progress bar’ style bar charts, one for each chapter planned for the final report and have been updating day on day. The satisfaction gained from seeing them creep up has actually been surprisingly effective in getting me into a writing mode each day!

I’ve gone with a traffic light colour palette, the top bar indicates how many words I planned to write, the second the word count to date and the bottom the upper limit I’ve set myself. I know obsessing over word count is a massive waste of time, and I don’t worry about them too much at all, but couldn’t pass up an opportunity for some opportunistic data visualization!

Progress.png

Standard summary report I’ve been producing

Neural nets in Remote Sensing

Neural nets, a summary: (The chain rule * your GPU RAM)

Around 2 years ago I remember having a discussion with Jan Boehm about photogrammetry after my first meeting as the shadow wavelength rep on the Remote Sensing and Photogrammetry committee. He mentioned Agisoft, which I was already using and familiar with at the time, but then mentioned the movement in dense matching algorithms towards use of neural nets, mentioning one which had been submitted to the KITTI stereo benchmark.

right_cnn

Disparity map using Žbontar’s methods

This piqued my curiosity, and I remember reading and being quite excited by Jure’s paper. While some concepts were new to me, the use of Convolutional neural networks (ConvNets) and the two types of architecture used to initialize the initial results, before moving towards post-processing using semi-global matching. I remember sinking a great deal of time into reading about the methods, exploring the github and methods used within the core of the paper, and subsequently hounding a colleague who was using a Titan-X for some deep learning work for some time with it.

I remember I took the ideas with me to EGU 2016, and even went to the point of acquiring a data set I thought would be worthy of testing it with from a German photogrammetrist, Andreas Kaiser. Alas, it wasn’t to be due to the hardware limitations and the fact that I wasn’t very familiar with the lua programming language. However I had learned a lot about the nature of deep learning, which I felt was a decent investment of my time.

The reason for this blog entry, however, isn’t to enlighten the reader of my failure to get up to speed with neural nets at the time, it’s much more hopeful than that! Fast forward two years, and development within the field of deep learning has come on leaps and bounds. With serious development time going into TensorFlow, and a beautiful and accessible front end in the form of keras, the python user really does have the tools to apply neural nets to all sorts of applications within image-based studies.

Having learned the basic ideas around neural nets from my initial excitement a long time ago I decided to try and get involved with the community once more. A few months back, a well timed kaggle competition came up which involved image classification, which raised an eyebrow. I contacted an old friend of mine who had just finished his PhD in medical imaging and we set to take up the challenge.

river

The task for the competition involved labeling satellite imagery

Since starting the task, I feel like I’ve come on leaps and bounds with not only the concepts behind ConvNets, but their architecture and application in the python framework. Whilst we generated lots of code (will be on github in due course), and had lots of ideas floating about, we finished a decidedly average mid-table – this first pass was as much an experience in learning about organisation as well as about imaging science, but it’s made me rethink about using ConvNets in a Remote Sensing/Photogrammetry environment.

Whilst we are seeing more contributions coming out of the community, and the popularity of other less technical concepts like support vector machines have shown I’m hoping to extend my skill set to include all of these in the future. If anyone who happens to be reading this feel the same, don’t hesitate to get in touch!