PS3 MDDN 242 2017

Scrape Montage

I am using the google bigquery database to get an collection of images with extreme sports in them, accessed from: https://bigquery.cloud.google.com/savedquery/763178943946:4448992055c64169b79c5aebef0b7446

I was interested in the first iteration of smartgrid not only separating out different sports, but also how people were posed. I ended up using images of people water, then snow skiing. These images were quite separated in the initial smartgrid, so it would be interesting to see how these parameters changed the output.

I tried running it with inception but it did not keep separated the different sport categories as well as the original did. It did have an interesting separation of what was happening in the background, we ended up with, in general, sky at the top, land and dark on the right and sea in the bottom left.

There are multiple images that I found interesting that are in the dataset. Both an image of a kite shaped like a human (looks like a skydiver) and of a kid on a surfboard infront of a painted background are included. I decided not to remove these images as I found it useful to understand the smartgrid algorithm if we see how these edge case photos get placed...

My final choice is to use resnet50. The final result has very nice separation of concepts as well as very nice boundaries. Some nice things I notices, the separation between snow and water sports is very smooth as a result of my left/right inputs. You can see how the center is very chaotic, with mistures of kyaking, ice climbing and surfing all near each other. I think that this result holds true the posing of people very well, with all of the close ups of the skydivers in the bottom right right beside the similar photos of underwater divers and the skateboarders with similar tricks put together. I especially like the mix of colors/visual similarity along with conceptual similarity this one has resulted in, with all of the far away paraglyders together on the bottom right and getting more zoomed in as you go left. Its also very interesting, with me keeping the outliars, how we have examples of images which are visually quite similar but are put in the right place, such as the what I assume are inverted images of a paddle boarder and a paratchuter, and you can still see that image of the kid with the fake background sitting along with all the other surfers.

NOTE

Google's image dataset pulls some incorrect images, so you'll have to delete all the images under 10KB and remove the one image that it stops on, as referred in the console.