Student work has exposed another advantage in artificial Intelligence – it’s extremely effective in locating the location where photos were taken.
The project, dubbed Predicting Images Geolocations (or PIGEON, for short) was created by three Stanford graduate students to pinpoint locations in Google Street View.
However, when presented with some personal photos that it had not seen before it was generally instances, able to make precise guesses as to where and when the photos were taken.
Like many applications of AI this power will likely to be a dual-edged sword. It can help people determine the location of old photos from family members, or permit scientists in the field to carry out quick inspections of entire regions for plant species that are invasive for just some of the likely useful applications.
However, it could also be used to disclose data about people they did not intend to disclose, according Jay Stanley, a senior policy analyst with the American Civil Liberties Union who is a specialist in technology. Stanley is concerned the same technology that is likely to become widespread can be used for surveillance by the government, corporate tracking or even for stalking.
“From the privacy standpoint the location of your home can be a highly sensitive piece of information,” he says.
AI is at its destination
It all started with a course in Stanford: Computer Science 330 Deep Multi-task, Deep Multi-task as well as Meta Learning.
Three of their friends, Michal Skreta, Silas Alberti and Lukas Haas, wanted to work on the help of a project and had a common interest:
“During the time, we were huge players of the Swedish game known as GeoGuessr,” says Skreta.
GeoGuessr is an interactive game that asks players to find photographs. It’s a simple set-up, and Skreta states: “You enter the game where you’re placed anywhere on Google Street View, and you’re required to put a marker on the map. This is your best guess as to the exact location.”
The game features more than 50 million players that compete at world championships, says Silas Alberti, another member of the group. “It includes Youtubers Twitch streamers and professional players.”
Students wanted to test how they could develop an AI-powered player that could be more efficient than humans. They began with an existing method of analyzing images dubbed CLIP. It’s a neural-network program which can understand visual images simply by reading the text they contain It’s developed by OpenAI the same company that also makes ChatGPT.
The Stanford students practiced on their own version using images taken from Google Street View.
“We made our own data comprised of approximately 500,000 street-view photographs,” Alberti says. “That’s really not a lot of information, and we were able to obtain impressive performance.”
The team also added more components of the software, such as one that helps the AI sort images by their location on the globe. Once it was completed it was believed that the PIGEON system was able to determine the exact location of an Google Street view image anywhere on the planet. It can guess the country 90% of the time. It will generally pinpoint an area within 25 miles from the actual location.
Then, they matched their algorithm against a person. In particular, a truly great human being named Trevor Rainbolt. Rainbolt is known as a legend when it comes to geospatial circles. He recently located an image of an unidentified tree in Illinois to test his luck however, he was unable to meet his adversaries with PIGEON. In the head-to-head contest, he was defeated in multiple rounds.
“We weren’t the only AI to play Rainbolt,” Alberti says. “We’re only the first AI to win in the face of Rainbolt.”
Pay attention to the little things
PIGEON excels due to its ability to detect all the tiny clues that humans have, as well as more subtle ones, such as subtle differences in the soil, vegetation, or weather.
The group claims that the technology is able to be used in a variety of applications. It could help identify power lines or roads which need to be fixed, to monitor biodiversity, or even be used to teach children.
Skreta is of the opinion that everyday people could find it beneficial: “You like this destination in Italy Where else in the world would you go to visit something like it?”
To check PIGEON’s performance I provided it with five of my own photographs from a trip I made across America several years ago. None of these were published online. Certain photos were shot in urban areas, however several were taken in locations that aren’t close to roads, or other recognized landmarks.
This didn’t seem to be a problem at all.
It predicted a campsite in Yellowstone to be within 35 miles of the actual site. The program also positioned another image that was taken on one of the streets located in San Francisco, to within just a few city blocks.
The photos are not always an easy match. For instance, the program erroneously linked one image that was taken in the western side in Wyoming to a place in the western side of Colorado over 100 miles from each other. The program also guessed that a photo taken of that of the Snake River Canyon in Idaho was actually Kawarau Gorge in New Zealand. Kawarau Gorge in New Zealand (in fairness, both landscapes are remarkably alike).
ACLU’s Jay Stanley thinks despite these mistakes the program clearly demonstrates the potential for power of AI.
“The reality that this was conducted as a student-led project is a reason to wonder what else could be accomplished through, for instance, Google,” he declares.
In reality, Google already has a feature called “location estimation” that makes use of AI to predict a photo’s place of origin. Currently, it only uses a catalog of roughly a million landmarks, rather than thecan turn off it..
Stanley believes that companies could in the near future utilize AI to monitor the places you’ve visited or that governments could look through your photos to determine whether you’ve been to a country in a list of countries to watch. Abuse and stalking are apparent threats, he claims. In the past Stanley claims that users could remove GPS tracking from pictures they publish online. This may no longer work.
It’s clear that the Stanford master’s students at Stanford are aware of the dangers. They’ve published an article about their method and co-authored with their professor Chelsea Finn — but they’ve resisted making their entire model publically available due to the concerns they have, they claim.
However, Stanley believes that the use of AI for geolocation will grow much more efficient in the coming years. Stanley doesn’t think there’s much to be done — other than being aware what’s inside the background photos you share on the internet.
9(MDEwMjQ0ODM1MDEzNDk4MTEzNjU3NTRhYg004))
