Geolocation, pivoting, collaboration: three pillars for an OSINT beginner
Geolocation, pivoting, collaboration: these are the main ingredients of this write-up, which is based on a Webinar hosted by the GIJN on November 27–28, 2020, but that has become an interesting battleground for an OSINT beginner like me to dive deeper in some strategy & tactics.
Some notes I took at a webinar on OSINT
This is my second piece ever on OSINT, within a series I have pompously named “OSINT from scratch”. Previously, I have written a very basic beginners’ intro to some tools and techniques I have used (at the end of September 2020) to monitor the outbreak of the conflict in Nagorno-Karabakh.
The general purpose of these blogs is to share my first step in the open-source intelligence realm with other enthusiasts and wannabes.
Namely, this article has the aim to organise the notes I have taken and then compiled during a 2-day webinar hosted by the Global Investigative Journalism Network on open-source investigations (November 27–28, 2020). As I am learning from any source possible, I took the opportunity of this virtual event to dig deeper into some hot topics out there in OSINT, especially geolocation and visual investigations. Not geared with a tech background, I always find these events gold mines for anyone to sharpen some transversal skills.
Most importantly, these formative events and workshops are essential to today’s journalism. Investigative journalism is increasingly becoming, in fact, the watchdog of the powerful. It’s like the Dark Night — even better, it is a network of everyday heroes safeguarding not just an ominous metropolis, but the entire world, shedding lights on injustices and helping bring wrongdoers to accountability.
Okay, here’s the plan.
I will describe some techniques and share some tips learnt during the webinar in the following main section of the article. In the end, you can find more context on those beautiful journalistic organisations and their efforts to combat injustices, shed light over covered truths, and hold the powerful accountable. Alternatively, you can jump to the end, and then get back up here to dig into the technical stuff.
The context: ARIJ20
Between December 4 and 6, 2020, Arab Reporters for Investigative Journalism (ARIJ) has been hosting their 13th regional forum, totally virtual, ARIJ20. Numerous events and workshops have been taking place during November as the official opening date was approaching, with a focus on security tips for journalists, investigative techniques, and open-source. On November 27–28, 2020, with support offered by the Global Investigative Journalism Network (GIJN), ARIJ hosted Nick Waters from Bellingcat and Bashar Deeb from Lighthouse Reports to discuss open-source investigations techniques for journalism.
I guess this is enough for the presentations — let’s get to the actual content of the webinar!
*Disclaimer: These notes do not do justice to the depth and breadth of the explanations given by Nick and Bashar in their interventions. Please take all the words I used to compile my notes, and the related potential flaws in this write-up, as my own. All credits to the owners of the training material.
Furthermore, be aware that here I decided to focus on Nick Waters’ talks on geolocation and visual investigation, keeping my notes on Advanced Googling and Vehicle tracking by Bashar Deeb for another blog.
Open sources: “Speak, friend, and enter”
Open-source data is (almost) anything you can find “out there” without the need for specific access rights, information openly available ranging from aircraft registration numbers to satellite images on Google Earth.
Such information, which can be challenging to gather but is no secret, is vital to today’s investigations, according to Nick Waters. As human beings are naturally social animals and social media platforms provide them with an easy-to-access window over the world and their communities, people film a lot of stuff. They actually witness any kind of event, so that out there there are posts, photos, and videos almost reducing to zero information real-timeliness.
Additionally, low-paid administrative employees in corrupted offices are amazingly prone to sell data, dispersing interesting information online, and satellite imagery is becoming accessible to anyone.
Hence, it seems there is the full recipe to verify, geolocate, and generally investigate stuff that happens — or is happening! — at the other corner of the world.
Ethical concerns
Nick is particularly fervent in underlining the ethical issues that come with OSINT, especially referring to witnesses. Certain investigations, in fact, are only published if there is no danger for bystanders (and any people involved) to be exposed.
Gandalf the Wizard would warn you like this: “We must face the long dark of Moria. Be on your guard. There are older and fouler things than Orcs, in the deep places of the world.”
Limitations of open sources
Open sources — as any source used to infer a logical judgment, support a claim, and build an argument — come with a set of important limitations one must take into account during investigations and researches:
1) data availability & tunnel vision — all data are created equal, but some data are more equal than others, meaning that there is plenty of information on certain issues, almost nothing on others; personal bias also contribute to limit our global picture of the information landscape, hijacking the scientificity of our collection and analytical process;
2) taboo subjects not represented — there are grey zones, with topics surrounded by low-reliable sources, and dark zones, populated by taboo themes not represented in the “official” records, e.g. in a certain region of the world you cannot critique X or openly talk about Y;
3) establishing intent — why? This should be the overall yoke of your investigation — there is no strategy without intent and direction, therefore planning is essential (but plans are also there to be upset);
4) preservation of sources & overall ethical considerations — see Ethical Concerns above.
GEOLOCATION
Geolocation refers to the practice of finding the whereabouts of an event through a photograph, a video, or any other information that can be analysed to spot some key indicators matching with an existing location (where has photo X been taken?).
You have photograph X showing a tank on a road, some building in the background, a patch of vegetation. It has been posted by a guy claiming it has been taken during that event in Anbura, in NW Syria, and we want to verify this claim by precisely geolocating the image. A visual inspection will reveal useful details: a precise pattern in the road pavement, or a small cell tower on a hill, that we can match with details visible on Google Earth Pro. If certain recognisable features in photo X are reasonably matching with features on the map, there we go! You have a potential location.
‘Caesar’ photographs and Syrian atrocities
This is the way the “Caesar” photographs have been geolocated by Bellingcat and others within the OSINT community. Those photographs, displaying tortured corpses of anti-Assad dissidents, were smuggled by a forensic photographer of the Syrian regime in 2013. According to visual investigations, the place in which they were taken is Military Hospital 601, Damascus, at less than 1 km from Assad’s Palace.
*Check here the geolocation breakdown image posted on Twitter by @ArtWendeley and the related video by Bellingcat.
Among fundamental hints to locate the place, Nick has underlined the centrality of the communication towers on Mount Qasioun on the background; this feature proved to be essential to understand that who has taken the picture must have had a clear view of the Mount and its infrastructure.
The following step was to check for important indicators (or markers) to match exactly the photograph with satellite images of the area, among which two parallel buildings, certain patterns of vegetation, and also smaller venues in the surroundings.
Finding Bana
Another case study, another brilliant example of geolocation.
The facts: in 2016, Bana Alabed, a 7-year old girl from East Aleppo, became famous on Twitter for her posts and periscope videos reporting from war-torn Syria, however raising suspicion about the authenticity of her account. Nick Water’s investigation to verify her online presence and provide corroborating evidence to her reporting is a lesson on its own. Yet, what he explained with his intervention, is how he geolocated the two pictures below, again leveraging acute observation, key features in the background, and nothing more technical than Google Earth.
Panorama building, reverse image search and keyword search
When it comes to geolocate videos it is suggested to use an online tool to build a panorama out of the video frames. You can use open-source and free solutions like GIMP to edit images and stitch frames together, as Bellingcat does and explains in the article on the Memphis Belle Case. At that point, you can have a panorama picture and you can start diving into its properties to look for key indicators or markers, e.g. cell towers, railroads, buildings, and other compounds. When you isolate something characteristic, you can then try to reverse search it (more brilliant articles and guides have dug deeper into this topic, check them out!) with Google, Bing, Baidu, or Yandex. Each search engine has its own peculiarities, e.g. Yandex is more powerful to locate stuff in Eastern Europe and Russia. A trick when you search for people: try to blur the faces of other individuals in the picture (if you cannot totally crop them out), to have the browser focusing only on the person of interest. Researching by image does not always yield optimal results. An alternative would be to use keywords. For instance, if you have a picture with a road crossing a wood and some containers close to a seemingly industrial facility with a red wall, you could try to tailor your search (also using different search engines!) to target “industrial complex” AND “red wall”; of course, the more context you have, the better your search: if you know the pic has been taken in the Balkans, you can add the name of a precise country and proceed by trial and error. If in a certain northern region there is more vegetation of the type in the photograph, while a similar area develops along a shore, you can start narrowing down your search even more, excluding the more obvious negative options.
Wrapping up:
1) you can search by reverse image;
2) you can tailor your search with keywords;
3) you should use the power of several search engines;
4) if you have a video, build a panorama out of its frames;
5) “zoom in”: narrow down your query based on contextual knowledge, try to tailor the search by geographical area, use precise keywords based on key indicators in the picture;
6) consider asking somebody with more specialised knowledge than yours!;
6) for human faces, some tools come and go, with more or less powerful algorithms; currently, Nick suggests using PimEyes;
7) look for any other clue in the picture, also for letters, digits, and pattern of colours: you can find a code/number, a street name/graffiti, or an advertisement/logo and then track it down using other sources (see the section below!).
ISIS’s supporters
Nick has used a set of pictures taken with smartphones by alleged ISIS’s supporters in different locations. In each image, the subject who takes the shot — and who is not visible — holds a piece of paper with some writings in Arabic; sometimes, the name of a European city is written at the bottom.
Nicks explains that when trying to geolocate an image, again, the surroundings and the background are crucial: the more features of the overall scene are visible, the higher the chances to locate the place. For instance, in the first case (image 1 from the left), a green patch of vegetation occupies the whole background, hiding any identifiable feature. If we are not lucky enough to spot a rare plant which only grows in certain areas of the world, then we are left with nothing. Of course, the writing itself could be fundamental.
I cannot read Arabic; therefore, the easiest solution is to use a free OCR (optical character recognition) software online, which basically can detect letters from images and then allow for translations.
This is the text detected:
NOTHING.
Okay, either the software is not good enough to properly catch Arabic handwriting, or the quality of the image, once cropped, is too low. For this reason, I have also tried to upload the entire picture to the OCR, and I tested two other online solutions, Yandex OCR and i2OCR. Both weren’t successful.
Hypothetical next moves:
1) if I wanted to practice some Arabic, I would try to re-write those few lines by myself, with the help of an alphabetical table; then, I would copy-and-paste the sentences in Google translate (high likelihood of nonsense results, due to my contextual ignorance, the complexity of the Arabic alphabet itself, and even more the composition of its syntax);
2) the best solution would be to simply ask a friend who can read Arabic. Brilliant, right? (More on this easy and nice trick of life that is “ask someone” in the paragraph on collaboration below).
Image 2 from the left in our case study picture is again hard to locate without previous knowledge and familiarity with that precise location.
Finally, image 3 comes with more evidence, as the writings claim we are in Germany, Münster, and there is much more information coming from the background. The solution was easy, according to Nick: an advertisement poster is visible on a column; a map for all the advertisements in the city happens to exist, also allowing to filter between column posters and larger areas. By checking each location, with old-fashioned manual work, it was possible to geolocate the photograph.
Exercise
Among some live exercises, Nick asked the attendees to geolocate the following pictures (again from the series of ISIS’s supporters in Europe):
The first picture has been geolocated in Rue Championnet, Paris — how to: as a starting point, we are in Paris; the sign that is partially visible from on the left side of the picture, almost entirely covered by the piece of paper, is composed of some red and blue stuff, actually two small red corners. Even if I immediately thought of the Carrefour logo, it is instead a Suzuki one. By searching for Suzuki car dealerships on Google Maps, it was then possible to match the picture with the exact location.
The second picture has been geolocated near Bruce Grove station, North London by leveraging some clues like the typical London double-deck, red bus, and underground indicators, but most importantly by guessing the word written on the blue bridge in the background.
Nick’s tips
Through these case studies, Nick provided the audience with fundamental guidelines. In a nutshell:
1) check the background for recognisable key features, especially infrastructures which are most likely to be seen on Google Earth for comparison;
2) use Google Earth Pro to actually “walk” the places when attempting to match features for geolocation, with the support of other maps like Wikimapia, which is “[…] a multilingual open-content collaborative map, where anyone can create place tags and share their knowledge”;
3) remember to sketch your own map when studying the picture; for instance, you can understand there is a certain pattern of rectangular buildings, two of them divided by a road, and a patch of vegetation with a certain angle, and that the observer would have a specific point of view/angle.
PIVOTING
I use this term to refer to both Bashar and Nick’s insightful explanation of the investigative mindset required to do some real OSINT magic. You need a creative approach (you, everyone, especially if not competent with some more technical skills) to make tools work for you — not the other way around.
It is easy to get lost in the universe of platforms and ready-made solutions to find that place, scrap that code, and so on. The fact is that there is probably too much out there, and one — especially a beginner like me — risks to lose focus.
This means 2 things falling under the label ‘pivoting’:
1) there is no point in knowing 100+ tools if you don’t know how pieces of information can make sense together and become new leads for the next step;
2) a tool can be imagined to perform a certain task, but the sky’s the limit — therefore an application conceived to upload your favourite beers can easily turn into a database of people hanging around a certain place, eventually allowing for tracking the location history of military personnel. Or, also, a fitness app like Strava can pinpoint strange activities in the total nowhere (military bases, yikes!).
Pivoting with tools — LinkedIn profiles
Bashar has touched upon this when explaining some tips about social media profile reconnaissance. Although using sock puppet accounts — fake accounts — is generally the optimal strategy, the following technique remarks on the point of ‘pivoting’.
When you are investigating someone and want to check out their LinkedIn profile without leaving traces, you cannot directly open it from the LinkedIn website. Therefore, you can use a site called Mobile-Friendly Test, whose purpose is to give you a preview of a web page and verify that it is processed properly on mobile. If you copy and paste your target’s LinkedIn profile URL, you will get a precise rendering of their account on the left side as a screenshot. You can then select the option HTML, and copy that. Lastly, Bashar suggests using Code Beautify, a platform to ameliorate, validate, or simply visualise different coding languages. If you paste your target’s LinkedIn HTML code in the HTML viewer and run it, you finally get the entire target’s profile ready to be explored in detail.
Slang and Leaks
I have already reported Nick and Bashar remarks on the importance of contextual knowledge as a starting point to support the inquiry: recognising a certain architectural feature, in fact, can be essential to narrow down the search to a geographic area, writing on a banner held at a protest can help individuate other clues, and so on. According to Nick, knowing some specific slang can be helpful too. To explain the concept, Nick referred to the British military saying “spin dits”, which stands for “to tell stories”: niche or specialised expressions can help pivoting from one meaning to another, allowing for a deeper understanding of contextual knowledge, or are used metaphorically to intend something else. For instance, the term “fullz” refers to leaked information about individuals detailing financial data, addresses, car plates registrations, and more. When investigating individuals online, exploiting the “digital shadow” everyone leaves behind them, leaked databases also prove to be an interesting source of information.
Yet, I have to stop your enthusiasm (that’s how you’re feeling by reading this, aren’t you?) regarding this topic, because I have poor knowledge so far and this is out of my comfort zone. However, Nick’s intervention on the subject was seriously fascinating!
COLLABORATION
What really impressed me at the end of the day was the overall environment imbuing the OSINT community itself. An environment characterised by a very high degree of collaboration, mutual exchange, and trust. I believe the most beautiful and insightful investigative results are directly stemming from continuous feedback mechanisms fueled by individuals and organisations combining their curiosity and skills. From a hint on Twitter, armies of unaffiliated OSINT analysts, enthusiasts, and journalists join forces to gather pieces of information in the process to find the truth. This spontaneous system works so well that crowdsourcing open-source intelligence has become the backbone of successful initiatives under the slogan #OSINTforgood, like Trace Labs CTF Search Parties to help find missing persons, up to institutional actions such as Europol Trace An Object to fight child abuse or the FBI Seeking Information project. This, of course, is just the tip of the iceberg: several Reddit channels and Discord servers deal with cool stuff from geolocation to disinformation, creating common spaces for people to discuss techniques, issues, and solutions.
Nick and Bashar made crystal-clear, throughout their lessons, that there is a lost ‘art of asking’ which always comes at hand. When in doubt, there is certainly somebody with specialised knowledge about a precise geographical area, or with an odd passion for a model of aircraft or vessel, or paintings, or items it would take you quite a lot of time to verify.
If there is a moral of the story, you’ve got it: learn to ask.
Conclusion
I am reading through these words for the thousandth time and I understand this is quite an approximation compared to the richness of the webinar. Yet, I hope to have reported some useful advice, “standing on the shoulders of giants”:
1) verify your sources, dive deeper, compare and contrast, find indicators to match X and Y;
2) use tools, do not be used by tools;
3) learn to pivot: if something has been conceived to do X, you can always try to use it to accomplish Y — if unsuccessful, at least you will have learnt something new:
4) listen, ask, collaborate;
5) there are great courses and workshops out there, however, consider some free webinars, and use them to pivot to specialised write-ups, blogs, and channels, and dig deeper into each topic discussed, follow the clues of something that interests you, and take it as a challenge.
This is it for this second blog of OSINT from scratch — I hope you enjoyed, please feel free to drop any feedback and comment you should have!
Investigative journalism as a public good: the networking super-power
Investigative journalism is a public good. It is not an isolated phenomenon, it is not a solitary hunt, it is a flow of distributed agency uniting people across cultures to combat injustices. Here, a few lines on the three main organisations involved in the design of the ARIJ20 webinar.
The Global Investigative Journalism Network (GIJN)
Established in 2003 following the second Global Investigative Journalism Conference in Copenhagen, “[t]he Global Investigative Journalism Network is an international association of journalism organizations that support the training and sharing of information among investigative and data journalists — with special attention to those from repressive regimes and marginalized communities”. Accordingly, the GIJN provides training, support, and information about the latest tools and techniques for journalists with conferences and workshops while building an impressive repository of training material. At the crossroad between OSINT and investigative journalism, for instance, there is this awesome guide on How-Tos for Journalists detailing some strategies to geolocate images, verify facts, or track vehicles in two parts (Part 1 and Part 2).
Arab reporters for investigative journalism (ARIJ)
Aligning to the mission of providing a commonly shared support platform for journalists with special attention to certain regions, other networks have been flourishing in the last two decades. Among them, Arab reporters for investigative journalism (ARIJ) is an organisation which supports reporters across the Araba world. In their own words, it is “the first and leading media organisation in the MENA region, dedicated to promoting investigative journalism across the Arab world. Based in Amman, ARIJ was founded in 2005 to support independent, quality and professional journalism, by offering training, media coaching, mentoring, funding and networking opportunities with local and international media outlets.”
Bellingcat
Bellingcat is “[…] an independent international collective of researchers, investigators and citizen journalists using open source and social media investigation to probe a variety of subjects”. Leveraging open-source data of any kind, from satellite images and leaked databases, Bellingcat’s investigators collaborate in a distributed, highly-cooperative team-force to debunk misinformation and expose the truth in a post-truth world. They are real masters of the OSINT tradecraft. If you want a taste, you can read about their case studies or you can level up your investigative toolbox with their kit.
