Trouble brewing in OSINT paradise

In an excellent blogpost from Matthias Wilson a few months back, Matthias stated that 'The Golden Age of OSINT is over' and predicted the landscape will change in the coming years. For anyone who doesn't know what OSINT stands for: Open Source INTelligence. These last few weeks we have seen the start of these changes taking effect.

"It all started when Facebook quietly retired.."

It all started when Facebook quietly retired its Graph Search functionality around June 7th. Investigation agencies, journalists, NGO's and OSINT investigators cried out for help, because all of their investigative capabilities on Facebook suddenly came to a halt.

In a couple of days, Henk van Ess. together with D. Nemec, devised a tool and uploaded it to Github to bring back (sort of) the advanced searching methods. The tool can be found here. Although it's a good start, some nifty searching techniques you had in the past with Graph Search are gone, for now.

The day before that, Michael Bazzel from IntelTechniques announced that his (free) online OSINT tools were no longer available, due to the amount of abuse, DDOS attacks and even a cease-and-decist letter. Another toolbox which was not available.

"This last week.."

This last week, around June 12, the social-media-account-search-tool, Pipl.com, changed from free-to-use to a paid subscription model, because of all the searches that were done, which increased costs for Pipl, and again abuse of the search tools (according to Pipl.com).

And yesterday Twitter announced it will remove the geo-tag functionality from tweets, due to the lack of interest from users (according to Twitter). Is it really a lack of interest of users to put a geo-tag in their tweet? Or is it the fact that social media platforms are taking privacy more seriously (and dodging law suits in the process)? I will let you make up your own answer on that question.

"So what now?"

So what now? Well, tools, public info and/or functionality being removed isn't new if you're a couple of years into OSINT. Back in the day we used to have a lot of great search engines or could easily find WHOIS information. Because of the GDPR, WHOIS information is becoming more and more restricted, as well as adversaries using services like DomainByProxy to stay hidden. As always, the OSINT community adapted to these changes and found other means to gain information from public sources (within the boundaries of the law). But can we do so again in the coming years or will this trend of closing down search tools and boarding up information behind laws and payment walls mean the end of OSINT?

In my own opinion, for large organizations and corporations, paying to gain access to this information shouldn't be an issue because of the budgets that are available. Most likely they have already access to paid registers at this moment, such as the Chamber of Commerce, financial databases, and so on. Interesting enough, the data will probably also be available if you're into targeted advertising and marketing (since they're paying to receive the dataset they need to target the right audience).

"..it will be much harder.."

For NGO's, journalists and investigators with small budgets for investigation (and Dutch cheapskates like myself), it will be much harder to find the same or complementary information. Creativity is needed to obtain this information within the boundaries of the law. Is scripting, scraping, emulating or tooling the answer? It helps, but I still enjoy good old-fashioned searching, clicking, reading and analyzing as the best start. And picking up a phone sometimes! I agree with Matthias on this, that your communication skills will be useful to gain more information, rather than relying just on tools. Open sources aren't just sources that are found on the internet.

Luckily, there is a large online community who has the same passion for OSINT including yours truly. These coming months we (the OSINT community) will have our work cut out for us, constantly adapting and updating, whilst sharing (maybe not all) our knowledge (don't want to wake up those social media giants again) to help everyone getting their online investigations going. Roll up them sleeves girls and boys, the time of 'lazy' intel gathering is over.

Henri

This article is written and represents my own opinion on this matter. If you have an opinion on this matter, feel free to share it in a direct message or comment. Feel free to share this article on LinkedIn and Twitter as well. 'Commercial' comments and promotions will be removed.

Copyright pictures from Disney Star Wars and HBO Game of Thrones.