📊 1-click analytical dashboard for OSINT researchers
Analytical tool to extract insights (shown on a simple dashboard) from social media posts about narratives, sentiments, initiators, influencers and clusters of accounts. It should be applicable for studying disinformation campaigns, analysing public opinion, and assessing risks related to some topics.
It's a project created by team Watch Cats during participation in Bellingcat's First In-person Hackathon.
Inspired by 4CAT and twitter explorer. The development process is documented in this Google document.
Available by the link: https://bellingcat-hackathon-watchcats-uearyc7iggn84xznppgq5k.streamlit.app/
Twitter posts on various topics (1-20K), including datasets enriched with topics and sentiments.
Instructions:
- How to prepare dataset with Zeeschuimer
- How to prepare CSV dataset
- Check utils/cluster_n_sentiments.ipynb for instructions on how to enrich datasets with sentiments and topics
How can I get topics and sentiments for my dataset? Cause it’s a resource- and time-consuming operation, we implemented it in the Jupyter Notebook script available on our GitHub. For tweets vectorization we are using hkunlp/instructor-large model, for clusterization – MiniBatchKMeans, for the detection of topics – GPT-4-Turbo API, for the sentiment analysis of tweets – cardiffnlp/twitter-roberta-base-sentiment-latest mode. All steps are reproducible.
For local installation you need Python and pip installed.
pip install -r requirements.txt
streamlit run dashboard.pyFor private cloud installation, you need:
- Login (register) to GitHub
- Fork this repository
- Login (register) in Streamlit by GitHub account
- Create a new project in Streamlit from a forked repository
- Deploy (no payment method required!)
- Voila!
utils folder contains:
- CSV tweet datasets formatter (to Twitwi)
cluster_n_sentiments.ipynb: ML stuff (enrichment of datasets with sentiments and topics)
This tool uses the following OSINT techniques:
An example of a hashtag network built with Twitter Explorer using one of the datasets
