SMAT-CLI

Provides command line tools for getting data from the Social Media Analysis Toolkit (SMAT) as well as a library for interacting with SMAT from your own code.

The Social Media Analysis Toolkit is a resource that allows activists, journalists, researchers, and other social good organizations to collect information about hate, mis/disinformation, and extremism from a variety of online platforms.

SMAT-CLI is a tool that makes getting that information from the API easy, either from your terminal or as part of your own application.

Installation

OS X & Linux:

pip install smat-cli

Though, I recommend using Pipx to install it asa a system tool.

pipx install smat-cli

Windows:

Coming soon!

Usage Examples

Let's say you want to collect 1000 posts from Telegram posted between Jan 6 to March 1, 2021. You can do that with the content command like this.

smat content -s telegram -l 1000 --since 2021-01-06 --until 2021-03-01 trump

If you want some aggregated data, you can use the timeseries command to fetch a count of posts mentioning Trump from Jan 6 to March 1, 2021 and aggregate those into daily buckets, you can use the following.

smat timeseries -s telegram -i day --since 2021-01-06 --until 2021-03-01 trump

You can also aggregate by any arbitrary key present in the data for the site. To get an idea of which keys are available, you can examine the results of a content command. Once you know the key you want to aggregate on, you can use activity to, for example, count the number of posts containing the term Trump in each Telegram channel from Jan 6 to March 1, 2022.

smat activity -s telegram -a channelusername --since 2021-01-06 --until 2021-03-01 trump

In addition, this package can be used in another application by importing Smat from smat_cli and using it to query the API from inside your program.

from smat_cli import Smat
api = Smat()
data = api.content(term="trump", site="telegram", ...)
for d in data:
    print(d["message"])

Development Setup

This project uses Poetry and the code is formatted with Black.

Tests can be run via Tox, which will run the tests in Python verions: 3.7, 3.8, 3.9, 3.10. For this to work properly, all of these Python versions must be installed. A Dockerfile has been included in the test_runner directory if necessary.

poetry install
tox

Release History

• 0.1.0
  • Initial release

Me

Daniel Hosterman – @dhosterman – daniel@danielhosterman.com

Distributed under the Unlicense license. See LICENSE for more information.

https://gitlab.com/dhosterman

Contributing

1. Fork it (https://github.com/yourname/yourproject/fork)
2. Create your feature branch (git checkout -b feature/fooBar)
3. Format your code with Black
4. Ensure there are tests for your changes and that they pass
5. Commit your changes (git commit -am 'Add some fooBar')
6. Push to the branch (git push origin feature/fooBar)
7. Create a new Pull Request