This repo contains two pipelines for collecting and processing podcast metrics data from Spotify, Apple, Podigee, and Anchor. It uses the following connector libs:

• Anchor: https://github.com/openpodcast/anchor-connector
• Apple: https://github.com/openpodcast/apple-connector
• Podigee: https://github.com/openpodcast/podigee-connector
• Spotify: https://github.com/openpodcast/spotify-connector

A pipeline consists of a Docker image which uses cron to run the connector periodically and forwards the fetched data to the Open Podcast API.

For local development, you can use the docker-compose.yml file to run the pipeline. To start the stack, consisting of the pipeline and a sample database, run:

make up

This will start the pipeline and a sample database. The pipeline will run every minute and fetch data from Spotify, Apple, and Anchor according to the podcastSources SQL table (see db_local_dev/schema.sql).

Alternatively, you can also start the database only. This is helpful if you want to run the pipeline in interactive mode

docker-compose up db

After that, you can start the manager container to run the pipeline:

# Interactive shell
docker-compose run --rm --entrypoint /bin/bash --env-file connector_manager/.env manager
# Run the pipeline in interactive mode
python -m manager --interactive

Have a look at https://github.com/openpodcast/stack to see a full stack which also consists of the pipeline containers to fetch data.

If you also want to collect metrics for your podcast, have a look at https://github.com/openpodcast/stack or reach out. We'll get you onboarded.

You can also help contribute or request new connectors.