This collection of scripts and data files was used in the PIDIMEHS project at the Netherlands eScience Center. PIDIlib is mainly provided to showcase the possibilities of combining Elasticsearch, Pandas and Matplotlib in Python in the study of history of politics and media. PIDIlib's main contents are:
- Ways of querying the PIDIMEHS KB newspaper Elasticsearch instance
- Using Pandas to analyze the queried aggregations
- Visualization using Matplotlib
- Data used for 'indicators of pillarization' in the PIDIMEHS project
- Jupyter notebooks that were used to produce the results presented in the PIDIMEHS technical paper.
The KB newspaper data can be freely obtained from Delpher. It is not allowed to redistribute this data, so we cannot give access to our own instance. To use PIDIlib, one will have to set up their own Elasticsearch instance. Alternatively, one may contact the KB or SURFsara to request access to the SURFsara instance (which we used as well).
For further information and results see: