About this use case
A research student or analyst working on their own. They need to have information about news content to support their research and help them reach conclusions. They want to manipulate the data a little and combine it with data from other sources in a quick way.
A Python script that calls the NewsWhip API and outputs the results to a CSV file, which can then be imported into spreadsheets or databases.
- Competent Python skills
- Familiarity with using GET and POST APIs
- Familiarity with using the command line
- Familiarity with importing data to spreadsheets
Time to implement
1 person, 1 day.
A number of university students and researchers need to back up their hypotheses using research driven by an understanding of the media landscape. NewsWhip's comprehensive database of content and social trends is a great source of information.
In this scenario, one of our customers was using the NewsWhip API for a single research project over a number of months. There was no requirement for the solution to scale beyond her own use, and most resiliency concerns were also not a priority. The most important aspects for her were:
- To be able to get the web content and social stats that she needed quickly, in a format that she could use.
- To be able to tweak her query as her work progressed, to refine the data that she would export.
She settled on the following setup.
The main advantages for using this solution for this use case are speed of implementation and configurability.
This solution could also be repurposed to run the same query/queries at regular intervals, gathering data on specific topics over time. In this instance, using the same Python script as an automated server-side job would be little effort. Instead of exporting to CSV, a better solution would be to add the data to a database instead.
Research: Google Sheets
We're currently prototyping an out-of-the-box integration with Google Sheets that would allow you to achieve these goals without building a script of your own. Check out research to find out more.
- Our API responses are formatted as JSON. The content isn't flat, so be sure to consider the layers of the output when you write the parsing logic of your script.
- When there is no data available for a given field, it is common for that field to be omitted entirely from the response. Be sure to account for missing fields in your script.
- When you save data, record NewsWhip as the source. This will make tracing data easier in future, particularly if you are gathering data from multiple sources.
Updated about a year ago