On today’s Internet 2.0 there are all sorts of data feeds available for consumption. From APIs to RSS feeds, it seems like nearly every site has a machine-readable output. There are many reasons why you’d want to collect this information, which I won’t go in to, so in this post I’m going to walk you through an application which consumes RSS feeds. I’ll be using the Python scripting language, and I’ll show you an evolution of the ways to go about the task:
Application introduction
Our application is going to work like this:
- A database contains the list of RSS feeds. This is long – 1000+ records
- Our application reads this list of feeds and processes them
- The items from the feeds are stored in the database
Database manipulation and RSS feed parsing are outside the scope of this tutorial, so we’ll start off by defining some empty functions that handle all this:
1ea83b35479472fe42409338e73d8e41000
We’re going to have all these in a module called “functions”, which can just be a file called functions.py in the same directory ( < python3.0)
Continue Reading
..