Commits

Palmer, 2E0EOL  committed c75cb85 Merge

Merge stable

  • Participants
  • Parent commits 5385d4c, 855d15f
  • Branches r/0

Comments (0)

Files changed (7)

+UNRELEASED
+-----
+  * fb06a3eee37b: Fixes for NOOP and DEBUG flags, which were upper-case
+  * 8b28bfcb2112: Documentation fix; we meant max feeds in the config, not streams
+  * 9404651ac4e3: New modes for maxchildren: 0 (master mode, no children) & -1 unlimited children
+  * 7a88178e20e2: Document maxchildren
+  * dc981680b159: Ensure we retry the same feed when we hit the maxchildren limit
+  * c9a5d02bb867: PID list; Deleting from an array/list, leaves am undef entry, avoid issue by using a hash
+  * 9686b8125008: Ensure that the master does not exit before all children have been reaped.
+  * 7e8d6474ab82: Show what each process is downloading, MASTER versus feedname, via the proctitle
+  * a82253054c72: Don't download from the master, because we need to reap children.
+  * 97d5138f396a: All arguments in _main are uppercase, this applies to MAXCHILDREN
+  * 653e12a873c7: When we run out of maxchildren, the parent handles this serially
+  * b346aec7df0a: Do exit from Child's caller, not within Child
+  * 652b58eb0319: Fork the children on a per-stream basis
+  * 77b6d2a9ffc8: Rename children maxchildren in config.
+  * 12bd90c966da: Move per-stream handling into a Child() function, though it is not forked.
+  * a5402fcd6625: Add children default to sample configuration file
+  * d6b9b642c556: Added note about -n flag into the NEWS file
+  * a2a4809f8c68: Add bullet-points to NEWS file
+  * 547a20d97e71: Added some notes into the NEWS file
+  * fa5aea1b6809: Move christian.org.uk into $RELIGION
+  * fa2cd57f91fa: When -n is specified, do not fetch other feeds
+  * a82bba3a4299: Break up downloads into categories using paths
+  * 142fabce355e: Added christian.org.uk feed
+  * 643f61e053c3: Add -q quiet mode into TODO
+  * 54971b2a7cd1: Issue #2 - Fix regression in 762258d9b82c
+  * 762258d9b82c: Fix issue #2 (Cron email reports contain raw progress output)
+  * e5030ee815d3: rsscache is no longer a fork, arc fork takes over the functionality
+  * d9ac0f1ddf5b: Mark URL for rate-limiting TODO entry
+  * 0dfb04059f52: Add additional Alistair Cooke's Letter From America feeds
+  * 562c97dff8d3: Add more Reith Lectures from 1948-1976 and BBC Thought for the day
+  * 87638d83907e: Add BBC Witness 2010 archive
+  * 6a419a37c84b: Add BBC Reith lectures
+  * b8c422bec640: Added BBC 'Witness' 2011 archive
+  * 3b746b7f75e1: Add Alistair Cooke - Letter From America
+  * 8427d186eb25: Add BBC "The Bottom Line" with Evan Davis
+  * d7cdbf46d34d: Add BBC Heart & Soul feed
+  * bb4903d30cd2: Add BBC 'Click' podcast
+  * e83f9eea22d7: Added BBC 'fooc' (From Our Own Correspondent)
+  * 7e10459b8b7f: Reverse order of changelogs, most recent first, in keeping with the version order.
+  * bc560cca9be9: Ensure that TODO list references all of the existing forks.
+  * 915a4a7ab6c4: Note that Debian packaging became available in 0.2.0 (historical note)
+
 0.2.1
 -----
+  * 10da18ca5027: Update copyright year in Makefile
+  * d37242dfc095: Ensure that an empty tag name doesn't cause an infinite loop
+  * eb224348f241: Eliminate dead-code.
+  * 563f3f6df0bc: Wrong logic, reversed.
+  * 355875ed3025: Avoidance of non-resolving variables
+  * 0203d5270448: Ensure that all environment variables and config defined globals are upper-case
+  * 9f5d21d26c11: Re-written ProcessTags due to ordering problems, where tags reference other tags
+  * 363cace517b5: Bugs discovered in tag parser
+  * 425ed27adf17: Update copyright year to 2013
+  * b74cfe290b7a: Add the BBC "In Our Time" programme
+  * 277a08b5fecc: More BBC productions are a specific bbc/ subdirectory
+  * 2a32d1d8ee1b: Added BBC Beyond Belief
+  * 242b2f6a58d7: Mark script executable
+  * 14443bfa90ba: Standard TODO indentation (two spaces)
+  * afa4fbd5a207: Add email notifications to the TODO list
+  * 9138a6d311e2: Move from /opt/bin to /usr/bin and support non-admin installs
+  * 95c6f16dff33: Just a spacing between BSD banner and shbang
+  * 4abb474507fd: Add BSD license to dlpodget file
+  * d7d1334e44c2: Write a simple README
+  * 5fbcda2c1e80: Note down a couple more ideas in the TODO file
+  * 0cfb72a3cccb: Add the BBC 'HardTalk' programme
+  * 4fe6a9821a61: Add BBC World Service Witness to RSS feed list
+  * ef0df5b736e3: Rename the ReadStream() function ReadFeed()
+  * 5535724d2123: Better error when local directories are missing etc
+  * 9db63a16b9cf: Disable Joe Rogan until timeout issues can be addressed
+  * addb377ccd70: Add BBC Friday comedy
+  * 8634466662b4: Handle connection refused and similar errors reading the stream.
+  * 00fd65434454: Possibly a more 'standard' link for the Joe Rogan Experience
+  * b475b9050524: Add the Joe Rogan Experience podcast
   * 02d693494837: Add BBC Radio 4 "The Archers"
-  * b475b9050524: Add the Joe Rogan Experience podcast
-  * 00fd65434454: Possibly a more 'standard' link for the Joe Rogan Experience
-  * 8634466662b4: Handle connection refused and similar errors reading the stream.
-  * addb377ccd70: Add BBC Friday comedy
-  * 9db63a16b9cf: Disable Joe Rogan until timeout issues can be addressed
-  * 5535724d2123: Better error when local directories are missing etc
-  * ef0df5b736e3: Rename the ReadStream() function ReadFeed()
-  * 4fe6a9821a61: Add BBC World Service Witness to RSS feed list
-  * 0cfb72a3cccb: Add the BBC 'HardTalk' programme
-  * 5fbcda2c1e80: Note down a couple more ideas in the TODO file
-  * d7d1334e44c2: Write a simple README
-  * 4abb474507fd: Add BSD license to dlpodget file
-  * 95c6f16dff33: Just a spacing between BSD banner and shbang
-  * 9138a6d311e2: Move from /opt/bin to /usr/bin and support non-admin installs
-  * afa4fbd5a207: Add email notifications to the TODO list
-  * 14443bfa90ba: Standard TODO indentation (two spaces)
-  * 242b2f6a58d7: Mark script executable
-  * 2a32d1d8ee1b: Added BBC Beyond Belief
-  * 277a08b5fecc: More BBC productions are a specific bbc/ subdirectory
-  * b74cfe290b7a: Add the BBC "In Our Time" programme
-  * 425ed27adf17: Update copyright year to 2013
-  * 363cace517b5: Bugs discovered in tag parser
-  * 9f5d21d26c11: Re-written ProcessTags due to ordering problems, where tags reference other tags
-  * 0203d5270448: Ensure that all environment variables and config defined globals are upper-case
-  * 355875ed3025: Avoidance of non-resolving variables
-  * 563f3f6df0bc: Wrong logic, reversed.
-  * eb224348f241: Eliminate dead-code.
-  * d37242dfc095: Ensure that an empty tag name doesn't cause an infinite loop
-  * 10da18ca5027: Update copyright year in Makefile
 
 0.2.0
 -----
+  * DDRP: Debian packaging
+  * DDRP: Download progress and rate display
   * DDRP: Partial downloaded file recovery
-  * DDRP: Download progress and rate display
 
 0.1.0
 -----
+2012-01-26
+----------
+  * It's important to note that the subdirectories under ~/podcasts have changed,
+    there are now separate directories for religion, politics, comedy etc.  These
+    can be seen in a new [paths] section in the config file, and the keys of these
+    directories can ne referenced from within each feed definition.
+  * -n flag is now supported, to download one stream by the section name only.
+
 2012-12-10
 ----------
-New release which supports BSEC and partial downloads
+  * New release which supports BSEC and partial downloads
 
 2012-12-08
 ----------
-Initial release
+  * Initial release
-  * Cached RSS XML files to avoid checking every run
+  * Cached RSS XML files to avoid checking every run (https://bitbucket.org/daybologic/dlpodget.arc)
   * GUI
-  * Unit testing
-  * Process forking (download multiple streams at once)
-  * Download rate limiting (on a per-feed basis)
-  * Emnail notifications
+  * Unit testing (https://bitbucket.org/daybologic/dlpodget.ut)
+  * Process forking - download multiple streams at once (https://bitbucket.org/daybologic/dlpodget.forking)
+  * Download rate limiting - on a per-feed basis (https://bitbucket.org/daybologic/dlpodget.rate)
+  * Email notifications (https://bitbucket.org/daybologic/dlpodget.email)
+  * Transcoding support (https://bitbucket.org/daybologic/dlpodget.codecs)
+  * Back-end database tracking, logging and statistics (https://bitbucket.org/daybologic/dlpodget.db)
+  * Robust handling of default values for incomplete configs (https://bitbucket.org/daybologic/dlpodget.defaults)
+  * Non-RSS feeds (https://bitbucket.org/daybologic/dlpodget.nonrss)
+  * User-defined download order (https://bitbucket.org/daybologic/dlpodget.order)
+  * Full documentation (https://bitbucket.org/daybologic/dlpodget.pod)
+  * Graceful timeout handling (https://bitbucket.org/daybologic/dlpodget.timeouts)
+  * OOP re-write (https://bitbucket.org/daybologic/dlpodget.perloop)
+  * C++ re-write
+  * Archive support - frozen feeds (https://bitbucket.org/daybologic/dlpodget.arc)
+  * Quiet mode (-q) for use from crontab

File debian/changelog

 dlpodget (0.2.1) stable; urgency=low
 
+  * 10da18ca5027: Update copyright year in Makefile
+  * d37242dfc095: Ensure that an empty tag name doesn't cause an infinite loop
+  * eb224348f241: Eliminate dead-code.
+  * 563f3f6df0bc: Wrong logic, reversed.
+  * 355875ed3025: Avoidance of non-resolving variables
+  * 0203d5270448: Ensure that all environment variables and config defined globals are upper-case
+  * 9f5d21d26c11: Re-written ProcessTags due to ordering problems, where tags reference other tags
+  * 363cace517b5: Bugs discovered in tag parser
+  * 425ed27adf17: Update copyright year to 2013
+  * b74cfe290b7a: Add the BBC "In Our Time" programme
+  * 277a08b5fecc: More BBC productions are a specific bbc/ subdirectory
+  * 2a32d1d8ee1b: Added BBC Beyond Belief
+  * 242b2f6a58d7: Mark script executable
+  * 14443bfa90ba: Standard TODO indentation (two spaces)
+  * afa4fbd5a207: Add email notifications to the TODO list
+  * 9138a6d311e2: Move from /opt/bin to /usr/bin and support non-admin installs
+  * 95c6f16dff33: Just a spacing between BSD banner and shbang
+  * 4abb474507fd: Add BSD license to dlpodget file
+  * d7d1334e44c2: Write a simple README
+  * 5fbcda2c1e80: Note down a couple more ideas in the TODO file
+  * 0cfb72a3cccb: Add the BBC 'HardTalk' programme
+  * 4fe6a9821a61: Add BBC World Service Witness to RSS feed list
+  * ef0df5b736e3: Rename the ReadStream() function ReadFeed()
+  * 5535724d2123: Better error when local directories are missing etc
+  * 9db63a16b9cf: Disable Joe Rogan until timeout issues can be addressed
+  * addb377ccd70: Add BBC Friday comedy
+  * 8634466662b4: Handle connection refused and similar errors reading the stream.
+  * 00fd65434454: Possibly a more 'standard' link for the Joe Rogan Experience
+  * b475b9050524: Add the Joe Rogan Experience podcast
   * 02d693494837: Add BBC Radio 4 "The Archers"
-  * b475b9050524: Add the Joe Rogan Experience podcast
-  * 00fd65434454: Possibly a more 'standard' link for the Joe Rogan Experience
-  * 8634466662b4: Handle connection refused and similar errors reading the stream.
-  * addb377ccd70: Add BBC Friday comedy
-  * 9db63a16b9cf: Disable Joe Rogan until timeout issues can be addressed
-  * 5535724d2123: Better error when local directories are missing etc
-  * ef0df5b736e3: Rename the ReadStream() function ReadFeed()
-  * 4fe6a9821a61: Add BBC World Service Witness to RSS feed list
-  * 0cfb72a3cccb: Add the BBC 'HardTalk' programme
-  * 5fbcda2c1e80: Note down a couple more ideas in the TODO file
-  * d7d1334e44c2: Write a simple README
-  * 4abb474507fd: Add BSD license to dlpodget file
-  * 95c6f16dff33: Just a spacing between BSD banner and shbang
-  * 9138a6d311e2: Move from /opt/bin to /usr/bin and support non-admin installs
-  * afa4fbd5a207: Add email notifications to the TODO list
-  * 14443bfa90ba: Standard TODO indentation (two spaces)
-  * 242b2f6a58d7: Mark script executable
-  * 2a32d1d8ee1b: Added BBC Beyond Belief
-  * 277a08b5fecc: More BBC productions are a specific bbc/ subdirectory
-  * b74cfe290b7a: Add the BBC "In Our Time" programme
-  * 425ed27adf17: Update copyright year to 2013
-  * 363cace517b5: Bugs discovered in tag parser
-  * 9f5d21d26c11: Re-written ProcessTags due to ordering problems, where tags reference other tags
-  * 0203d5270448: Ensure that all environment variables and config defined globals are upper-case
-  * 355875ed3025: Avoidance of non-resolving variables
-  * 563f3f6df0bc: Wrong logic, reversed.
-  * eb224348f241: Eliminate dead-code.
-  * d37242dfc095: Ensure that an empty tag name doesn't cause an infinite loop
-  * 10da18ca5027: Update copyright year in Makefile
 
  -- David Duncan Ross Palmer <palmer@overchat.org>  Wed, 02 Jan 2013 22:26:38 +0000
 
 dlpodget (0.2.0) stable; urgency=low
 
+  * DDRP: Download progress and rate display
   * DDRP: Partial downloaded file recovery
-  * DDRP: Download progress and rate display
 
  -- David Duncan Ross Palmer <palmer@overchat.org>  Mon, 10 Dec 2012 22:18:20 +0000

File debian/control

  libwww-perl,
  libxml-feed-perl,
  dl-libconfreader-perl,
- liburi-perl
+ liburi-perl,
+ libio-interactive-perl
 Description: Podcast downloader and management system
  Daybo Logic's podcast system keeps track of new podcasts posted on an RSS
  feed and downloads whatever you do not yet have, checking file lengths
 use Daybo::ConfReader;
 use Getopt::Std;
 use URI::Escape;
+use IO::Interactive qw(is_interactive);
+use POSIX ":sys_wait_h";
 use strict;
 use warnings;
 use diagnostics;
 	return $uri_parts[-1];
 }
 
-sub ReadFeed($$$)
+sub ReadFeed($$)
 {
 	my @entries = ( );
-	my ( $Feeds, $Url, $Name ) = @_;
+	my ( $Feeds, $Feed ) = @_;
+	my ( $Url, $Name ) = ( $Feed->{rss}, $Feed->{name} );
 	my $feed = XML::Feed->parse(URI->new($Url));
 	if ( !$feed ) {
 		printf(STDERR "Stream %s error: %s\n", $Name, $@);
 
 sub DownloadStream($$$)
 {
-	my ( $ua, $f, %get_params );
+	my ( $ua, $f, %get_params, $is_interactive );
 	my $local_length = 0;
 	my ( $Feeds, $entry, $Name ) = @_;
 	my $local_podcast = $Feeds->{$Name}->{localpath} . '/' . $entry->{filename};
 			return;
 		}
 	}
-	$ua = LWP::UserAgent->new( show_progress => 1 );
-	unless ( $Feeds->{_main}->{noop} ) {
+	$is_interactive = is_interactive();
+	$ua = LWP::UserAgent->new( show_progress => $is_interactive );
+	printf(STDERR "Downloading %s ... ", $entry->{uri}) if ( !$is_interactive );
+
+	unless ( $Feeds->{_main}->{NOOP} ) {
 		if ( open($f, '>>:raw', $local_podcast) ) {
 			binmode($f);
 		} else {
 		%get_params,
 		':content_cb' => sub {
 			my ( $chunk ) = @_;
-			print $f $chunk unless ( $Feeds->{_main}->{noop} );
+			print $f $chunk unless ( $Feeds->{_main}->{NOOP} );
 		}
 	);
-	close($f) if ( !$Feeds->{_main}->{noop} );
+	close($f) if ( !$Feeds->{_main}->{NOOP} );
+	printf(STDERR "done.\n") if ( !$is_interactive );
 }
 
 sub ProcessTags($$)
 	return $V;
 }
 
+sub Child($$)
+{
+	my ( $Feeds, $Feed ) = @_;
+	my @entries = ReadFeed($Feeds, $Feed);
+	foreach my $entry ( @entries ) {
+		print STDERR 'Processing entry: ' . Dumper $entry if ( $Debug );
+		DownloadStream($Feeds, $entry, $Feed->{name});
+	}
+}
+
 sub main()
 {
+	my $proctitle = $0; # Save original process title
 	my $conf;
 	my @entries = ( );
 	my @defKeys;
 	my %feeds = ( );
 	my %opts = ( );
+	my %pids;
 	my $confSeen = 0;
 	my @confFiles = (
 		'dlpodget.rc',
 		"$ENV{HOME}/.dlpodget.rc"
 	);
 
-	return 1 if ( !getopts('d', \%opts) );
+	return 1 if ( !getopts('dn:', \%opts) );
 	$Debug = 1 if ( $opts{'d'} );
 	print(STDERR "Explicit debug mode enabled by -d\n") if ( $Debug );
 
 		}
 	}
 	if ( $conf ) {
+		my %reserveSec = map {  $_ => 1 } ( '_main', 'paths' ); # Reserved section names
 		my ( @sections, $secC );
 		$secC = $conf->GetSections(\@sections);
 
 				$feeds{_main}->{ uc($mk) } = $conf->GetDatum(undef, $mk);
 			}
 			# Set debug flag via config if it existed.
-			if ( $feeds{_main}->{debug} ) {
-				$Debug = $feeds{_main}->{debug};
+			if ( $feeds{_main}->{DEBUG} ) {
+				$Debug = $feeds{_main}->{DEBUG};
 				printf(STDERR "Set Debug via config: %s\n", $Debug);
 			}
 		}
 
+		if ( 'paths' ~~ @sections ) { # Has a paths section?
+			my @secKeys;
+			if ( $conf->GetKeys('paths', \@secKeys) ) {
+				foreach my $pk ( @secKeys ) {
+					$feeds{_main}->{ uc($pk) } = $conf->GetDatum('paths', $pk);
+				}
+				# For backward compatibility, set localpfx with [paths] root
+				$feeds{_main}->{'LOCALPFX'} = $feeds{_main}->{'ROOT'}
+					if ( $feeds{_main}->{'ROOT'} );
+			}
+		}
+
 		for ( my $secI = 0; $secI < $secC; $secI++ ) {
 			my @secKeys;
+			next if ( $opts{'n'} && $sections[$secI] ne $opts{'n'} );
 			my $keyC = $conf->GetKeys($sections[$secI], \@secKeys);
 			foreach ( my $keyI = 0; $keyI < $keyC; $keyI++ ) {
-				next if ( $sections[$secI] eq '_main' ); # Skip reserved section
+				next if ( $reserveSec{$sections[$secI]} ); # Skip reserved sections
 				my $v = $conf->GetDatum($sections[$secI], $secKeys[$keyI]);
 				$v = ProcessTags(\%feeds, $v);
 				printf(
 		}
 	}
 
+	$0 = "$proctitle [MASTER]" if ( $feeds{_main}->{MAXCHILDREN} != 0 );
 	foreach my $feedName ( keys(%feeds) ) {
+		my $reaped_pid;
+		my $child_limit_reached = 0;
 		my $feed = $feeds{$feedName};
 		next if ( !$feed->{enable} );
 		next if ( !$feed->{download} );
-		@entries = ReadFeed(\%feeds, $feed->{rss}, $feedName);
-		foreach my $entry ( @entries ) {
-			print STDERR 'Processing entry: ' . Dumper $entry if ( $Debug );
-			DownloadStream(\%feeds, $entry, $feedName);
+		$feed->{name} = $feedName;
+		if ( $feeds{_main}->{MAXCHILDREN} == 0 ) { # Master performs downloads
+			$0 = "$proctitle [$feedName]";
+			Child(\%feeds, $feed);
+			next;
+		} elsif ( $feeds{_main}->{MAXCHILDREN} < 0 || scalar(keys(%pids)) < $feeds{_main}->{MAXCHILDREN} ) {
+			my $pid = fork();
+			die "cannot fork: $!" if ( !defined($pid) );
+			if ( $pid == 0 ) { # Child process
+				$0 = "$proctitle [$feedName]";
+				Child(\%feeds, $feed);
+				exit(0); # Children should not return
+			} else {
+				$pids{$pid} = $feedName;
+			}
+		} else {
+			warn "Not enough children, waiting.";
+			$child_limit_reached = 1;
 		}
+
+		if ( $child_limit_reached ) {
+			$reaped_pid = wait();
+		} else {
+			$reaped_pid = waitpid(-1, WNOHANG); # Any children finished?
+			next if ( $reaped_pid <= 0 ); # No, carry on.
+		}
+
+		# A child returned, remove it from the PID list
+		die 'Internal error' if ( !exists($pids{$reaped_pid}) );
+		delete($pids{$reaped_pid});
+
+		redo if ( $child_limit_reached );
+	}
+	while ( scalar(keys(%pids)) ) {
+		my $pid = wait();
+		last if ( $pid == -1 );
+		delete($pids{$pid});
 	}
 
 	return 0;
 ;
 ; More generally, we have a default section, which contains the following options:
 ;
-;   enable    = <1|0>, default 1, set to 0 for a convenient way to ignore the entire file.
-;   localpfx  = <prefix which will be exported to sections as $LOCALPFX>
-;   noop      = <1|0>, default 0, set to 1 to disable modification of local files.
+;   enable      = <1|0>, default 1, set to 0 for a convenient way to ignore the entire file.
+;   localpfx    = <prefix which will be exported to sections as $LOCALPFX>
+;   noop        = <1|0>, default 0, set to 1 to disable modification of local files.
+;   maxchildren = <0-n>, default 0, max concurrent downloads. nb. setting this to 1
+;                 is less efficient than 0, since a dedicated child is cloned for each feed.
+;                 -1 means no limit (no more than the number of feeds in the configuration).
 
 enable    = 1
-localpfx  = $HOME/podcasts
 noop      = 0
 debug     = 0
+maxchildren  = 5
+
+[paths]
+root      = $HOME/podcasts
+bbc       = $LOCALPFX/bbc
+religion  = $LOCALPFX/religion
+politics  = $LOCALPFX/politics
+tech      = $LOCALPFX/tech
+comedy    = $LOCALPFX/comedy
 
 [infowars]
-localpath = $LOCALPFX/infowars
+localpath = $POLITICS/infowars
 rss       = http://xml.nfowars.net/Alex.rss
 check     = 1
 download  = 1
 enable    = 1
 
 [bsec]
-localpath = $LOCALPFX/bsec
+localpath = $RELIGION/bsec
 rss       = http://feeds.feedburner.com/BradleyStokeEvangelicalChurchSermons?format=xml
 check     = 1
 download  = 1
 enable    = 1
 
 [bbcaq]
-localpath = $LOCALPFX/bbc/aq
+localpath = $POLITICS/aq
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/bbcaq/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [bbctoday]
-localpath = $LOCALPFX/bbc/today
+localpath = $BBC/today
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/today/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [bsdtalk]
-localpath = $LOCALPFX/bsdtalk
+localpath = $TECH/bsdtalk
 rss       = http://feeds.feedburner.com/Bsdtalk
 check     = 1
 download  = 1
 enable    = 1
 
 [thearchers]
-localpath = $LOCALPFX/bbc/thearchers
+localpath = $BBC/thearchers
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/archers/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [joerogan]
-localpath = $LOCALPFX/joerogan
+localpath = $COMEDY/joerogan
 rss       = http://blog.joerogan.net/feed
 check     = 1
 download  = 1
 enable    = 0
 
 [fricomedy]
-localpath = $LOCALPFX/bbc/r4comedy
+localpath = $COMEDY/r4comedy
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/fricomedy/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [bbcwitness]
-localpath = $LOCALPFX/bbc/witness
+localpath = $BBC/witness
 rss       = http://downloads.bbc.co.uk/podcasts/worldservice/witness/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
+[bbcwitness_2011]
+localpath = $BBC/witness
+rss       = http://downloads.bbc.co.uk/podcasts/worldservice/w2011/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[bbcwitness_2010]
+localpath = $BBC/witness
+rss       = http://downloads.bbc.co.uk/podcasts/worldservice/w2010/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
 [bbchardtalk]
-localpath = $LOCALPFX/bbc/hardtalk
+localpath = $POLITICS/hardtalk
 rss       = http://downloads.bbc.co.uk/podcasts/worldservice/ht/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [beyondbelief]
-localpath = $LOCALPFX/bbc/beyondbelief
+localpath = $RELIGION/beyondbelief
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/belief/rss.xml
 check     = 1
 download  = 1
 enable    = 1
 
 [iot]
-localpath = $LOCALPFX/bbc/iot
+localpath = $BBC/iot
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/iot/rss.xml
 check     = 1
 download  = 1
 enable    = 1
+
+[fooc]
+localpath = $BBC/fooc
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/fooc/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[bbcclick]
+localpath = $TECH/click
+rss       = http://downloads.bbc.co.uk/podcasts/worldservice/digitalp/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[heartsoul]
+localpath = $RELIGION/heartsoul
+rss       = http://downloads.bbc.co.uk/podcasts/worldservice/heartsoul/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[bbctbl]
+localpath = $BBC/tbl
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/bottomline/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke]  ; Letter From America
+localpath = $BBC/acooke
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/acooke/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfabushjr]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfabushjr/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfaclintonpt2]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfaclintonpt2/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfaclinton]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfaclinton/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfabushsr]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfabushsr/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfaregan]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfaregan/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfanixoncarter]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfanixoncarter/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[acooke_lfaearlyyrs]
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/lfaearlyyrs/rss.xml
+localpath = $BBC/acooke
+check     = 1
+download  = 1
+enable    = 1
+
+[reith]
+localpath = $BBC/reith
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/reith/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[reitharc]
+localpath = $BBC/reith
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/rla76/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[reitharc2]
+localpath = $BBC/reith
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/rla48/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[bbctftd]
+localpath = $RELIGION/tftd
+rss       = http://downloads.bbc.co.uk/podcasts/radio4/thought/rss.xml
+check     = 1
+download  = 1
+enable    = 1
+
+[christian.org.uk]
+localpath = $RELIGION/christian.org.uk
+rss       = http://www.christian.org.uk/downloads/audio/seriespodcast.xml
+check     = 1
+download  = 1
+enable    = 1