Commits

Palmer, 2E0EOL committed 7c94595 Draft Merge

Freshen branch from master

Comments (0)

Files changed (8)

 006a5b5eacd2959bda5c1a4b4e67b107bbcfbbc0 0.1.0
 596941eccf4d904e5701b21712cba11fdaa7c09e 0.2.0
 9068388bd93836dd779f5c7fb73472135de68639 0.2.1
+b0c1fd7cea3455f9a2da5802d2af839aca15b6c5 0.3.0
+[???]
+_main configurarion section must not have the explicit name 'main'
+0.3.0
+-----
+  * fb06a3eee37b: Fixes for NOOP and DEBUG flags, which were upper-case
+  * 8b28bfcb2112: Documentation fix; we meant max feeds in the config, not streams
+  * 9404651ac4e3: New modes for maxchildren: 0 (master mode, no children) & -1 unlimited children
+  * 7a88178e20e2: Document maxchildren
+  * dc981680b159: Ensure we retry the same feed when we hit the maxchildren limit
+  * c9a5d02bb867: PID list; Deleting from an array/list, leaves am undef entry, avoid issue by using a hash
+  * 9686b8125008: Ensure that the master does not exit before all children have been reaped.
+  * 7e8d6474ab82: Show what each process is downloading, MASTER versus feedname, via the proctitle
+  * a82253054c72: Don't download from the master, because we need to reap children.
+  * 97d5138f396a: All arguments in _main are uppercase, this applies to MAXCHILDREN
+  * 653e12a873c7: When we run out of maxchildren, the parent handles this serially
+  * b346aec7df0a: Do exit from Child's caller, not within Child
+  * 652b58eb0319: Fork the children on a per-stream basis
+  * 77b6d2a9ffc8: Rename children maxchildren in config.
+  * 12bd90c966da: Move per-stream handling into a Child() function, though it is not forked.
+  * a5402fcd6625: Add children default to sample configuration file
+  * d6b9b642c556: Added note about -n flag into the NEWS file
+  * a2a4809f8c68: Add bullet-points to NEWS file
+  * 547a20d97e71: Added some notes into the NEWS file
+  * fa5aea1b6809: Move christian.org.uk into $RELIGION
+  * fa2cd57f91fa: When -n is specified, do not fetch other feeds
+  * a82bba3a4299: Break up downloads into categories using paths
+  * 142fabce355e: Added christian.org.uk feed
+  * 643f61e053c3: Add -q quiet mode into TODO
+  * 54971b2a7cd1: Issue #2 - Fix regression in 762258d9b82c
+  * 762258d9b82c: Fix issue #2 (Cron email reports contain raw progress output)
+  * e5030ee815d3: rsscache is no longer a fork, arc fork takes over the functionality
+  * d9ac0f1ddf5b: Mark URL for rate-limiting TODO entry
+  * 0dfb04059f52: Add additional Alistair Cooke's Letter From America feeds
+  * 562c97dff8d3: Add more Reith Lectures from 1948-1976 and BBC Thought for the day
+  * 87638d83907e: Add BBC Witness 2010 archive
+  * 6a419a37c84b: Add BBC Reith lectures
+  * b8c422bec640: Added BBC 'Witness' 2011 archive
+  * 3b746b7f75e1: Add Alistair Cooke - Letter From America
+  * 8427d186eb25: Add BBC "The Bottom Line" with Evan Davis
+  * d7cdbf46d34d: Add BBC Heart & Soul feed
+  * bb4903d30cd2: Add BBC 'Click' podcast
+  * e83f9eea22d7: Added BBC 'fooc' (From Our Own Correspondent)
+  * 7e10459b8b7f: Reverse order of changelogs, most recent first, in keeping with the version order.
+  * bc560cca9be9: Ensure that TODO list references all of the existing forks.
+  * 915a4a7ab6c4: Note that Debian packaging became available in 0.2.0 (historical note)
+
 0.2.1
 -----
   * 10da18ca5027: Update copyright year in Makefile
+2013-02-21
+----------
+  * We have made the move from Daybo::ConfReader to Config::IniFiles, this requires minor
+    config changes, you must now put all _main parameters explicitly under [main],
+    this is because libconfig-inifiles-perl is a common Debian package, where as Daybo
+    Logic's internal config library isn't, as yet.
+
+2013-01-26
+----------
+  * It's important to note that the subdirectories under ~/podcasts have changed,
+    there are now separate directories for religion, politics, comedy etc.  These
+    can be seen in a new [paths] section in the config file, and the keys of these
+    directories can ne referenced from within each feed definition.
+  * -n flag is now supported, to download one stream by the section name only.
+
 2012-12-10
 ----------
-New release which supports BSEC and partial downloads
+  * New release which supports BSEC and partial downloads
 
 2012-12-08
 ----------
-Initial release
+  * Initial release
+dlpodget (0.3.0) stable; urgency=low
+
+  * fb06a3eee37b: Fixes for NOOP and DEBUG flags, which were upper-case
+  * 8b28bfcb2112: Documentation fix; we meant max feeds in the config, not streams
+  * 9404651ac4e3: New modes for maxchildren: 0 (master mode, no children) & -1 unlimited children
+  * 7a88178e20e2: Document maxchildren
+  * dc981680b159: Ensure we retry the same feed when we hit the maxchildren limit
+  * c9a5d02bb867: PID list; Deleting from an array/list, leaves am undef entry, avoid issue by using a hash
+  * 9686b8125008: Ensure that the master does not exit before all children have been reaped.
+  * 7e8d6474ab82: Show what each process is downloading, MASTER versus feedname, via the proctitle
+  * a82253054c72: Don't download from the master, because we need to reap children.
+  * 97d5138f396a: All arguments in _main are uppercase, this applies to MAXCHILDREN
+  * 653e12a873c7: When we run out of maxchildren, the parent handles this serially
+  * b346aec7df0a: Do exit from Child's caller, not within Child
+  * 652b58eb0319: Fork the children on a per-stream basis
+  * 77b6d2a9ffc8: Rename children maxchildren in config.
+  * 12bd90c966da: Move per-stream handling into a Child() function, though it is not forked.
+  * a5402fcd6625: Add children default to sample configuration file
+  * d6b9b642c556: Added note about -n flag into the NEWS file
+  * a2a4809f8c68: Add bullet-points to NEWS file
+  * 547a20d97e71: Added some notes into the NEWS file
+  * fa5aea1b6809: Move christian.org.uk into $RELIGION
+  * fa2cd57f91fa: When -n is specified, do not fetch other feeds
+  * a82bba3a4299: Break up downloads into categories using paths
+  * 142fabce355e: Added christian.org.uk feed
+  * 643f61e053c3: Add -q quiet mode into TODO
+  * 54971b2a7cd1: Issue #2 - Fix regression in 762258d9b82c
+  * 762258d9b82c: Fix issue #2 (Cron email reports contain raw progress output)
+  * e5030ee815d3: rsscache is no longer a fork, arc fork takes over the functionality
+  * d9ac0f1ddf5b: Mark URL for rate-limiting TODO entry
+  * 0dfb04059f52: Add additional Alistair Cooke's Letter From America feeds
+  * 562c97dff8d3: Add more Reith Lectures from 1948-1976 and BBC Thought for the day
+  * 87638d83907e: Add BBC Witness 2010 archive
+  * 6a419a37c84b: Add BBC Reith lectures
+  * b8c422bec640: Added BBC 'Witness' 2011 archive
+  * 3b746b7f75e1: Add Alistair Cooke - Letter From America
+  * 8427d186eb25: Add BBC "The Bottom Line" with Evan Davis
+  * d7cdbf46d34d: Add BBC Heart & Soul feed
+  * bb4903d30cd2: Add BBC 'Click' podcast
+  * e83f9eea22d7: Added BBC 'fooc' (From Our Own Correspondent)
+  * 7e10459b8b7f: Reverse order of changelogs, most recent first, in keeping with the version order.
+  * bc560cca9be9: Ensure that TODO list references all of the existing forks.
+  * 915a4a7ab6c4: Note that Debian packaging became available in 0.2.0 (historical note)
+
+ -- David Duncan Ross Palmer <palmer@overchat.org>  Mon, 04 Feb 2013 12:11:40 +0000
+
 dlpodget (0.2.1) stable; urgency=low
 
   * 10da18ca5027: Update copyright year in Makefile
 Depends: ${shlibs:Depends}, ${misc:Depends},
  libwww-perl,
  libxml-feed-perl,
- dl-libconfreader-perl,
+ libconfig-inifiles-perl,
  liburi-perl,
  libio-interactive-perl
 Description: Podcast downloader and management system
 use LWP::UserAgent;
 use XML::Feed;
 use Data::Dumper;
-use Daybo::ConfReader;
+use Config::IniFiles;
 use Getopt::Std;
 use URI::Escape;
 use IO::Interactive qw(is_interactive);
 	printf(STDERR "Downloading %s ... ", $entry->{uri}) if ( !$is_interactive );
 	printf(
 		STDERR "Setting timeout %u and retries %d\n",
-		$Feeds->{_main}->{TIMEOUT}, $Feeds->{_main}->{MAXTRIES}
+		$Feeds->{main}->{TIMEOUT}, $Feeds->{main}->{MAXTRIES}
 	) if ( $Debug );
-	$ua->timeout($Feeds->{_main}->{TIMEOUT});
-	$tries = $Feeds->{_main}->{MAXTRIES};
+	$ua->timeout($Feeds->{main}->{TIMEOUT});
+	$tries = $Feeds->{main}->{MAXTRIES};
 
-	unless ( $Feeds->{_main}->{noop} ) {
+	unless ( $Feeds->{main}->{NOOP} ) {
 		if ( open($f, '>>:raw', $local_podcast) ) {
 			binmode($f);
 		} else {
 			%get_params,
 			':content_cb' => sub {
 				my ( $chunk ) = @_;
-				print $f $chunk unless ( $Feeds->{_main}->{noop} );
+				print $f $chunk unless ( $Feeds->{main}->{NOOP} );
 			}
 		);
 	}
-	close($f) if ( !$Feeds->{_main}->{noop} );
+	close($f) if ( !$Feeds->{main}->{NOOP} );
 	printf(STDERR "done.\n") if ( !$is_interactive );
 }
 
 	my $tagRx = qr/^\$([A-Z0-9]+)/o;
 	my $avoid = 0;
 	while ( (my $idx = index($V, '$', $avoid)) > -1 ) { # Find remaining user-variable references
-		$avoid = 0;
 		my $var = substr($V, $idx);
 		if ( $var =~ $tagRx ) {
-			my $v = $Feeds->{_main}->{ uc($1) };
+			my $v = $Feeds->{main}->{ uc($1) };
 			warn(sprintf('%s -> %s', $1, $v || '(undef)')) if ( $Debug );
 			if ( !defined($v) ) {
 				$avoid = $idx+1;
 	my %feeds = ( );
 	my %opts = ( );
 	my %pids;
-	my $confSeen = 0;
 	my @confFiles = (
 		'dlpodget.rc',
 		"$ENV{HOME}/.dlpodget.rc"
 	$Debug = 1 if ( $opts{'d'} );
 	print(STDERR "Explicit debug mode enabled by -d\n") if ( $Debug );
 
-	$conf = Daybo::ConfReader->new();
 	foreach my $confFile ( @confFiles ) {
 		next unless ( -f $confFile );
-		if ( $conf->SetFn($confFile) ) {
-			$conf->Reload();
-			$confSeen++;
-			last;
+		$conf = Config::IniFiles->new(-file => $confFile, -commentchar => ';');
+		if ( !$conf ) {
+			print(STDERR "Fault with $confFile: " . join(',', @Config::IniFiles::errors) . "\n");
+			return 1;
 		}
+		last;
 	}
-	$conf = undef if ( !$confSeen );
 
-	if ( $conf && $conf->GetKeys(undef, \@defKeys) ) {
-		if ( 'enable' ~~ @defKeys ) {
-			if ( !$conf->GetDatum(undef, 'enable') ) {
-				print(STDERR "Configuration disabled.\n");
-				$conf = undef;
-			}
+	if ( $conf && (@defKeys = $conf->Parameters('main')) ) {
+		if ( !$conf->val('main', 'enable', 1) ) {
+			print(STDERR "Configuration disabled.\n");
+			$conf = undef;
 		}
 	}
 	if ( $conf ) {
-		my %reserveSec = map {  $_ => 1 } ( '_main', 'paths' ); # Reserved section names
+		my %reserveSec = map {  $_ => 1 } ( 'main', 'paths' ); # Reserved section names
 		my ( @sections, $secC );
-		$secC = $conf->GetSections(\@sections);
+		@sections = $conf->Sections();
+		$secC = scalar(@sections);
 
 		# First load the generation configuration information
 		# Include environment variables.
 		foreach my $ek ( keys(%ENV) ) {
-			$feeds{_main}->{ uc($ek) } = $ENV{$ek};
+			$feeds{main}->{ uc($ek) } = $ENV{$ek};
 		}
 		if ( scalar(@defKeys) ) {
 			foreach my $mk ( @defKeys ) {
-				$feeds{_main}->{ uc($mk) } = $conf->GetDatum(undef, $mk);
+				$feeds{main}->{ uc($mk) } = ProcessTags(\%feeds, $conf->val('main', $mk));
 			}
 			# Set debug flag via config if it existed.
-			if ( $feeds{_main}->{debug} ) {
-				$Debug = $feeds{_main}->{debug};
+			if ( $feeds{main}->{DEBUG} ) {
+				$Debug = $feeds{main}->{DEBUG};
 				printf(STDERR "Set Debug via config: %s\n", $Debug);
 			}
 		}
 
 		if ( 'paths' ~~ @sections ) { # Has a paths section?
 			my @secKeys;
-			if ( $conf->GetKeys('paths', \@secKeys) ) {
+			if ( (@secKeys = $conf->Parameters('paths')) ) {
 				foreach my $pk ( @secKeys ) {
-					$feeds{_main}->{ uc($pk) } = $conf->GetDatum('paths', $pk);
+					$feeds{main}->{ uc($pk) } = $conf->val('paths', $pk);
 				}
 				# For backward compatibility, set localpfx with [paths] root
-				$feeds{_main}->{'LOCALPFX'} = $feeds{_main}->{'ROOT'}
-					if ( $feeds{_main}->{'ROOT'} );
+				$feeds{main}->{'LOCALPFX'} = $feeds{main}->{'ROOT'}
+					if ( $feeds{main}->{'ROOT'} );
 			}
 		}
 
 		for ( my $secI = 0; $secI < $secC; $secI++ ) {
-			my @secKeys;
+			my ( @secKeys, $keyC );
 			next if ( $opts{'n'} && $sections[$secI] ne $opts{'n'} );
-			my $keyC = $conf->GetKeys($sections[$secI], \@secKeys);
+			@secKeys = $conf->Parameters($sections[$secI]);
+			$keyC = scalar(@secKeys);
 			foreach ( my $keyI = 0; $keyI < $keyC; $keyI++ ) {
 				next if ( $reserveSec{$sections[$secI]} ); # Skip reserved sections
-				my $v = $conf->GetDatum($sections[$secI], $secKeys[$keyI]);
+				my $v = $conf->val($sections[$secI], $secKeys[$keyI]);
 				$v = ProcessTags(\%feeds, $v);
 				printf(
 					STDERR
 		}
 	}
 
-	$0 = "$proctitle [MASTER]" if ( $feeds{_main}->{MAXCHILDREN} != 0 );
+	$0 = "$proctitle [MASTER]" if ( $feeds{main}->{MAXCHILDREN} != 0 );
 	foreach my $feedName ( keys(%feeds) ) {
 		my $reaped_pid;
 		my $child_limit_reached = 0;
 		next if ( !$feed->{enable} );
 		next if ( !$feed->{download} );
 		$feed->{name} = $feedName;
-		if ( $feeds{_main}->{MAXCHILDREN} == 0 ) { # Master performs downloads
+		if ( $feeds{main}->{MAXCHILDREN} == 0 ) { # Master performs downloads
 			$0 = "$proctitle [$feedName]";
 			Child(\%feeds, $feed);
 			next;
-		} elsif ( $feeds{_main}->{MAXCHILDREN} < 0 || scalar(keys(%pids)) < $feeds{_main}->{MAXCHILDREN} ) {
+		} elsif ( $feeds{main}->{MAXCHILDREN} < 0 || scalar(keys(%pids)) < $feeds{main}->{MAXCHILDREN} ) {
 			my $pid = fork();
 			die "cannot fork: $!" if ( !defined($pid) );
 			if ( $pid == 0 ) { # Child process
 ;   timeout   = <secs>, default 30, set to 0 to move back to library defined defaults (180)
 ;   maxtries  = <n>, The number of times to attempt to download a specific podcast
 ;
-; More generally, we have a default section, which contains the following options:
+; More generally, we have a main section, which contains the following options:
 ;
 ;   enable      = <1|0>, default 1, set to 0 for a convenient way to ignore the entire file.
-;   localpfx    = <prefix which will be exported to sections as $LOCALPFX>
 ;   noop        = <1|0>, default 0, set to 1 to disable modification of local files.
+;   debug       = <1|0>, default 0, Enable verbose maintainer information
 ;   maxchildren = <0-n>, default 0, max concurrent downloads. nb. setting this to 1
 ;                 is less efficient than 0, since a dedicated child is cloned for each feed.
 ;                 -1 means no limit (no more than the number of feeds in the configuration).
 
+[main]
 enable    = 1
 noop      = 0
 debug     = 0
 download  = 1
 enable    = 1
 
-[acooke]  ; Letter From America
+; Letter From America
+[acooke]
 localpath = $BBC/acooke
 rss       = http://downloads.bbc.co.uk/podcasts/radio4/acooke/rss.xml
 check     = 1