Source

mana-core-pyjobtransformscore / ChangeLog

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
package PyJobTransformsCore

=====
NOTE: + Changes without a tag number are in SVN trunk and they are
=====   the proposals for the next tag.
      + Tags for PyJobTransformsCore are ordered in inverse chronological order.
================================================================================

2010-09-17  Andy Buckley  <andy.buckley@cern.ch>
	* cmt/requirements: Fixing non-export of the declare_jobtransform
	pattern due to use of a private section in the arg-pickling
	part. Not sure why the end_private wasn't putting CMT backinto
	"public mode": I've just moved the private bit to the end of the
	requirements file.
	* Tag: PyJobTransformsCore-00-08-63

2010-09-16  Andy Buckley  <andy.buckley@cern.ch>
	* cmt/requirements: Another attempt to robustify the path to the
	share/mk-trf-argpickle script: now using
	$(PyJobTransformsCore_root) in place of "../" and putting it in a
	private section so that client packages won't try to run it.
	* Tag: PyJobTransformsCore-00-08-62

2010-09-14 Alvin Tan <clat@hep.ph.bham.ac.uk>
	* Missing GUID for all subclasses of PoolDataFile (include TAG
	files) results in a validation error.
	* Tag PyJobTransformsCore-00-08-61

2010-09-14 Alvin Tan <clat@hep.ph.bham.ac.uk>
	* accommodate '-' character in PVN for names that need to contain
	the '-' character.  i.e. for expansion handled by
	expandParallelVectornotation() and not glob().
	* Tag PyJobTransformsCore-00-08-60

2010-08-24  Andy Buckley  <andy.buckley@cern.ch>
	* cmt/requirements: Re-reverting to version with call to
	mk-trfarg-pickle script. Now works, by virtue of not trying to
	install the script.
	* Tag PyJobTransformsCore-00-08-59

2010-08-17 Alvin Tan <clat@hep.ph.bham.ac.uk>
	* Validation is now enabled by default.
	* Tag PyJobTransformsCore-00-08-58

2010-08-17 Walter Lampl <walter.lampl@cern.ch>
	* Emergency fix in trfutils.py: Number of events was no extracted
	from HIST files.
	* Tag PyJobTransformsCore-00-08-57

2010-08-14 Walter Lampl <walter.lampl@cern.ch>
	* full_trfarg.py/ OutputSkimmedBSFileArg: Next attept to fix the
	filenames changed by EventStorage lib.
	* Tag PyJobTransformsCore-00-08-56

2010-08-12 Walter Lampl <walter.lampl@cern.ch>
	* full_trfarg.py: OutputSkimmedBSFileArg allow for file-name
	altered by the underlying TDAQ software
	* Tag PyJobTransformsCore-00-08-55

2010-08-04 Alvin Tan <clat@hep.ph.bham.ac.uk>
	* call to h.GetNbinsX() may fail (attribute not found) if h is not
	the correct object.
	* Raise an error if existing input filename and defined output
	filename clash.  Fix for bug #71082.
	* OutputSkimmedBSFileArg should allow file to not be created.
	However, this only applies to the validation step. The subsequent
	prepareFileInfo() method will fail. Created a custom
	prepareFileInfo() for OutputSkimmedBSFileArg include this
	additional existence test.
	* Tag PyJobTransformsCore-00-08-54

2010-08-03 Alvin Tan <clat@hep.ph.bham.ac.uk>
	* Temporarily removed changes to cmt/requirements file and
	reverted back to working version.
	* Tag PyJobTransformsCore-00-08-53

2010-07-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added trfutil.py::timelimited_exec() to run snippets of Python code in
      a separate thread for x secs with y retries and z secs sleep between retries.
      ROOT file open (for event counting) and user-defined customised validation routines are
      executed this way now.
      trfutil.py::timelimited_exec1() executes routines in a subprocess with the same
      timeout and retry facility as timelimited_exec(). All POOL file event
      counting, validation routines and GUID extraction use this function.
      The default is 600 secs before timeout, 2 retries and 10 seconds sleep interval.
    * Disable DBRLEASE version checking when set to 'current'.
    * Tag PyJobTransformsCore-00-08-52

2010-07-22 Andy Buckley <andy.buckley@cern.ch>
    * Adding a placeholder CMT build action which will be used to create pickled argument
      lists at "make" time.

2010-07-13 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Validation is now switched off by default. i.e. --omitvalidation=ALL becomes the default.
    --omitvalidation=NONE will enable all validation routines. Specific ommissions work as usual.
    * Tag PyJobTransformsCore-00-08-50

2010-07-06 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Started adding doxygen comments in code.
    * test/genMetadataXML.py will now generate metadata.xml in new format with
      additional commandline option '--new' and in old format if '--old' (or omitted)
    * Fix for bug #69583.
    * Tag PyJobTransformsCore-00-08-49

2010-06-24 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Allow newly modified expandStringToList() function to accept non-existent files.
    * Fixed athena_wrapper.py's Athena (success) exit code issue.
    * Ignore filter for error caused by Py:ResourceLimit failure to set max resource limit
    * Fixed dataset extraction failure caused by expandStringToList() modification.
    * Disable event count for NTuples by default.
      Changed default value of "tree_names" parameter from None to [].
    * Remove hardcoded retry and sleep intervals. Values added to trfutil:TRF_SETTING dictionary.
    * Tag PyJobTransformsCore-00-08-48

2010-06-18 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added retry functionality for event counting.
    * LFN look-up (in PFC) performed if input file not local. 'LFN:' filename prefix
      will force LFN lookup.
    * Tag PyJobTransformsCore-00-08-47

2010-05-24 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fix for bug #67836. GUID extraction failure for POOL, TAG and BS files will
      result in job aborts. GUID extraction failures in other file types will
      result in new GUIDs being generated.
    * Tag PyJobTransformsCore-00-08-46

2010-05-21 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Partial fix for bug #67726.
    * Tag PyJobTransformsCore-00-08-45

2010-05-19 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Partial fixed for bug #67475. Allow a transform to be designated as the last in a chain.
      This allows for only the last trf in a chain to produce the final output log files and job reports.
    * Tag PyJobTransformsCore-00-08-44

2010-05-10 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed bug #67220. Slimmetadata script unable to deal with multiple '=' characters .
    * Added test/genMetadataXML.py script to assist with metadata.xml extraction in
      problematic jobs
    * Tag PyJobTransformsCore-00-08-43

2010-05-05 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed bug #66969. Limit for errors (i.e. per severity) not enforced correctly.
      Added minor optimisations.
    * Tag PyJobTransformsCore-00-08-41

2010-04-26 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Add error filter for Pythia message "ERROR WITH DELM".
    * Tag PyJobTransformsCore-00-08-40

2010-04-20 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * GUID retrieval mechanism sequence changed to FClistGUID, pool_extractFileIdentifier and AthFile.
    * Fixed bug #65873. Superfluous log message (of wrong severity level) removed.
    * Tag PyJobTransformsCore-00-08-39

2010-04-13 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Changed default GUID retrieval mechanism from AthFile to pool_extractFileIdentifier.
    * Tag PyJobTransformsCore-00-08-38

2010-03-24 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Introduce job level 'nevents' entry (max of all output file event counts)
      in jobReport.gpickle.
    * Tag PyJobTransformsCore-00-08-35

2010-03-23 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Add validation retry.
    * Tag PyJobTransformsCore-00-08-34

2010-03-18 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Temporary dmesg dump when signal received by trf.
    * Tag PyJobTransformsCore-00-08-33

2010-03-15 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added error mask for Sherpa (request by Junichi)
    * Tag PyJobTransformsCore-00-08-31

2010-03-10 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * New format job report ready. Specify --usenewmeta=True to create
      jobReport.gpickle and metadata.xml in the new format.
    * Combined/replaced JobConfigError and OptionError with TransformArgumentError.
    * Reviewed all code to ensure that the use of slices (e.g. temp[0]) is
      correctly protected against exceptions where required.
    * VTimer.py:pymon() code placed in try/except clause. An exception will result in
      cpu, vmem and rss values being set to -1.
    * Tag PyJobTransformsCore-00-08-30

2010-03-01 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * trfs without input files fail due to a faulty check. Fixed!
    * Tag PyJobTransformsCore-00-08-28

2010-02-25 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * PyJobTransformsCore-00-08-26 is broken due to some syntax errors in log messages.
    * Tag PyJobTransformsCore-00-08-27

2010-02-24 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * In FakeAthena.py, reinstating include.block() functionality albeit differently.
    * Tag PyJobTransformsCore-00-08-26

2010-02-12 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * -t/--test now used to apply the omission of checks (e.g. eventCount() )
      everywhere as opposed to only within validation routines.
      - omits checks specified by --omitvalidation
      - omits the extraction/generation of guids
    * Removed superfluous logger argument from all FileType:eventCount() functions.
    * Moved some functionalities from full_trfarg.py:NutpleFileArg into
      trfutil.py:NtupleFile and create trfutil.py:MonitorHistFile.
    * Tag PyJobTransformsCore-00-08-24

2010-02-09 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * When --ignoreerrors=True (or --ignoreunknown), A transform is deemed successful
      when it's validation is successful and its associated athena.py exit code is 0.
      Note: This is so even when there are errors detected by the trf.
      Fix for bug #60546.
    * Guard against AthFileObj.nentries returning 'N/A'. Parallel fixed for bug #62018.
    * TransformValidationError not visible when trfutil is imported. Fixed.
    * Enabled support for libtcmalloc_minimal (default).
    * Tag PyJobTransformsCore-00-08-23

2010-01-26 David Cote  <david.cote@cern.ch>
    * Introduced new method ntup_entries (in trfutil.py) used by eventCount methods
      of HIST, NTUP and TAG (in full_trfargs.py)
    * Tag PyJobTransformsCore-00-08-22

2010-01-26 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Re-introduced timing of validation routines.
    * Fixed --ignoreunknown bug where the option is not propagated properly to sub-transforms.
    * Changed exception raised by validation routine failures to TransformValidationError.
      This is to prepare for the revision of how exitcode is derived.
    * Tag PyJobTransformsCore-00-08-21

2010-01-24 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Errors matching those found in atlas_error_patters.db are treated as errors
      of severity FATAL. This has been corrected and the severity will now depend
      on the error itself. If severity cannot be deduced, it is treated as a FATAL error.
    * VTimer made more specialised. Results presented in hierarchical format
      corresponding to the execution structure of a transform and it's sub-transforms.
    * Tag PyJobTransformsCore-00-08-20

2010-01-13 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added ignore error filters to mask out TRT alignement errors.
      (Requested by Davide Costanzo)
    * Tag PyJobTransformsCore-00-08-19

2010-01-12 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed bug that did not consider the case where maxEvents=-1 when applying
      efficiency correction and event number comparisions.
      (Thanks to Borut for noticing this problem!)
    * Tag PyJobTransformsCore-00-08-18

2009-12-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed slimmetadata which mangled the & character when when shell used was
      switched causing metadata chaos!
    * Tag PyJobTransformsCore-00-08-17

2009-12-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Moved getGUID() from PoolDataFile class into RootTTreeFile class
    * Tag PyJobTransformsCore-00-08-16

2009-12-14 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * First attempt at fixing problem in VTimer where stop times may be None when calculating cpuTime and wallTime. This fix is not complete but will not cause more problems. More investigation required.
    * Tag PyJobTransformsCore-00-08-15

2009-12-12 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * slimmetadata encountering 'Word too long' error. Modified slimmetadata to
      use sh instead of csh as the latter fails when the environment contains
      variables with strings of more than 1024 characters. The $PATH environment
      variable is known to exceed this limit. This causes metadata.xml slimming failure.
    * Tag PyJobTransformsCore-00-08-14

2009-12-08 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Renamed metadata key from 'BeamType' to 'beamType'.
    * Enhanced VTimer. Mutiple timers supported.
      VTimer result template changed as requested to facilitate parsing.
      Support for pickled VTimer results (dictionary).
      Reimplement pymon() as the original function does not take into account child processes.
      Moved functionality from trfutil.py into newly created VTimer.py
    * LoggedMessageSvc enabled by default.
    * Added an explicit 'maxEvents = -1' job option if 'maxEvents' not specified.
      This will allow Reco_trf.py and related transforms to work without having to
      specify the argument explicitly.
    * Tag PyJobTransformsCore-00-08-13

2009-11-30 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * When maxEvent is omitted (when it is expected), a default value of -1 is now used.
    * Tag PyJobTransformsCore-00-08-12

2009-11-30 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * LFNs of input/output files listed in jobReport.gpickle should not contain full paths. Fixed.
    * --ignoreerrors changed to only accept True or False.
      --ignoreerrors=True is now equivalent to --ignoreerrors=ALL.
    * --extraignorefilters=XXX.db is now equivalent to --ignoreerrors=XXX.db
    * minor fixes.
    * maxEvent=-1 at the command line will no longer be replaced with maxEvent=NO_OF_EVENTS_IN_FILE
    * Tag PyJobTransformsCore-00-08-11

2009-11-23 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * VTimer cpu time assumed to be s rather than ms. Fixed.
    * test/testErrorPattern.py is now timed.
    * Tag PyJobTransformsCore-00-08-10

2009-11-23 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Force trf to return successful when Athena returns successful
      even when errors are detected. Previously, if a FATAL error exists,
      the Athena success is overruled.
    * Use of LoggedMessageSvc triggered by --usenewmeta=True (temporary).
      Now creates different files for each transform step.
    * Minor change to VTimer to use logger if it exists rather than print.
    * Fixed bug that set usenewmeta to True by default. The default is now False.
    * Tag PyJobTransformsCore-00-08-09

2009-11-21 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed checklog.py to use new (since 00-08-00) job report structure.
    * Use AthFile.cache where possible to provide GUID.
    * Treat all log lines of severity INFO as errors to be ignored.
      INFO lines undergo a single filter rather than 3 filters.
    * Extend 'fileType' metadata to all FileArg subclasses.
    * Add support for LoggedMessageSvc. Experimental.
    * Tag PyJobTransformsCore-00-08-08

2009-11-13 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added --usesqlite option to provide a finer-grained control over the
      type of database to use.
    * Added a 'intermediate' property for DataFileArg and its subclasses.
      Data files with this property will not be validated.
    * Tag PyJobTransformsCore-00-08-07

2009-11-12 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * AthFile.flush_cache() on loading the AthFile package.
    * Added 'fileType' metadata to file entry in metadata.xml
    * Added --mcinput option to distinguish between Monte Carlo and real data.
      The ommission of this option implies that real data is used and
      so SQLiteSupport is disabled
    * Tag PyJobTransformsCore-00-08-06

2009-11-11 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added -c/--command command line option as used in athena.py
    * Tag PyJobTransformsCore-00-08-05

2009-11-06 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Validation routines are timed (wall time, cpu time, rss, vmem)
    * Add a 'temporary' property for FileArg and its subclasses. To allow for selective processing of file metadata i.e. temporary files and their metadata are ignored.
    * Modified DBReleaseArg to allow for more parameters in the constructor so as to
      pass more information to the parent class.
    * Tag PyJobTransformsCore-00-08-04

2009-11-06 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * made metadata filters more flexible
    * Improve the behaviour and robustness of SQLiteSupport class.
      Pre-existing links are not removed now unless they reside on another NFS mount.
    * Tag PyJobTransformsCore-00-08-03

2009-10-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed the problem where the include() function (in FakeAthena.py) was not
      visible to job option scripts as it was missing from the environment
      dictionary used.
    * The _fileInfo attribute and its corresponding accessor was only available
      for data file arguments. This posed a problem for the likes of
      InputGeneratorFileArg.
      The _fileInfo attribute and accessor have now been promoted to the FileArg class.
    * Tag PyJobTransformsCore-00-08-02

2009-10-27 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added some metadata filters
    * Fixed a couple of FakeAthena.py bugs
    * Tag PyJobTransformsCore-00-08-01

2009-10-21 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Merge from PyJobTransformsCore-00-07-63-branch
    * Tag PyJobTransformsCore-00-08-00

2009-10-20 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Re-organise share/atlas_error_ignore.db to consolidate filters for rel 14 and 15.
      Without this reorganisation, rel 16 will not be filtered!
    * test/testErrorPattern.py able to accept entire log file now rather than just a string.

2009-10-18 Ketevi A. Assamagan <ketevi@bnl.gov>

	* need to ignore muon cabling error
	* Tag PyJobTransformsCore-00-07-66

2009-10-13 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Omit all validation when --ignoreall is set.
    * Modified share/testErrorPatter.py to accept log file.
    * Corrected error/ignore filter. Blank spaces at start of pattern not ignored!
    * Tag PyJobTransformsCore-00-07-65

2009-10-07 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Change to logic of 'NEEDCHECK' status
    * Tag PyJobTransformsCore-00-07-64

2009-10-06 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Minor modifications
    * Basis for branch point for new metadata format
    * Tag PyJobTransformsCore-00-07-63

2009-10-01 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added SIGBUS to list of signals to handle.
    * Changed command line option usenewmeta's valid choices from a tuple to
      a list to avoid string conversion problem
    * Re-enable '-t' option but warn of depreciation.
    * Added 'testEventMinMax' to VALIDATION_DICT to replace '-t' option.
    * Tag PyJobTransformsCore-00-07-62

2009-09-30 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Removed debug leftover from AtlasErrorCodes.py which raised an error
      when acroynm or code missing from ErrorInfo object.
    * Tag PyJobTransformsCore-00-07-61

2009-09-30 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Added 'CANBEMIGR' to list of recognised stager_gry status.
    * Tag PyJobTransformsCore-00-07-60

2009-09-29 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Force copy in slimmetadata script
    * Tag PyJobTransformsCore-00-07-59

2009-09-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Reintroduce XXX_runathena script for debugging purposes.
    * Tag PyJobTransformsCore-00-07-58

2009-09-28 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fix for bug #56179
    * Tag PyJobTransformsCore-00-07-57

2009-09-25 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed problem caused by the use of --tcmalloc and --stdcmalloc parameters
      concurrently.
    * Enable the various validation routines to be switched off at the
      command line using the new --omitvalidation=XXX,YYY,...
      See help for available XXX,YYY,etc. values.
    * Modified jobReport.gpickle to return 'NEEDCHECK' in its 'trfAcronym' entry if
      the trfExitCode=0 but trfCode !=0.
    * Removed the hardcoded default memory manager tcmalloc from the trf core
      since it is athena's default anyway.
    * Tag PyJobTransformsCore-00-07-56

2009-09-22 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Stop corruption detection of unsupported file types.
    * Tag PyJobTransformsCore-00-07-53

2009-09-18 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Attempt to fix bug #52067 with removal of commands.getoutputstatus() and
      replace with subprocess.check_call() in JobReport.slimdownMetadata().
    * Tag PyJobTransformsCore-00-07-52

2009-09-12 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * Fixed bug that allowed FATAL errors to be ignored when
      using --ignoreerrors=ALL.
    * Tag PyJobTransformsCore-00-07-51

2009-09-09 Alvin Tan <clat@hep.ph.bham.ac.uk>
    * validateFile() in trfutil.py will now throw specific output file exceptions.
    * Catch exceptions thrown by validateFile function in post run actions and
      add relevant transform errors.
    * Undo modification made to athena exit code and acronym handling in runJob().
    * Fixed bug in JobReport.exitCode() which throws an exception when the job report
      does not contain the athena exit code.
    * Fixed bug in JobReport.getExitError() which failed to flag a missing
      athena exit code in the report as an error.
    * Modified athena_wrapper.py to return athena exitcode.
      jobReport_athena.pickle is now only created if an exception is raised by Athena.
    * Fixed bug with JobReport's addOutputFile() and addInputFile().
    * Minor changes to how Athena is wrapped and run.
      Removed unnecessary execution of athena in multiple layers of shells.
    * Re-enabled output file validation with Event/PyDumper/SgDumpLib.py.
      DPD file corruption is still disabled. As DPD files with 0 events are not permitted.
    * Support for GriBB has been removed from all relevant files.
    * Tag PyJobTransformsCore-00-07-50

2009-09-07 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Disabled output file validation with Event/PyDumper/SgDumpLib.py. Stability issues.
   * Tag PyJobTransformsCore-00-07-48

2009-09-07 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Corrected JobReport:producerFilename() causing exception when
     JobReport.__producer not defined.
   * Changed metadata collection mechanism. Removed reliance on slimmetadata script.
     Introduced --usenewmeta=True to output metadata.xml in new format.
     WARNING: Work in progress.
   * Added mechanism to read pre-defined metadata filters. Adhoc metadata filters and
     explicit metadata inclusion is also supported using the --extrametadatadict option
     by giving it a python script written in the same format as extraMetadata.py.
   * Added mechanism allow adhoc inclusion of error ignore filters.
     Specify the --ignoreerrors option at the command line and give it a text file
     written in the same format as atlas_error_ignore.db.
     A special case of --ignoreerrors=ALL will force the trf to return successful
     as long as athena is successful as well.
   * DPDFiles will not be validated using the new SgDumpLib for now.
   * Tag PyJobTransformsCore-00-07-47

2009-08-31 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Added triggerDB to list of files in SQLiteSupport class
   * Removed use of XXX_messageSvc_jobOptions.py
   * Removed unnecessary '-l' parameter at the command line.
     '-l' hould now be added to --athenaopts instead.
   * Added output file validation with Event/PyDumper/SgDumpLib.py. May fall
     back on standard event count routine if run_sg_dump() receives a signal.
   * Removed superfluous eventCount() code for InputDPDFileArg and
     OutputDPDFileArg
   * Moved subclass specific functionality in OutputFileArg:postRunAction()
     into the relevant subclasses.
   * Encode embedded quotes when creating metadata.xml file.
     Modified slimmetadata script to handle the '&' character as it
     has special meaning for sed.
   * Added customisable validation behaviours for OutputFileArg
     and it's subclasses.
   * Add dummy support for xrootd access i.e. should not fail.
   * Tag PyJobTransformsCore-00-07-46


2009-08-19 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Merged PyJobTransformsCore-00-07-41-02 with PyJobTransformsCore-00-07-43
   * Tag PyJobTransformsCore-00-07-44

2009-08-10 David Cote <david.cote@cern.ch>
    * Removed TRF_GEO_MISMATCH from share/atlas_error_patterns.db, needed for ESD-ESD reprocessing
    * Tag PyJobTransformsCore-00-07-43

2009-08-04  Sebastien Binet  <binet@farnsworth>

	* tagging PyJobTransformsCore-00-07-41-02
	* implementing TAGFile.eventCount (AthFile does not completely
	  support (yet?) TAG files even though they are POOL files)
	* M python/trfutil.py

2009-07-20 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Added event count matching mechanism/facility
   * Tag PyJobTransformsCore-00-07-41-1

2009-07-17  Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Corrected a typo in a print statement in the signal handler
     that causes an exception
   * Tag PyJobTransformsCore-00-07-41

2009-07-15   Alvin Tan <clat@hep.ph.bham.ac.uk>
   * ByteStream event counting added (since PyJobTransformsCore-00-07-39).
   * Added signal handling to fix for bug #52543.
   * Tag PyJobTransformsCore-00-07-40

2009-07-08 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Changed of arguments when constructing the athena command.
     athenaopts will now come behind all the job option files.
   * Tag PyJobTransformsCore-00-07-39

2009-07-03 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Fix PyJobTransformsCore-00-07-38 which does not handle exitcode
     consistently.
   * Tag PyJobTransformsCore-00-07-38-01

2009-06-30 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Further issues with --ignoreunknown resolved. Now includes check for
     ATH_FAILURE which will overrule --ignoreunknown if the former is present.
   * Raise more specific errors when encountering SystemExit (ATH_FAILURE) and
     KeyboardInterrupt (TRF_KEY_INTERRUPT) exceptions rather than
     simply ATH_EXC_PYT.
   * Resolved bugs related to composite transforms executing it's subsequent
     steps regardless of the previous step's exit code.
     This is a modification to the fix for bug #51093 to make it more effective.
   * tag PyJobTransformsCore-00-07-38

2009-06-26 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * set default memory allocation mechanism to tcmalloc
   * tag PyJobTransformsCore-00-07-37

2009-06-04 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Fixed bug #51093... really!
   * Fixed broken PyJobTransformsCore-00-07-35
   * tag PyJobTransformsCore-00-07-36

2009-06-03  Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Fixed bug #51093
   * tag PyJobTransformsCore-00-07-35

2009-05-15  Sebastien Binet  <binet@farnsworth>

	* tagging PyJobTransformsCore-00-07-32
	* python/trfutil.py: adapt to new AthFile api (fopen returns a list of
	  AthFiles when it is given a list of filenames)

2009-05-07 Alvin Tan   <clat@hep.ph.bham.ac.uk>
   * added ignore error filter for pool on Ketevi's request
   * tag PyJobTransformsCore-00-07-29

2009-05-05 Alvin Tan   <clat@hep.ph.bham.ac.uk>
   * Fixed bug #45005. Modified generic error filter to recognise the error.
   * tag PyJobTransformsCore-00-07-28

2009-04-17 David Cote  <david.cote@cern.ch>
   * Added support for ,, syntax a la BasicExec, for ListOfStringsArg (e.g. preInclude)
   * Removed overly zealous ERROR messages
   * tag PyJobTransformsCore-00-07-27

2009-04-16 David Cote  <david.cote@cern.ch>
   * Introduced support for blank spaces, double quotes, and unambiguous line-change with preExec/postExec
     This involves: new BasicExec class, modified ListOfStrings class and new method StringToExec.
     I also removed now useless checkString method.
   * tag PyJobTransformsCore-00-07-26

2009-04-16    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Added new error pattern as required by bug #49082
   * tag PyJobTransformsCore-00-07-25

2009-04-07    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Removed the use of NamedTemporaryFile() function from
     JobReport.slimdownMetadata() to avoid /tmp (or is it AFS) problems.
   * tag PyJobTransformsCore-00-07-24

2009-04-03    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Added ignore error filters as requested in bug #47761.
   * tag PyJobTransformsCore-00-07-23

2009-04-01    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Placed access to AthFile's fopen() within try/except clause
   * tag PyJobTransformsCore-00-07-22


2009-03-31    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Fixed bug causing failure when reading empty lines in .db files.
   * Removed spurious "Ignoring ALL errors." message.
   * tag PyJobTransformsCore-00-07-21


2009-03-30    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Updated v14.2-specific ignore errors filters to work for all v14 releases.
   * tag PyJobTransformsCore-00-07-20

2009-03-23    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Fixed bug #48413
   * tag PyJobTransformsCore-00-07-19

2009-03-17    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Corrected the updated filters in previous tag.
   * tag PyJobTransformsCore-00-07-18

2009-03-16    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * Updated filters in atlas_error_ignore.db to work in release 15.
   * tag PyJobTransformsCore-00-07-17

2009-03-16    Alvin Tan <clat@hep.ph.bham.ac.uk>
   * corrected typo in trf.py:getSkeletonTemplate(). Return value was
     non-existent.
   * In trf.py:#1471, placed job report read() function in try/except to
     avoid an IOError when trying to read a yet to be created job report
     pickle file.
   * Added --rss and --vmem options to allow max resident memory and
     max virtual memory to be set.
   * corrected behaviour of --ignoreunknown. Fixed bug #45885.
   * tag PyJobTransformsCore-00-07-16

2009-03-04 Alvin Tan <clat@hep.ph.bham.ac.uk>
   * corrected typo in trf.py when initialising the variable
     _runArgsArgumentsTemplate
   * Fixed bug #45769. Generic error filter was inadequate.
   * tag PyJobTransformsCore-00-07-15

2009-03-03 Emil Obreshkov <Emil.Obreshkov@cern.ch>
	* share/atlas_error_ignore.db cleaning cibak changes from the HEAD
	* tag PyJobTransformsCore-00-07-13

2009-03-01 Emil Obreshkov <Emil.Obreshkov@cern.ch>
	* share/atlas_error_ignore.db more errors to be ignored
	* tag PyJobTransformsCore-00-07-01-03

2009-02-27  Krzysztof Ciba  <cibak@lxplus235.cern.ch>
	* share/atlas_error_ignore.db added patters for TrigIDSCAN_FullScan.IDScanZFinder_FullScan
	* cvs tag PyJobTransformsCore-00-07-01-02

2009-02-15 Emil Obreshkov <Emil.Obreshkov@cern.ch>
	* new error to ignore from Ketevi
	* tag as -00-07-01-01

2009-02-13 Alvin Tan
   * Corrected typo causing the slimdownMetadata problem.
   * Corrected typo in TransformConfig.py's JobConfig.hasProperty() method which
     will result invariably in an exception.
   * tag PyJobTransformsCore-00-07-12

2009-02-11 Alvin Tan
   * Modified JobReport.py. A staticmethod slimdownMetadata() was introduced
     in the JobReport class which invokes a script (share/slimmetadata) to
     remove redundant metadata. cmt/requirements was modified to accommodate
     this script.
     Many thanks to Pavel for providing this script as a temporary solution.
   * tag PyJobTransformsCore-00-07-11

2009-01-30 David Cote
   * Introduced OutputSkimmedBSFileArg in python/full_trfarg.py and modified postRunAction()
     method of class OutputFileArg in python/basic_trfarg.py to accept not created output
     file only if it's an instance of OutputSkimmedBSFileArg.
   * tag PyJobTransformsCore-00-07-10

2009-01-29 Alvin Tan
   * Modified athena_wrapper.py to allow tcmalloc to be disabled.
   * Modified RunOptions in runargs.py to accept tcmalloc and leakcheckexecute options
   * Modified trf.py to handle the abovementioned options and pass them to athena correctly.
   * Removed Martin Woudstra's email from code to avoid confusion.
   * tag PyJobTransformsCore-00-07-09

2009-01-21 Alvin Tan
   * python/athena_wrapper.py: fixed bug 45485 (tcmalloc issue)
   * tag PyJobTransformsCore-00-07-08

2009-01-21 David Cote
	* Modified postRunAction in basic_trfarg to handle empty output DPD files.
	* tag PyJobTransformsCore-00-07-07

2008-12-18  David Rousseau  <droussea@lxbuild084.cern.ch>

	* python/trfutil.py (PoolDataFile.eventCountAthena): fix merging
	* tag PyJobTransformsCore-00-07-05

2008-12-12  David Rousseau  <droussea@lxbuild084.cern.ch>
	* python/trfutil.py (checkString): replace ";" by " " (to allow
	using space at Tier1)
	* tag PyJobTransformsCore-00-07-03

2008-11-25 David Cote <david.cote@cern.ch>
	* New checkString function in python/trfutil.py
	* tag PyJobTransformsCore-00-07-02

2008-10-15  Krzysztof Ciba  <cibak@lxplus245.cern.ch>
	* share/atlas_error_ignore.db added pattern for fixing bug #42879
	* cva tag PyJobTransformsCore-00-07-01

2008-10-08 David Cote <david.cote@cern.ch>
	* Revert to PyJobTransformsCore-00-06-98 (-99 didn't work for argDic with non-string values)
	* Added self._runOpts.extractOptions(argList) at the beginning of exeArgDict().
	* Tag PyJobTransformsCore-00-07-00

2008-10-08 David Cote <david.cote@cern.ch>
	* Bug fix in trf.exeArg: it ignored RunOptions. We now translate the argDic into an argList,
	  then make sure exeSysArgs() and exeArgDic() uses the exact same method downstream.
	* Tag PyJobTransformsCore-00-06-99

2008-09-26 David Cote <david.cote@cern.ch>
	* New full argument BasicStringArg.
	* DPDConfigArg copied to ListOfStringsArg, DPDConfigArg conserved for backward compatibility
	* Tag PyJobTransformsCore-00-06-98

2008-09-16  Krzysztof Ciba  <cibak@lxplus229.cern.ch>

	        * share/atlas_error_ignore.db fixed pattern for bug #41518
		* share/atlas_error_pattern.db share/atlas_error_categories.db added new valid patterns for ATH_CON_ORA_SES errors
		* cvs tag PyJobTransformsCore-00-06-97

2008-09-15  Krzysztof Ciba  <cibak@lxplus216.cern.ch>

	        * small fix in processing of "--ignoreunknown" options
		* share/atlas_error_patters.db share/atlas_error_categories.db removed ATH_CON_ORA_SES error
		* share/atlas_error_ignore.db added pattern for Herwig(?) "THIS ERROR OCCURED DURING A SECONDARY SCATTER AND WAS"


2008-09-14 David Cote <david.cote@cern.ch>
	* Added toPython() method in class DpdConfigArg and make it a list instead of string (on top of PyJobTransformsCore-00-06-94).
	* Tag PyJobTransformsCore-00-06-95

2008-09-10  Krzysztof Ciba  <cibak@lxplus256.cern.ch>

	        * --ignoreunknown option added to JobTransform class for ignoring all TRF_UNKNOWN errors from
	athena log checks


2008-09-09 PyJobTransformsCore-00-06-91
	- cibak
        - new error pattern to atlas_error_ignore.db added


2008-09-07  PyJobTransformsCore-00-06-90
	- David.Cote@cern.ch
	- Introduced class DpdConfigArg in python/full_trfarg.py (required by PATJobTransforms-00-00-27)

2008-08-13  PyJobTransformsCore-00-06-88
        - Jamie.Boyd@cern.ch
	- Update error ignore list for cosmic recprocessing


2008-08-13:   PyJobTransformsCore-00-06-87
	Martin.Woudstra@cern.ch
	- fix mistake in JobTransform.sharedRunOpts

2008-08-11  PyJobTransformsCore-00-06-86
	  - Jamie.Boyd@cern.ch
	  - Add in ignore list errors from RecExCommissioning running
===============================================================================
2008-07-23 PyJobTransformsCore-00-06-85
2008-07-23 David.Quarrie@cern.ch
   - Extend that error filtering to all 14.X.Y releases
2008-07-22 Martin.Woudstra@cern.ch
  - Add new option: --argdict=<arguments_file.pickle>
	to run trf from pickle file with argument (name,value) dictionary
	Requested by Tier0

===============================================================================
2008-07-15  PyJobTransformsCore-00-06-84
  - Changing filtering schema numbering according with the new release strategy
  - Adding filtering for
     CoreDumpSvc          INFO Handling signals: 11(Segmentation fault) 7(Bus error) i
     4(Illegal instruction) 8(Floating point exception)
===============================================================================
2008-06-16  PyJobTransformsCore-00-06-83
  - Modifications for the DPD creation used in FDR2
  - CleanUp of the error filtering file, the lines that were not in used were
    eliminated
===============================================================================
2008-05-27  PyJobTransformsCore-00-06-82
  - Adding DPD file arguments for the DPD in FDR2

===============================================================================
2008-05-23  PyJobTransformsCore-00-06-81

  - Adjusted the counting of events to the change done in Athena core in which the
   line "end of event" was changed by "done processing event"
===============================================================================
2008-05-06  PyJobTransformsCore-00-06-80

M. Gallas
  - ConditionsTag  moved to ConditionsTagArg (in principle only changes in the
  RecoJobTransForms/scripts are needed. Production should not be affected and
  they will be happy.


===============================================================================
2008-05-05  PyJobTransformsCore-00-06-79

M. Gallas
  - More error filtering for Trigger reco_ESD
===============================================================================
2008-05-05  PyJobTransformsCore-00-06-78

M. Gallas
  - Added filtering for trigger RDOtoBS as a hack for 14.1.0.1
  - Added filtering for stuck events in G4
===============================================================================
2008-04-18  PyJobTransformsCore-00-06-77

M. Gallas
  - 3 lines of error filtering for reco ESD and AOD in 14.1.0
===============================================================================
2008-04-16  PyJobTransformsCore-00-06-76

M. Gallas
  - Starting the error filtering for 14.1.0 almost from scratch
===============================================================================
2008-04-07  PyJobTransformsCore-00-06-75

M. Gallas
  - A bit more of error filtering
===============================================================================
2008-04-04  PyJobTransformsCore-00-06-74

M. Gallas
  - A bit more of error filtering

===============================================================================
2008-04-03  PyJobTransformsCore-00-06-73

M. Gallas
  -  The argument "ConditionsTag" is made available centrally for all the trfs
  that can need it.
===============================================================================
2008-03-25  PyJobTransformsCore-00-06-72

M. Gallas
  - Renamed the release number for filtering to 14.0.0 or 14.[1-3].0  (instead 13.X.0
as it was in 13.2.0). We have started with the same list as in 13.0.30.3 (14/11/07)
and two adittional requests for 14:

Py:Configurable        , ERROR attempt to add a duplicate \(CellCalibrator.CellCalibrator.H1WeightCone7H1Tower\) ... dupe ignored
TGCgeometrySvc         , ERROR Cannot find file  'FileName':TGCgeometry.ATLAS-CSC-02-00-00.8.lut

  The idea is to drop old error filtering in the following days

===============================================================================
2008-03-13  David Rousseau  <droussea@lxplus222.cern.ch>
         PyJobTransformsCore-00-06-71
   -  share/atlas_error_ignore.db: mask
Py:Configurable, ERROR attempt to add a duplicate \(CellCalibrator.CellCalibrator.H1WeightCone7H1Tower\) ... dupe ignored
===============================================================================
04-03-2008  PyJobTransformsCore-00-06-70

M. Gallas
 - Changes for Python 2.5 affecting athena_wraper.py exceptions

===============================================================================
28-01-2008  PyJobTransformsCore-00-06-69

 M. Gallas
 - Possible quick fix-hack for the poll() python bug (affecting the untar of the
   DBRelease)
 - Updated the Release number to 13.2.0 in the error ignore list
 - Removed the automatic re-run of the job in dbg mode once the job has failed,
   as it was requested by the production team

===============================================================================
10-12-2007  PyJobTransformsCore-00-06-68

 M. Gallas
 - Mods suggested by David Roussseau to improve the sorting of error line

===============================================================================
14-11-2007  PyJobTransformsCore-00-06-67

 M. Gallas
 - Migration to configurables for the EventSelector and updating the ignored
   DB error lines for using 13.1.0
===============================================================================
14-11-2007  PyJobTransformsCore-00-06-66

 M. Gallas
 - Dummy tag after the cvs branch for 13.0.40 or 13.0.30.4. This tag is the
same as PyJobTransformsCore-00-06-65 and:
   13.0.X should use now the branch PyJobTransformsCore-00-06-65-branch
   13.X.0 should use PyJobTransformsCore-00-06-67

===============================================================================
06-11-2007  PyJobTransformsCore-00-06-65

 M. Gallas
 - Adding 3 lines for ERROR filtering in the tag trf

===============================================================================
31-10-2007  PyJobTransformsCore-00-06-64

 M. Gallas
  - Adding support for BS ouputfiles

===============================================================================
26-10-2007  PyJobTransformsCore-00-06-63
  M. Gallas

  - Adding error filtering for reco_TAG using trigger
  - Adding initial support for BS files

===============================================================================
11-10-2007  PyJobTransformsCore-00-06-62
  M. Gallas

  - Adding Trigger ERROR filtering for 13.0.30.2 and preparing 13.0.40

===============================================================================
25-09-2007  PyJobTransformsCore-00-06-61
  M. Gallas

  - Removed one line ERROR from Pool which is not needed any more (D. Malon)
  - Added 13.0.30 as base release in the error filtering

===============================================================================
31-07-2007  PyJobTransformsCore-00-06-60
  M. Gallas
  - Adding one more line to the ERROR DB filtering list for trf_reco-AOD
===============================================================================
28-07-2007  PyJobTransformsCore-00-06-59
  M. Gallas
  - Adding one more line to the ERROR DB filtering list for AtlFast
===============================================================================
28-07-2007  PyJobTransformsCore-00-06-58
  M. Gallas
  - Adding two more lines to the ERROR DB filtering list
===============================================================================
23-07-2007  PyJobTransformsCore-00-06-57
  M. Gallas
  - Adding 13.0.10 13.x.0 development branch Configurable errors in case of 13.0.20
===============================================================================
23-07-2007  PyJobTransformsCore-00-06-56
  M. Gallas
  - Adding 13.0.10 13.x.0 development branch Configurable errors in case of a bug fix
nightly. It may happen that depending on CMT you get rel_0 or bug_0

===============================================================================
12-07-2007  PyJobTransformsCore-00-06-55

  M. Gallas
  - Added protection to the trf when it look for files in the dir. It may happen
  that the ./AFSnumber will be deleted
  - Added the InputBSFileArg argument type for the BS trfs
  - Modified stuff for the gpickle summary. Now it is activated except for the
  InputFile info which is not clear what we want
  - Updated the ERROR database for 13.0.20 and one more error for reco
===============================================================================
19-06-2007  PyJobTransformsCore-00-06-54
  M. Gallas
  - Adding filtering of errors for reco (ESD->AOD) provided by D. Rousseau
    This will go only in the 13.0.10.1 cache
===============================================================================
15-06-2007  PyJobTransformsCore-00-06-53
  M. Gallas
  - We don't write gpicke in the first 13.0.10.1 cache
  - Adding error filtering lines for generators and reco_AOD in the minimal
    configuration.
===============================================================================
11-06-2007  PyJobTransformsCore-00-06-52

  M. Gallas
  - Added error filtering list for reco in 13.0.10.1
===============================================================================
06-06-2007  PyJobTransformsCore-00-06-51

  M. Gallas
  - Updating the sqlite db to 200 in  trfutil.py. The sqlite200 is used with
    the DB release 4.0.1 (COOL2) and it is only valid in 13.0.X
  - the gpickle will not be used in 13.0.10 release and it will come with the
  first production cache
     + Using protocol 0 for the gpickle file
     + Added more info into the gpickle file
  - Adding appropiate support for CASTOR in T0 and CANBEMIGR as Armin request
   In the setup of CASTOR we use version 2 but we take the values of STAGE_HOST and
   STAGE_SVCCLASS from the shell. This willl make possible to run the trfs in the T0
  - Removed the ERROR lines ignored in 12.0.X production caches.

===============================================================================
07-05-2007  PyJobTransformsCore-00-06-50
  M. Gallas
  - Added first of the gpickle format for the jobreport output.
  - Switched off gbb (it hangs in slc4 simulation jobs)

===============================================================================
26-04-2007 PyJobTransformsCore-00-06-49
  M. Gallas
  - Starting 13.0.X nigthlies
  - It seems that the that LD_PRELOAD="$LD_PRELOAD:libAthenaServices.so"
   within athena_wrapper.py is not needed in 13.0.0

===============================================================================
25-04-2007 PyJobTransformsCore-00-06-48

   M. Gallas
   -   Empty tag after branching for dev in 13.0.0. The ongoing production with
   12.0.6.5 caches will get PyJobTransformsCore-00-06-47 or PyJobTransformsCore-00-06-47-XX

===============================================================================
===============================================================================
13-04-2007 M. Gallas
        * share/atlas_error_ignore.db: add error masks for streaming test (from Ayana)
          MuonRdoToMuonDigit                       ,  ERROR ElemetID not found
          TgcRdoToTgcPrepData                      ,  ERROR ElemetID not found
        Tagged as PyJobTransformsCore-00-06-47

21-03-2007 M. Gallas
        * share/atlas_error_ignore.db: add TriggerPrep ERROR for HPTV in 12.0.6
        Tagged as PyJobTransformsCore-00-06-46

19-03-2007 Martin Woudstra
	* python/envutil.py: don't use future (python 2.6) keyword 'with' as variable name

15-03-2007 Martin Woudstra
	* share/atlas_error_ignore.db: add LeptonTruthTool ERROR to ignore for 12.0.6
	Tagged as PyJobTransformsCore-00-06-45

12-03-2007 M. Gallas (PyJobTransformsCore-00-06-44)
	* share/atlas_error_ignore.db:
	   - Adding 12.0.6 errors during HPTV (from Kyle)
08-03-2007 Martin Woudstra
	* share/atlas_error_ignore.db:
	   - ignore configurable errors also for 12.0.6
           - fix testing change in one of the lines

06-03-2007 Martin Woudstra / Manuel Gallas
	* python/AtlasErrorCodes.py:
	   - fix bug in getting error/ignore patterns for a release
	   - added special category: OK_CATEGORY
	   - don't load error categories if OK_CATEGORY is the only one needed
	* python/full_trfarg.py:
	   - class JobOptionsArg
	     + allow multiple default packages
	     + add current directory as first 'package'
	   - class JobConfigArg
	     + allow multiple default packages
	     + add current directory and 'JobTransforms' as default packages
	* python/trf.py: some more debug log statements
	* share/atlas_error_ignore.db: ignore 12.0.6 errors in also bugfix nightlies
	* share/atlas_error_caterogies.db: remove category OK (now hardcoded)

28-02-2007 Martin Woudstra
	* python/basic_trfarg.py: InputFile is no longer added to metadata
	* python/trf.py: fix bug in checkInputEvents()

20-02-2007 M. Gallas
        * share/atlas_error_ignore.db: added error for 12.0.6 The same as in 12.0.5
16-02-2007 Martin Woudstra
	* share/atlas_error_ignore.db: added error for 12.0.5

14-02-2007 Martin Woudstra
	* python/trf.py: don't allow 2 arguments with the same name
	* share/atlas_error_ignore.db: added some more errors for 12.0.5

02-02-2007 Martin Woudstra
	* share/atlas_error_ignore.db: added some more errors for 12.0.5

31-01-2007 Martin Woudstra
	* share/atlas_error_ignore.db: be more flexible in spaces for some errors
	       TrigDiMuonFast_1_L2 ERRORs had wrong number of spaces...

30-01-2007 Martin Woudstra
	* share/atlas_error_ignore.db: update errors for Configurables in dev branch to include Py: prefix

29-01-2007  Martin Woudstra
	* share/atlas_error_ignore.db: add MooreToTrack ERROR to ignore for 12.0.5

25-01-2007 Martin Woudstra
	* share/atlas_error_ignore.db: add errors to ignore for 12.0.5 TAG transform

23-01-2007 Martin Woudstra
	* python/basic_trfarg.py: InputTarFileArg: replace option --modification time with -m
	* python/trferr.py: bugfix in TransformErrorHandler

22-01-2007 Martin Woudstra
	* share/atlas_error_ignore.db: added error to ignore for 12.0.5: CBNT_AodTrigger

19-01-2007 Martin Woudstra
	* python/full_trfarg.py: cover cases TriggerConfigArg=NONE and DEFAULT
	* share/atlas_error_categories.db: add category ATH_CON_ORA_SES
	* share/atlas_error_patterns.db: add pattern for ATH_CON_ORA_SES
	* share/atlas_error_ignore.db: add some more errors to ignore for 12.0.5

18-01-2007 Martin Woudstra
	* python/xmlutil.py: minor embellishment in XMLNode printout
	* python/trf.py:
	   - do not print RunDirListing on screen
	   - add cpu cache size to JobReport
	* python/envutil.py: added function cmt_package_version()
	* python/basic_trfarg.py: bugfixes in InputTarFileAndSetupArg
	* share/atlas_error_ignore.db: add a bunch of errors to ignore for 12.0.5

09-01-2007 Martin Woudstra
	* python/JobReport.py: overwrite existing Info if same name
	* python/TransformConfig.py: more consistent comments
	* python/basic_trfarg.py: add --modification-time option when untarring InputTarFile
	* python/trf.py: add run directory info to jobReport
	* share/atlas_error_ignore.db:
	   - added for 12.0.3,12.0.31,12.0.4: IdScan_Muon_L2.IDScanZFinder ERRORs
	   - added for ALL: ToolSvc.Stream WARNING Skipping this one

21-12-2006 Martin Woudstra
	* python/full_trfarg.py: TriggerConfigArg: OFF = no trigger (instead of NONE)
	* share/atlas_error_ignore.py:
	  - add MuGM:MdtReadout ERRORs to ignore
	  - add TrackCollectionCnv ERRORs to ignore
	  - add ToolSvc ERROR unable to finalize the following tools : ToolSvc.CBNT_IndexTool4AA

20-12-2006 Martin Woudstra
	* python/FakeAthena.py: remove object in inheritance tree (added in TransformLogger)
	* python/TransformLogger.py: derive TransformLogger from object
	* python/basic_trfarg.py: bugfixes in class InputTarFileAndSetupArg
	* python/envutil.py: use fileutil for more robust file finding
	* python/fileutil.py:
	   - make all functions more robust against temporary file access problems
	   - added getmtime()
	* python/rfio.py: add function getmtime()
	* python/trf.py: use fileutil for more robust directory listing
	* python/trferr.py: add filename to IOError and OSError (via EnvironmentError)
	* share/atlas_error_ignore.py: added some errors for 12.0.4 reco-with-trigger

19-12-2006 Martin Woudstra
	* python/trfutil.py: added class SANFile
	* python/full_trfarg.py: added arguments OutputSANFileArg and TriggerConfigArg

18-12-2006 Martin Woudstra
	* share/atlas_error_categories.db: added category ATH_G4_STUCK
	* share/atlas_error_patterns.db: added pattern for ATH_G4_STUCK

15-12-2006 Martin Woudstra
	* python/trfutil.py: use glob.glob instead of glob to avoid crash
	Tagged as PyJobTransformsCore-00-06-28

15-12-2006 Martin Woudstra
	* python/FilePreStager.py:
	  - fixed bug in handling of INVALID status
	  - rename NOTSTAGED to ONTAPE
	* python/CastorPreStager.py: adapt to above changes
	* python/envutil.py:
	  - added functions find_file_split and find_files_split
	  - improved find_file(s) functions using the *_split function
	  - fixed bug in find_python_module(s) in handling of selecting .pyc vs .py
	* python/trf.py: add check on required DBRelease number
	* share/atlas_error_ignore.db: added 12.0.4 ToolSvc.SoftMuonLikelihoodMultiDTool ERROR
	Tagged as PyJobTransformsCore-00-06-27

20-11-2006 Martin Woudstra
	* python/JobReport.py: add ErrorSeverity field to jobInfo.xml
	Tagged as PyJobTransformsCore-00-06-26

17-11-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: bugfix in XML printout for ErrorInfo

11-11-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: bugfix for 'ALL' patterns
	* python/FilePreStager.py: add print INFO when files are staged
	* python/JobReport.py: added function errorAcronym()
	* python/fileutil.py:
	  - added function lstat()
	  - added function mode_string()
	  - removed function resolve_symlinks(), because this already exists as os.realpath()
	* python/rfio.py: added function lstat()
	* python/trf.py:
	  - move from fileutil.resolve_symlinks() to os.realpath()
	  - start using class Tee for logfile (for automatic echo to screen)
	  - print out workdir recursive file list in case of failure
	  - remove empty lines from stderr logfile (no idea why they are there at all)
	* python/trferr.py: AthenaLogChecker: set some logging from info to debu
	* python/trfutil.py:
	  - move from fileutil.resolve_symlinks() to os.realpath()
	  - added class RootTTreeFile
	  - started working on eventcount using root
	  - added class RootTTreeFile
	  - get_atlas_release(): add nightly builds on build machines
	* share/atlas_error_categories.db: added category ATH_ELINK
	* share/atlas_error_ignore.db:
	   - added AtheneRefIOHandles ElementLink ERROR for 12.0.31
	   - generalised ElementLink ERROR toPersistent for 12.0.31
	   - ignore configurable JobOptions ERRORs for 12.3.0
	* share/atlas_error_patterns.db: added patterns for ATH_ELINK

06-11-2006 Martin Woudstra
	* share/atlas_error_ignore.db: ignore for 12.0.3/31: DRDR CaloCellContainer ERROR
	* python/trfutil.py: added class TAGFile
	* python/full_trfarg.py: added arguments InputTAGFileArg, OutputTAGFileArg
	* python/trferr.py: bugfix in handling of RunTime exception

27-10-2006 (same day) Martin Woudstra
	* share/atlas_error_patterns.db: fix in GBB patterns (were not caught anymore)
	* share/atlas_error_ignore.db: generalised LArG4HitManagement ERROR
	Tagged as PyJobTransformsCore-00-06-22 (for use with 12.0.31.3)

27-10-2006 Martin Woudstra
	* General: read in error definitions as late as possible
	* python/FakeAthena.py: bugfix in class Include
	* python/basic_trfarg.py:
	  - replace python module tarfile with direct use of tar
	  - minor change in help printout
	* python/trferr.py:
	  - bugfix in TransformErrorHandler
	  - remove all explicit loading of error definitions
	* python/AtlasErrorCodes.py:
	  - minor change in printout of ErrorCategory and ErrorCategoryCount
	  - bugfix in counting of number of ErrorCategories
	  - bugfix in use of ignore error lists
	  - read in error categories on-demand
	  - clear old definitions if new ones are read in (optional,default=True)
	  - switch back on printout of read defitions
	* python/athena_wrapper.py: replace errorcode by erroracronym in ErrorInfo's.
	* python/JobReport.py:
	  - store errorcategories per acronym instead of per code
	  - various python syntax prettifications
	* python/trf.py: replace errorcode by erroracronym in ErrorInfo's.
	* share/atlas_error_categories.db: added ATH_JOP_ERROR, ATH_EXC, ATH_EXC_PYT, ATH_EXC_CXX
	* share/atlas_error_patterns.db: added several ATH_JOP_ERROR patterns for Configurable errors.
	* share/atlas_error_ignore.db:
	   - for 12.0.3 and 12.0.31: ignore ToolSvc.CscCalibTool ERROR
	   - for rel_0-6 and dev_0-6: ignore a bunch of Configurable errors
	Tagged as PyJobTransformsCore-00-06-21

23-10-2006 Martin Woudstra
	* share/atlas_error_ignore.db: for 12.0.3 and 12.0.31: ignore LArWheelSolid ERROR

20-10-2006 Martin Woudstra
	* share/atlas_error_ignore.db: for 12.0.3 and 12.0.31:
	   - added 3 ERRORs to ignore for KalmanVertexTrackUpdator
	   - activated 2 LAr ERRORs to ignore also for 12.0.31
	   - added ERROR to ignore for EFemtrackmatch
	   - added ERROR to ignore CounterTool not found
	Tagged as PyJobTransforms-00-06-19 (for use with 12.0.31.2)

20-10-2006 Martin Woudstra
	* share/atlas_error_ignore.db: as used in 12.0.31.1
	Tagged as PyJobTransforms-00-06-18 (for use with 12.0.31.1)

10-10-2006 Martin Woudstra
	* cmt/requirements: add private/end_private in pattern declare_jobtransforms to speed up cmt
	* python/basic_trfarg.py: improve unpacking of tarball input argument

09-10-2006 Martin Woudstra
	* python/FilePreStager.py: fix bug
	* python/basic_trfarg.py:
	   - more severe error checking and more debug printout in unpacking tarballs
	   - move from Error code numbers to acronyms
	* python/fileutil.py: remove stripping of rfio:
	* python/rfio.py: : do stripping of rfio: inside rf* functions
	* python/full_trfarg.py: use new error acronym: TRF_DBREL_TARFILE
	* python/trf.py:
	   - add os.sep to patterns of environment variables to ignore
	   - earlier checking of availability of top joboptions
	   - remove cleaning up of ROOT stderr message (moved to trferr.py)
	* python/trferr.py:
	   - TransformErrorDiagnoser:
	      + keep flexible list of external diagnoser functions
	      + more extensive and consistent looking for missing libraries
	      + extraction of stderr and stdout from message no longer needed
	* share/atlas_error_catergories.db: added: TRF_DBREL_TARFILE,ATH_OUTFILE,ATH_OUTFILE_TOOLARGE
	* share/atlas_error_ignore.db: added 2 LArG4HitManagement ERRORs to ignore for 12.0.3
	* share/atlas_error_patterns.db: added pattern for ATH_OUTFILE_TOOLARGE
	Tagged as  PyJobTransforms-00-06-15

28-09-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: matchErrorPattern(): set 'who' to None if only whitespace
	* python/trferr.py: catch another string indicating a missing library
	* python/fileutil.py: added class Tee (like unix tee: write to file and dump to screen)
	* share/atlas_error_patterns.db: added pattern for ATH_DLL_LOAD

27-09-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: ErrorInfo: add member acronym (was used: bug!)
	* python/trferr.py:
	   - TransformError classes:
	      + accept any named arguments as extra info
	      + change argument from error number to error acronym
	* python/trfutil.py:
	   - write eventcount stdout and stderr to files (for backup)
	   - assign stdout and stderr separately to error report
	* python/trf.py: adapt to new ErrorInfo and TransformError's
	* share/atlas_error_categories.db:
	   - add categories TRF_KEY_INTERRUPT,ATH_SITE_LIBS,ATH_FAILURE
	   - TRF_CBNTAUDIT: change code from 60601 to 60701
	Tagged as PyJobTransforms-00-06-12

27-09-2006 Martin Woudstra
	* python/trferr.py: remove print statement
	Tagged as PyJobTransforms-00-06-11

27-09-2006 Martin Woudstra
	* cmt/requirements: install all find_*.py utility scripts
	* python/AtlasErrorCodes.py:
	   - change ErrorInfo.exitstatus into 2-tuple (exitcode,interpretation)
	   - ErrorInfo.defaultBacktrace(): remove AthenaCommon/Include.py entries from stack trace
	* python/envutil.py: bugfix in examine_library()
	* python/trf.py: change to new exitstatus (2-tuple)
	* python/trferr.py:
	   - bugfix in TransformErrorDiagnoser.diagnoseSharedLibError()
	   - extend list of system libraries
	* python/trfutil.py:
	   - FileType.eventCount(): keep 20 last lines of logfile, add exitstatus 2-tuple
	* share/find_library.py: add __doc__
	* share/find_data.py: new utility script to find files in DATAPATH
	Tagged as PyJobTransforms-00-06-10

22-09-2006 Martin Woudstra
	* python/trfutil.py, trf.py, AtlasErrorCodes.py: add some more error diagnostics

21-09-2006 Martin Woudstra
	* python/basic_trfarg.py:
	   - nicer interface: add and use member function InputTarFileAndSetupArg.untarAndSetup()
	   - move member function checkFile() up in inheritance structure to class FileArg
	* python/full_trfarg.py: nicer interface: add and use member function DBReleasArg.untarAndSetup()
	* python/trfutil.py: added class VersionString and function get_atlas_release()
	* python/JobReport.py: add member runStage
	* python/trf.py:
	   - more refined re-running after athena crash.
	   - use trfutil.get_atlas_release()
	* share/checklog.py: \
	   - added running of ErrorDiagnoser for additional error analysis
	   - add jobReport of each file as subReport (for individual counting)

20-09-2006 Martin Woudstra
	* python/basic_trfarg.py:
	   - bugfix in DataFileArg.decodeFilenameFields() (not yet used)
	   - added class InputTarFileAndSetupArg from full_trfarg.py
	   - use new baseclass FileArg in *File*Arg arguments
	* python/full_trfarg.py:
	   - moved class InputTarFileAndSetupArg to basic_trfarg.py
	   - rename class InputDBReleaseArg to DBReleaseArg
	   - Get class DBReleasArg to work
	* python/trf.py: adapt for correct handling of DBRelease
	* python/trfutil.py:
	   - SQLiteSupport: protect against removing symlinked directories
	* python/envutil.py: source_setup: revert back: do source from wherever it is called

19-09-2006 Martin Woudstra
	* share/atlas_error_categories.db: added TRF_ARG_MAXEVT_TOOFEW and TRF_ARG_MAXEVT_TOOMANY
	* python/trf.py:
	   - turn OptionError exception into a TransformArgumentError exception
	   - check on maximum number of events at start of job
	   - check min/max #events with original maxEvents (uncorrected for efficiency)
	   - min/max #events failure: exit with new error acronyms (see above)
	* python/full_trfarg.py: added classes InputTarFileAndSetupArg and InputDBReleaseArg
	* python/basic_trfarg.py: class InputTarFileArg: added member function destinationDir()
	* python/envutil.py: source_setup: do source in directory of script

18-09-2006 Martin Woudstra
	* python/fileutil.py: exists_suffix() protect against non-existing directory
	* python/trfutil.py:
	   - SQLiteSupport(): delete existing sqlite files and only copy files if DBRELEASE is set
	   - get_files(): add possibility to symlink if files are in a subdir of workdir, and copy otherwise
	* python/trf.py:
	   - writeLastCommand(): write out only re-run of current (sub-)transform
	   - make filename and name determination more robust in case of nested transforms
	* share/atlas_error_ignore.db: added error to ignore for 12.0.3

15-09-2006 Martin Woudstra
	* python/trf.py, trferr.py, trfenv.py: fix inconsistent naming: TransformEnvironmentError <-> TransformEnvError

05-09-2006 Martin Woudstra
	* python/trfutil.py: improve handling of type and contents in filenames
	* python/basic_trfarg.py: improve handling of type and contents in filenames
	* python/envutil.py: find_libraries() now includes *.so.<version_numbers>

01-09-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: new members for ErrorInfo: exitstatus,stdout,stderr
	* python/envutil.py: new function: examine_library()
	* python/trferr.py: implement class TransformErrorDiagnoser to improve error diagnosis
	* python/trfutil.py:
	   - improve error reporting in case of athena crash during event counting
	   - empty stderr buffer to avoid hanging of POOL utility FClistGUID
	* python/trf.py:
	   - add environment printout to logfile
	   - don't write stdout logfile when athena is re-run after crash
	   - use new feature: TransformErrorDiagnoser
	* share/atlas_error_categories.db: added: TRF_GEO, TRF_GEO_MISMATCH, ATH_SITE, ATH_SITE_SYSLIBS
	* share/atlas_error_patterns.db: added pattern for TRF_GEO_MISMATCH

	31-08-2006 Martin Woudstra
	* python/trfutil.py:
	   - FileType.eventCount: replace grep with python code
	   - add utility function tail() (as unix tail command)
	   - improve error messaging in case of failure of event count
	* python/trf.py:
	   - call athena using subprocess for better control of stdout/stderr.
	   - improve error messaging in case of athena crash by re-running with options -s -l DEBUG

29-08-2006 Martin Woudstra
	* python/JobReport.py: add writing of ErrorAcronym,ErrorDescription,ErrorMessage to jobInfo.xml
	* share/atlas_error_ignore.db: remove (uncomment) some errors fixed in Digitization-00-08-28

28-08-2006 Martin Woudstra
	* python/full_trfarg.py:
	   - JobOptionsArg:
	     + treat empty string ('') as 'NONE'
	     + change from singe file to list
	   - JobConfigArg: add configuration object, and use in exeJobOptions()
	* python/basic_trfarg.py: input/output file classes:  treat empty string ('') as 'NONE'
	* python/TransformConfig.py: added property maxeventsstrategy
	* python/trf.py:
	   - move to using maxeventsstrategy from JobConfig
	   - default config argument now TransformConfig("defaultConfig") (was None)
	   - set jobConfig object in any JobConfigArg arguments.
	   - don't rescale maxEvents for efficiency if maxEvents <= 0

25-08-2006 Martin Woudstra
	* python/trfutil.py:
	   - move to subprocess everywhere and empty stderr buffer where needed to avoid hanging
	   - add logger to FileType.eventCount()
	* python/basic_trfarg.py: added member functions getGUID() and fileSize() to datafile classes.
	* python/trf.py: make maxEvents vs input event count strategy configurable
	* test/eventCount_test.py: new! test the event counting.

24-08-2006 Martin Woudstra
	* python/trfutil.py: start using subprocess for event counting
	* python/trfenv.py: require python version 2.4.0 because of module subprocess
	* python/trferrpy: add running processes diagnosis info for Gribb timeout error
	* python/trf.py:
	   - improve determination of release number
	   - turn too few input events from raising exception to warning message
	* python/basic_trfarg.py: change type() and metaType() to what is used in ProdSys
	* python/full_trfarg.py : change type() and metaType() to what is used in ProdSys
	* share/atlas_error_patterns.db: added AcerMc no-more-events error

23-08-2006 Martin Woudstra
	* share/atlas_error_ignore.db: added errors to ignore in 12.0.2
	* python/trferr.py:
	   - fixed bug in TransformErrorHandler
	   - started class TransformErrorDocter
	* python/runargs.py: allow negative numbers as argument values (if not an option)

22-08-2006 Martin Woudstra
	* share/atlas_error_ignore.db: added errors to ignore in 12.0.2
	* python/basic_trfarg.py:
	   - added timing printout for event counting
	   - added class DataFileArg to reorganise inheritance tree of input/output files

21-08-2006 Martin Woudstra
	* python/TransformConfig.py:
	   - allow None as value for all Descriptors
	   - class JobConfig:
	      + rename 'descriptors' to 'properties'
	      + replace __getitem__ by getProperty and __iter__
	      + add function hasProperty
	* python/basic_trfarg.py: add eventCount to InputFileArg and OutputFileArg
	* python/runargs.py: add __str__ (help printout) to class RunOptions
	* python/trf.py:
	   - use RunOptions help string
	   - add event counting and checking to input files
	* python/trfutil.py:
	   - PoolDataFile: support event counting over list of files
	* share/atlas_error_ignore.db: add ERROR to ignore for 12.0.2

18-08-2006 Martin Woudstra
	* python/TransformConfig.py: add list of descriptors, add metadata, add maxeventsfactor
	* python/runargs.py: implement options parsing from command line argument
	* python/trf.py:
	   - use new options parsing from runargs.RunOptions
	   - add processing of TransformConfig object (minevents,maxeventsfactor,auxfiles,efficiency)
	     These were formerly in EvgenJobTransforms/scripts/csc_evgen_trf.py
17-08-2006 Martin Woudstra
	* python/CVSutil.py: new file with utility for parsing cvs keywords
	* python/TransformConfig.py: rewritten all classes to fix bug in design
	* python/runargs.py: adapted to new TransformConfig.py
	* python/TransformLogger.py: add messageLevels conversion mapping (from trf.py)
	* python/trf.py: adapted to new runargs.py, and remove messageLevel mappings
	Various files: use new CVSutil utility
16-08-2006 Martin Woudstra
	* python/JobReport.py: add notOptions to dumping
	* python/trf.py: use notOptions of JobReport
11-08-2006 Martin Woudstra
	* python/TransformConfig.py: split checking of type and value to fix bug in class ListOfStrings.
	* python/runargs.py: adapt CommandLineOption to changes in TransformConfig
	* test/TransformConfig_test.py: added test

10-08-2006 Martin Woudstra
	* Several files: finised work for adding options as optional arguments
	* python/trf.py
	    - options are now passed on to sub-transforms
	    - be more informative on missing arguments
	    - some options (loglevel and athenaopts) are also accepted as named arguments (without --)
	* python/TransformConfig.py: various improvements and simplifications
	* python/AtlasErrorCodes.py: known error pattern matching: go from search to match
	* python/trferr.py: class AthenaCheckLog: print pattern that matched error, change printing levels
	* python/runargs.py: added classes CommandLineOption and RunOptions

09-08-2006 Martin Woudstra
	* Several files: more work for adding options as optional arguments
08-08-2006 Martin Woudstra
	* Several files: more work for adding options as optional arguments
07-08-2006 Martin Woudstra
	* Several files: start work for adding options as optional arguments
04-08-2006 Martin Woudstra
	* Several files: write out short version in inputFileList in metadata.
03-08-2006 Martin Woudstra
	* python/FilePreStager.py: Fixed bug in class FileStagerRobot in case not all files are staged
	* python/full_trfarg.py: fixed bug in JobOptionsArg to handle 'NONE' correctly with package
	* python/trf.py: add comment line before MessageSvc.OutputLevel override
	* share/atlas_error_ignore.db: ignore all ERRORs for 12.0.1 to get reconstruction succeed
02-08-2006 Martin Woudstra
	* python/TransformConfig.py:
	    - bugfixes in handling of exceptions in Descriptors
	    - make independent of other python modules by adding a backup class TransformConfigError
	* python/JobReport.py: fix bugs in counting of errors
28-07-2006 Martin Woudstra
	* python/full_trfarg.py: add MaxEvents to metadata
27-07-2006 Martin Woudstra
	* python/JobReport.py: support different printout levels
	* python/fileutil.py: added function: resolve_symlinks()
	* python/trf.py:
	    - fixed bug in determination of version from cmt/version.cmt
	    - use new JobReport print levels to print out info at various places
	    - change message levels to strings
	    - replace option -m,--msglevel with -l,--loglevel for consistency with athena.
	* python/trfutil.py: set default msgLevel to 'INFO'
26-07-2006 Martin Woudstra
	* python/AtlasErrorCodes.py: class ErrorInfo: removed data members 'program' and 'command', added 'producer'.
	* python/JobReport.py:
	    - add producer to all error info's.
	    - replace indexing by 'program' by indexing by 'producer'
	    - have separate numbers for maximum number of fatals,errors and warnings to keep.
	      Currently: keep 10 fatals, 10 errors, 0 warnings (all per producer)
	* python/trf.py:
	    - do imports of FilePreStagers only when needed
	    - raise exception if not all files could be pre-staged
	    - increase GriBB CPU time limit from 48 to 72 hours.
	    - add option to switch off printing of jobReport (useful for nested trf's)
	    - set runOpts.msgLevel
	* python/trfutil.py: add member msgLevel to class RunOptions
25-07-2006 Martin WOudstra
	* python/athena_wrapper.py:
	    - solved LD_PRELOAD issue for 64 bit machines (still using a hack)
	    - some cleanup of dependencies
	    - added catching of a string raised as an exception
	* python/trfenv.py: cleanup to solve LD_PRELOAD issue for 64 bit machines
	* python/trf.py:
	    - added use of FileStagerRobot to pre-stage castor files
	    - add list of input files
	* python/trfutil.py: make get_files() more robust for symbolic links
	* python/trferr.py: class TransformErrorHandler: add handling of SyntaxError exception
	* python/AtlasErrorCodes.py: getCode and getAcronym load error categories on demand
24-07-2006 Martin Woudstra
	* python/FilePreStager.py:
	    - finished classes FileStagerRobot and FilePreStager
	    - added global variable theFileStagerRobot
	* test/CastorPreStager_test.py: adapt to updated FilePreStager classes
	* python/TransformLogger.py: added print*() functions (for multiline prints)
	* python/CastorPreStager.py:
	     - adapted to new FilePreStager
	     - added global variable theCastorPreStager
	* python/athena_wrapper.py: try solving 64 bit machine issues
	* python/envutil.py: change search order for python module name extensions
21-07-2006 Martin Woudstra
	* python/trf.py: make athena script (called by gbb) executable
	* python/FakeAthena.py: write error message for deprecated use of theApp.setup()
	* share/atlas_error_categories.db: added category TRF_GBB_NOTSTART
	* share/atlas_error_patterns.db: added pattern for TRF_GBB_NOTSTART

20-07-2006 Martin Woudstra
	* python/castor.py: removed (and replaced by CastorPreStager.py)
	* python/CastorPreStager.py: name says it all
	* python/FilePreStager.py: base classes for file pre-staging
	* python/CastorPreStager_test.py: name says it all (replaces test_castor.py)
	* share/atlas_error_ignore.db: added LArG4HitManagement ERROR for 12.0.1

19-07-2006 Martin Woudstra
	* python/xmlutil.py: bugfix in XMLNode.__eq__()
	* python/AtlasErrorCodes.py: class ErrorInfo: set default severity to FATAL for non-zero error codes.
	* python/trfenv.py:
	    - better printout if wrong python version
	    - raise exception in setup_athena_runtime() if libraries in LD_PRELOAD are not found
	* python/trf.py:
	    - enhance environment info gathering
	    - set error code to TRF_ATHENACRASH in case athena jobReport is missing.
	* share/atlas_error_categories.db: added error code 10430 ATH_DLL_PRELOAD

18-07-2006 Martin Woudstra
	* python/trf.py: continue work on more environment info to the jobReport
	* python/xmlutil.py: allow list/tuple with strings as contents of XMLNode
	* python/*_trfarg.py: add __nonzero__() functions for boolean testing of arguments
14-07-2006 Martin Woudstra
	* python/xmlutil.py: added printout (__str__) to XMLNode
	* python/JobReport.py: modified printout (__str__) of JobInfo and FileInfo
	* python/trf.py: add some more environment info to the jobReport
	* python/castor.py: new! Utilities for pre-staging of castor files
13-07-2006 Martin Woudstra
	* python/basic_trfarg.py: improve error message for OutputFileArg
	* python/trferr.py: bugfix: added import trfconsts
12-07-2006 Martin Woudstra
	* cmt/requirements: fully wildcard use AtlasPolicy
	* python/trf.py:
	    - added the feature to set athena options (needed for pileup)
	    - added command line option -a,--athenaopts
	* python/basic_trfarg.py: classes InputFileArg,OutputFileArg: convert 'None' argument value to upper case
	* python/full_trfarg.py: class JobOptionsArg: convert 'None' argument value to upper case
	* python/JobRep
	ort.py: suppress some info in error printout
11-07-2006 Martin Woudstra
	* python/full_trfarg.py: Fixed bug in JobOptionsArg.defaultHelp() (wrong help for derived class)
11-07-2006 Alessandro De Salvo
	* share/atlas_error_ignore.db: new sim error to ignore
10-07-2006(later same day) Martin Woudstra
	* python/trf.py: adding scanning of logfile for metadata
10-07-2006 Martin Woudstra
	* python/trf.py: bugfix: move #events check from pre/postRunActions() to doJob(),
	                 otherwise the preRun check is not executed by a trf that sets
	                 the min/max values in preRunActions() (e.g. evgen)
07-07-2006 Martin Woudstra
	* python/basic_trfarg.py: modified handling of metadata
	* python/full_trfarg.py:  modified handling of metadata
	* python/trf.py:
	    - adapt to modified argument metadata
	    - preRunActions() and postRunActions() called from within runJob()
	      (so they are no longer called in composite transforms)
06-07-2006 Martin WOudstra
	* python/*_trfarg.py: added property isMetaData.
	* python/TransformLogger.py: new file containing class TransformLogger
	* python/trfutil.py: removed class TransformLogger (moved to it's own file)
	* python/trferr.py: TransformErrorHandler now uses TransformLogger
	* python/trf.py:
	    - move to using TransformLogger consistently
	    - adding arguments to metadata if they are declared as such
	* python/JobReport.py: fixed bug in counting of #errors,warnings,fatals in addReport()
04-07-2006 Martin Woudstra
	* python/envutil.py:
	    - fixed bug in find_python_module() (calls a private function now)
	    - added find_python_modules()
	* share/find_python_module.py: new! executable utility to find python modules
	* share/trf_ls: new! executable utility to find jobtransforms
	* python/trf.py: some tuning of GriBB limits
03-07-2006 Martin Woudstra
	* python/trf.py: added use of GriBB in running athena
	* share/gbb: new utility for process monitoring (from Alessandro de Salvo)
	* share/checklog.py: added option -r <atlas_release>
	* share/atlas_error*.db:
	    - added separate field for <who_prints_the_error>, to allow for variable number
	      of whitespaces between <who_prints_it> and <error_message_regexp>
	    - added error categories and patterns for StoreGate access errors.
	* python/AtlasErrorCodes.py:
	    - adapted to new format (extra field) of db files
	    - added class ErrorInfo (was in module JobReport before)
	    - more intelligent error pattern matching: use the known 'who'
	* python/JobReport.py: removed class ErrorInfo (moved to module AtlasErrorCodes.py)
30-06-2006 Martin Woudstra
	* python/trfutil.py: SQLiteSupport copies all *_sqlite files from geomDB
29-06-2006 Martin Woudstra
	* python/trf.py:
	   - add config argument to __init__
	   - add metadata of config object to output files
	   - change default MessageSvc.OutputLevel to INFO
	* share/atlas_error_ignore.db: added patterns for 12.0.1
28-06-2006 Martin Woudstra
	* python/FakeAthena.py:  add SystemOfUnits and PhysicalConstants (from AthenaCommon) to runtime environment
	* python/full_trfarg.py: JobConfigArg now derives from JobOptions
	* python/trferr.py:
	    - AthenaLogChecker: require exactly one word at the beginning of the line before WARNING|ERROR|FATAL
	      in order to be considered for error checking
	* python/TransformConfig: added metadata to class TransformConfig
27-06-2006 Martin Woudstra
	* python/FakeAthena.py: class FakeAppMgr no longer derives from AthAppMgr (no side-effects)
	* python/envutil.py: find_python_module(): convert python module syntax (.) into filename syntax (/)
	* python/full_trfarg.py: added class JobConfigArg
	* share/atlas_error_ignore.db: added pattern for 12.0.1 (StoreGateSvc ERROR retrieve(default))
26-06-2006 Martin Woudstra
	* python/basic_trfarg.py: No longer copy PoolFileCatalog.xml from POOLFILE (was not used)
	* python/trf.py: exit with error if DBRelease not setup for STANDALONE, or is not requested version
	* python/TransformConfig.py: new! base classes to hold the configuration flags for the transform
	* python/trferr.py: added class TransformConfigError and error code TRF_CONFIG_ERROR
	* share/atlas_error_categories.db: added categories TRF_DBREL*, TRF_CONFIG
23-06-2006 Martin Woudstra
	* python/envutil.py: added function find_libraries()
	* share/find_library.py: new executable utility to find shared libraries in LD_LIBRARY_PATH
	* share/atlas_error_categories.db: added category ATH_CON_MYSQL_GONE
	* share/atlas_error_patterns.db: added pattern for ATH_CON_MYSQL_GONE
22-06-2006 Martin Woudstra
	* python/trfutil.py:
	    - add argument errorIfNotFound to get_files and GetFiles
	    - SQLiteSupport only requires sqlite files to be found if CMTSITE=STANDALONE, to allow nightly tests.
	* python/trf.py:
	    - added override setting of MessageSvc.OutputLevel at end of top jo's
	    - added option -m,--msglevel
	* share/atlas_error_ignore.db:
	    - added  ERROR (poolDb): (False error message if PoolFileCatalog.xml is missing)
	    - added ^\s*FATAL ERROR\s*$ (printed by McAtNlo if event fails)
	* share/atlas_error_patterns.db: removed ERROR (poolDb)
	* share/atlas_error_categories.db: removed category 10700,ATH_POO_NOCAT,Missing PoolFileCatalog.xml
20-06-2006 Martin Woudstra
	* python/trfutil.py: class SQLiteSupport: get files from DATAPATH (to work around a bug in CORAL)
	* python/full_trfarg.py: updated execution of jobOptions file of class JobOptionsArg
	* python/basic_trfarg.py: BoolArg now only accepts 'True' or 'False' (case insensitive)
	* python/FakeAthena.py: Bugfix to capture full contents of jobOptions files (including nested ones)
16-06-2006 Martin Woudstra
	* python/envutil.py: added function has_wildcards()
	* python/trfutil.py:
	    - get_files() (and GetFiles()) now include creation of local subdir
	    - SQLiteSupport(): now only copying a number of files from SQLITE_FILE_PATH
	    - removed class SQLiteGeomSupport (superseded by SQLiteSupport())
	    - removed class RDBAccessOverride (no longer needed with CORAL)
	* share/atlas_error_categories.db: added new error category (SQLite file lock problem)
	* share/atlas_error_patterns.db: added pattern for above new error category
15-06-2006 Martin Woudstra
	* python/FakeAthena.py: new file to be able to 'execute' jobOptions files
	* python/full_trf.py: updated support for parsing the jobOptions file
	* python/basic_trf.py: class InputTarFileArg: added protection against NONE
	* share/atlas_error_categories.db: added new error category (missing PoolFileCatalog)
	* share/atlas_error_patterns.db: added pattern for above new error category
	* trf.py: fixed bug in JobTransform.setMinMaxEvents()
13-06-2006 Martin Woudstra
	* python/full_trf.py: class JobOptionsArg: added support for parsing the jobOptions file
	* python/basic_trf.py: class InputTarFileArg: merged member functions filelist() and filelistRE()
	* python/trfutil.py: added depth argument to function find_joboptions()
	* share/atlas_error_ignore.db: add 1 errors to ignore for release 12.0.0
	* share/atlas_error_categories.db: minor change to one error pattern
12-06-2006 Martin Woudstra
	* share/atlas_error_ignore.db: add 2 errors to ignore for release 12.0.0
	* python/trf.py: added determination of database release
09-06-2006 Martin Woudstra
	* python/trf.py:
	    - added support for minimum/maximum number of events in output files
	    - added default version: the cvs tag (as found in file cmt/version.cmt)
08-06-2006 Martin Woudstra
	* python/trferr.py: bugfix in AthenaLogChecker
	* python/trfutil.py: bugfix in GetFiles()
	* python/JobReport.py: new file gathering things related to making a jobreport
	     - class JobReport:
	          + removed producer argument from constructor (otherwise it gets lost by pickle)
		  + added producerVersion
06-06-2006 Martin Woudstra
	* python/basic_trfarg.py: removed "List" suffix from InputFileListArg.defaultName()
	* python/full_trfarg.py: added class InputGeneratorFileArg
	* python/envutil.py: find_files(): fixed bug and made it faster for non-wildcarded files
	* python/trfutil.py: add depth argument to get_files() and class GetFiles()
	* python/trf.py:
	     - generate runArgsArgument templates at the last moment
	     - add determination of Atlas Release from file directory is not given by ATLAS_RELEASE
02-06-2006 Martin Woudstra
	* share/atlas_error_ignore.db:
	    - prefixed all ( and ) with a \ to make them litteral: Tagged with PyJobTransforms-00-02-08
	    - cleaned up, and starting afresh for release 12.0.0: Tagged with  PyJobTransforms-00-02-09
	* share/checklog.py: new executable script to do standalon athena logfile checking
	* python/trferr.py: fixed bugs in class AthenaLogChecker()
	* python/basic_trfarg.py:
	    - *FileArg classes: added defaults for 'name' in constructors, and therefore
	      had to invert order of 'name' and 'type'.
	* python/full_trfarg.py:
	    - adapt to changes in basic_trfarg.py. Remove 'type' from constructors of *FileArg classes
	    - added class GeometryVersionArg
	* python/envutil.py:
	    - find_files() bugfix: avoid finding multiple copies of files
	    - find_file_env() and find_files_env() now split path with os.pathsep (:) or comma (,)
	* test/basic_trfarg_test.py: adapted to changes in basic_trfarg.py
	* test/full_trfarg_test.py: adapted to changes in full_trfarg.py
01-06-2006 Martin Woudstra
	* python/trf.py: added RunOptions
	* python/xmlutil.py: fixed weakness in default values for arguments
	* python/trfutil.py:
	    - class GetFiles(): added handling of wildcards
	    - new function get_files() a-la get_files command
	* python/trfenv.py: added functions find_files() and find_files_env()
	* python/full_trfarg.py:
	    - renamed class OutputHistogramFileArg to HistogramFileArg
	    - renamed class OutputNtupleFileArg to NtupleFileArg
	* test/argstest_trf.py: adapted to changes in full_trfarg.py
	* share/atlas_error_categories.db: added 4 more TRF_*FILE_* errors
	* share/atlas_error_patterns.db: removed 2 entries containing a |
31-05-2006 Martin Woudstra
	* python/trfutil.py:
	    - fixed bug in JobReport.writeMetadataXML()
	    - implemented real eventcount in class PoolDataFile
	    - added EvgenFile metadata: randomseed, joboptions
	* python/trf.py: added support for trf arguments metadata
29-05-2006 Martin Woudstra
	* python/trfutil.py: added class GetFiles
19-05-2006 Martin Woudstra
	* share/atlas_error_ignore.db: added new error to ignore
	* python/trf.py: added printout of number of errors,warnings,fatals found in logfile.
	* python/trferr.py: Fixed bugs in AthenaLogChecker
17-05-2006 Martin Woudstra
	Added logfile parsing for errors,warnings,fatals, error patterns & ignore patterns
	* share/*.db: New error (ascii) db files
16-05-2006 Martin Woudstra
	* python/trf.py:
	    - new member function runJob() doing the actual job (athena or sub-trfs)
	    - moved code from execute() to doPreRunActions(), doPostRunActions(), and runJob()
	      for easier implementation of composite trf's (overriding runJob())
	* python/trfutil.py:
	    - SQLiteSupport: do not touch DATAPATH, and do not delete files after job
15-05-2006 Martin Woudstra
	* python/envutil.py:
	    - find_file(): added test to skip non-existing directories
	    - _get_sys_path_extras(): make sure I use current python version
12-05-2006 Martin Woudstra
	Many changes to write out jobInfo.xml
10-05-2006 Martin Woudstra
	Several python files adapted to generate metadata.xml file. Many changes.
9-05-2006 Martin Woudstra
	* python/envutil.py:
	    - bugfix in updating of sys.path in function update_env()
	    - added support for negative depth in find_file(): search upwards in directory tree
	* python/trfutil.py:
	    - renamed Filename to FileType, and removed some member functions
	    - *File classes inherit from FileType
	    - new class FileList to decode file list given in a string.
	      Takes over the functionality of class FileListArg of basic_trfarg.py
	* python/basic_trfarg.py:
	    - removed class FileListArg (replaced by class FileList above)
	    - class InputFileListArg now inherits from InputFileArg
	    - several classes: replaced inheritance from Filename by aggregration of a FileType
	* python/full_trfarg.py: type+contents now captured in a FileType derived object
	* test/basic_trfarg_test.py: replaced class TestFileListArg by class TestInputFileListArg.
8-05-2006 Martin Woudstra
	* python/skeleton.py: removed (old and not used)
	* python/fileutil.py: new! Set of file utilities a-la module os and os.path for use accross several
	  file access systems (local files, rfio files (e.g. castor))
	* python/trfutil.py:
	    - class Filename:
	        + moving static functions to fileutil.py
	        + removing member __filename again
	    - simplified functions strip_suffix() and strip_prefix()
	* python/envutil.py:
	    - added function find_library() (from trfutil.py)
	    - added function find_python_module()
	    - added function find_executable() to replace AthenaCommon.Utils.unixtools.which()
	    - added function find_file() to replace AthenaCommon.Utils.unixtools.FindFile()
	    - added some environment variable names as python 'constants'
	    - improved handling of PYTHONPATH in function update_env()
	* python/rfio.py:
	    - rfstat() and rfdir() now return also the exitstatus
	    - fix bug in RFIOError.__init__()
	    - added functions exists(), access()
5-05-2006 Martin Woudstra
	* python/envutil.py:
	    - improved update_env()
	    - added function find_path_env()
	* python/rfio.py: new! collection rfio file utilities
	* python/trfutil.py:
	    - moving to using new rfio module
	    - starting to change use of class Filename (now has __filename data member)
3-05-2006 Martin Woudstra
	* python/envutil.py: new!
	    - containing the environment utilities of trfutil.py
	    - updated *_env_* functions to treat PYTHONPATH as a special case
            - added functions update_env(), replace_path_env(), getenv_from_output(),
	      source_setup(), setup_cmt(), setup_cmt_package()
	* python/trfutil.py:
	    - moved environment utilities out to envutil.py
	    - made all class data members private (with __ (double underscore) prefix)
	    - class Filename and PoolDataFile: added GUID
	* python/basic_trfarg.py: InputDataFileArg.preRunAction: add setting of POOL_CATALOG env var.
	* test/envutil_test.py: new! contains the tests for envutil.py (taken from trfutil_test.py)
	* test/trfutil_test.py: moved environment tests out to envutil_test.py
27-04-2006 Martin Woudsta
	* python/trfutil.py:
	    - added class TransformLogger, and starting to use it
	    - class SQLiteSupport
	        + preRunAction(): also copy if source on afs (doesn't harm)
	        + postRunAction(): remove copy, and restore DATAPATH
	    - class RDBAccessOverride
	        + postRunAction(): remove temporary joboptions file
	    - class ServiceOverride
	        + postRunAction(): remove temporary joboptions file
24-04-2006 Martin Woudstra
	* python/trfutil.py:
	    - class Filename: bugfix fileSize()
	    - class SQLiteSupport: preRunAction()
	        + added protection for copying a file onto itself (needed in python 2.4.2)
	        + no copy if source is on afs
21-04-2006 Martin Woudstra
	* python/trfutil.py:
	    - class StringNumberList: put '[' and ']' in constants openBracket and closeBracket.
	    - class Filename:
	        + added member functions fileSize(), checkUsability(), rfioItem()
	        + modified existsAttempt() to take highest attempt number
	        + remove() prints warning for files other than local
	* python/basic_trfarg.py:
	    - bugfix in Argument._notImplemented()
	    - StringArg.typeFormat() return 'r' instead of 's'
	    - StringArg.pythonVariableTemplate() no longer needed because of above change
	    - class FileListArg:
	         + bugfix in basicType()
	         + added typeFormat()
	         + added support for a comma separeted list of filenames
	    - classes In/OutputFile*Arg use new function Filename.checkUsability()
	    - class InputDataFileArg now derives from InputFileListArg instead of InputFileArg
	      This changes the behaviour of all Input*FileArg arguments in full_trfarg.py:
	      Now also lists of input files are supported for all those.
	* test/basic_trfarg_test.py: added tests for class FileListArg
19-04-2006 Martin Woudstra
	python/trferr.py:
	    - renamed class ErrorReport to JobReport
	    - added base class JobInfo
	python/trfutil.py:
	    - class Filename: fixed bugs in handling of rfio files
	python/trfenv.py:
	    - split off LD_PRELOAD into separate function: setup_athena_runtime()
	    - new function: cleanup_athena_runtime() to clear LD_PRELOAD
	python/trf.py: use new functions setup_athena_runtime() and cleanup_athena_runtime()
	               and new classname JobReport.
18-04-2006 Martin Woudstra
	python/basic_trfarg.py: fixed bug in return type of StringChoicesArg.setValue()
	python/trf.py:
	    - added member function getArgument(self,name)
	    - simplified argumentValueDict()
31-03-2006 Martin Woudstra
	python/basic_trfarg.py:
	    - class StringChoicesArg: input value is now optionally case-insensitive.
	      If case-insensitive then the value is compared to the possible choices
	      (in case-insensitive way) and changed to the correct the case if a match
	      if found. The value the object receives gets the same case as given in
	      the list of choices. Fixed bug in exception error message.
	      setDefault() checks that default is one of choices.
	    - classes InputFileArg,InputFileListArg,OutputFileArg:
	      switched off the check on the file suffix.
	    - Optional Argument help string: added [] around position name
	python/trf.py:
	    - added support for named command line arguments (name=value)
	    - contents of top joboptions files added to logfile
	    - add(), _addArgument() and addAuthor() now also accept keyword arguments
	    - removing errorReportFile before running
	python/trfutil.py: class Filename added function fileSuffixRE() for printout
	python/athena_wrapper.py: fixed bug: replaced ErrorInfo by trferr.ErrorInfo
30-03-2006 Martin Woudstra
	python/trferr.py:
	    - class ErrorReport:
	      * fixed bug in read()
	      * write() now also writes human readable .txt file
	      * added ExitCode in printout
	    - class ErrorInfo:
	      * changed 'program' to 'command' and set by default
	        to full command line (incl arguments)
	      * fixed bug in default stack trace
	      * pretty printing of stack trace in __str__()
	    - renamed JOBOPTIONS_PYTHON_ERROR to PYTHON_SYNTAX_ERROR
	python/trf.py:
	    - renamed exePre/PostRunActions to doPre/PostRunActions
	    - now using ErrorReport pickle file to communicate errors between
	      athena running in subshell and the transform
	    - print error reports on screen in all exe*() functions.
	python/trfutil.py:
	     - added function remove_filename_suffix()
	     - fixed bug in function find_datafile()
	     - fixed bug in Filename.listdir()
	python/athena_wrapper.py: moved handling of exception SystemExit to errorHandler.
29-03-2006 (later on same day) Martin Woudstra
	cmt/requirements: put dependency on <package>_init_py first
	share/ensure_init.py: protect against non-existing or empty directory
29-03-2006 Martin Woudstra
	cmt/requirements: added action to create __init__.py if not yet present
	share/ensure_init.py: script used to perform above action
	python/basic_trfarg.py: improved default help of StringChoicesArg, and
	                        changed order of arguments to __init__()
24-03-2006 Martin Woudstra
	python/trferr.py:
	    - got classes ErrorInfo and ErrorReport to work
	    - added module global variable: errorReportFile
	    - added handling of exceptions of TransformErrorHandler:
	        TransformError, IncludeError, SystemExit, NameError, AttributeError
	python/trf.py: using ErrorInfo and ErrorReport classes
	python/full_trfarg.py: bugfix: added from PyJobTransformsCore.trferr import *
	python/trfutil.py: fixed some bugs in class Filename
	python.trfconsts.py: added variable PATH
23-03-2006 Martin Woudstra
	python/trfutil.py:
	    - funtion find_library() now tries also with '.so' extension and 'lib' prefix.
	    - function find_in_stack() takes optional argument stackList
	python/trferr.py: exception handlers now return ErrorInfo object
22-03-2006 Martin Woudstra
	python/trferr.py:
	    - added classes ErrorInfo and ErrorReport
	    - added function typeName()
	    - renamed messages to _typeNames
	    - added error severity value constants
	    - TransformError class can now return an ErrorInfo object
	python/trf.py: start using ErrorInfo and ErrorReport. NOT FINISHED!
20-03-2006 Martin Woudstra
	python/trfutil.py: A
	    - debugged functions append_path_env*, prepend_path_env*
	    - fixed bug in function find_datafile
	    - fixed bugs in RDBAccessOverride.preRunAction()
	    - added class ServiceOverride
	    - class Filename: Added supports rfio: file access (e.g. castor).
	                      Removed isRegularFile() member function.
	python/basic_trfarg.py:
	    - default value for help() is now the __doc__ string of the class.
	python/full_trfarg.py:
	    - changed all help to 'default'
	    - JobOptionsArg: added function defaultHelp()
	test/myunittest.py:
	    - TestCase.tearDown(): make removal conditional (in case setUp was not called)
	test/trfutil_test.py:
	    - added tests for functions append_path_env*, prepend_path_env*
	    - moved to using myunittest module
	    - argtest_trf.py: added the new Arguments + SQLiteGeomSupport
17-03-2006 Martin Woudstra
	python/trfutil.py:
	    - added classes RDOFile,ESDFile,AODFile,PreJobOptionsFile,PostJobOptionsFile,
	      RDBAccessOverride,SQLiteSupport,SQLiteGeomSupport
	    - added functions append_path_env*, prepend_path_env*, find_datafile
	python/basic_trfarg.py:
	    - fixed bug in ArgChoices.checkChoices(), made members private
	    - InputDataFile.preRunAction() copies $POOLFILE. Serves as
	      baseclass for full input data args
	    - new class OutputDataFile serves as baseclass of full output data args
	python/full_trfarg.py:
	    - Data file arguments now derive from In/OutDataFileArg base classes
	    - Added classes In/OutputRDOFileArg, In/OutputESDFileArg, In/OutputAODFileArg
	python/trfconsts.py: new! Holds useful constants (were in trfenv.py before)
	python/trf.py:
	    - added support for PreJobOptionsFile and PostJobOptionsFile
	    - error communication between athena & trf now via exitStatusFile
16-03-2006 Martin Woudstra
	python/basic_trfarg.py: added class BoolArg
14-03-2006 Martin Woudstra
	cmt/requirements:
	    - pattern declare_jobtransforms now uses standard paths
	    - expand_files.py script now installed in InstallArea scripts area
	cmt/expand_files.py: moved to ../share
	python/trfutil.py: added class JobOptionsFile
	python/trferr.py:
	    - added class JobOptionsNotFoundError
	    - *FileError classes now take filename as explicit additional argument
	python/trf.py: preparing for adding user defined joboptions
	python/basic_trf.py: fixed a few bugs in InputFileArg.preRunAction()
7-03-2006 Martin Woudstra
	python/trferr.py: added catching of exception KeyboardInterrupt
	python/trf.py:
	    - writing output to logfile
	    - added 'exec' to last.<mytrf> file to avoid re-running problems
	      due to it being overwritten while being run.
	python/athena_wrapper.py:
	    - removal of old commented-out code.
	    - exit with 'UnknownException' if errorHandler does not handle error.
6-03-2006 Martin Woudstra
	Imported everything from PyJobTransforms
	Moved examples to the respective domain packages
	cmt/requirements:
	      - Installing jobtransforms in dedicated area (InstallArea/trfs)
	      -	Adding trfs installarea path to PYTHONPATH,PATH & JOBOPTSEARCHPATH
	test/*.py: adapted to use of PyJobTransformsCore
	python/*.py: adapted to use of PyJobTransformsCore
	python/trf.py: add default name for skeleton joboptions file
3-03-2006 Martin Woudstra
	cmt/expand_files.py: new! Utility to expand filenames from source & destination dir
	cmt/requirements: using new utility expand_files.py to gather all transforms
	cmt/requirements: default directory for finding jobtransforms is ../scripts
2-03-2006 Martin Woudstra
	python/trfutil.py: fixed bug in Author.__str__()
	python/runargs.py: added function RunArguments.__str__()
1-03-2006 Martin Woudstra
	python/trfutil.py: load_module() more rubust
	test/trfutil_test.py: some more tests for load_module() and class StringNumberList
28-02-2006 Martin Woudstra
	test/myunittest.py: now handles temporary files
	test/MyTest.py: renamed to myunittest.py
	test/basic_trfarg.py: now using myunittest.py
	example/dc3_simul_trf.py: new! Jobtransform for simulation (not finished yet)
27-02-2006 Martin Woudstra
	test/*test.py: fixed some bugs
	test/MyTest.py: new! base class for unit testing
	python/basic_trfarg.py: bugfix in InputTarFileArg
	python/trfenv.py: bugfix in setup_runtime()
23-02-2006 Martin Woudstra
	cmt/requirements: added pattern declare_jobtransforms
	python/trfargs: moved all files in this directory up one level. This dir is now obsolete.
	python/*.py: adapted for new location of files that were in subdir trfargs
17-02-2006 Martin Woudstra
	python/trfenv.py: added function setup_runtime() that must be called explicitly,
	                  otherwise os.system() can not do any command without the athena full
	                  runtime environment (I guess because of LD_PRELOAD)
	python/trf.py: calling trfenv.setup_runtime() explicitly before running athena
	python/trfutil.py: - added class StringNumList, which does the FileList decoding.
	                   - class Filename now allows regexp as type,contents and suffix.
	python/trfargs/basic_trfarg.py:
	                   - moved intelligence of FileListArg to class StringNumberList
			   - added class InputTarFileArg
	test/filelist_test.py: removed. Contents moved to trfutil_test.py
	test/load_test.py: removed. Contents moved to trfutil_test.py
	test/trfutil_test.py: new! gathers all unittests for trfutil.py
	test/basic_trfarg_test.py: new! gathers all unittests for basic_trfarg.py
16-02-2006 Martin Woudstra
	test/filelist_test.py: new! Unittest for testing FileListArg
	python/trfargs/basic_trfarg.py: debugged FileListArg.
14-02-2006 Martin Woudstra
	python/trfargs/basic_trfarg.py: added classes FileListArg and InputFileListArg. Not yet tested.
	python.trferr.py: cosmetic change: don't print ExitStatus to screen if status 0 (appeared in help)
10-02-2006 Martin Woudstra
	python/trferr.py: added class TransformErrorHandler to handle errors
	python/athena_wrapper.py: moved error handling to external error handler
	python/trfutil.py: added load_module() and load_errorhandlers() functions
08-02-2006 Martin Woudstra
	python/*.py: improve error reporting in case a DLL is not found
01-02-2006 Martin Woudstra
	python/trf.py: - simplified member function names
	               - made data members private
	               - removed obsolete *Instance* member functions
	               - removed auto-skeleton generation (was not ready anyway)
		       - remove usage of Pre/PostRunCommand classes
	python/trfutil.py:
	               - simplified member function names
	               - made class data members private and changed some names to avoid clashes
		       - removed overkill classes Pre/PostRunCommand
	python/trfxml.py: minor improvements
	python/trfargs/basic_trfarg.py:
	               - simplified member function names
	               - made data members private
		       - changed member function names so type() and metaType() now correspond to the Production System naming scheme
		       - moved fully specified argument classes to module full_trfarg.py
	python/trfargs/full_trfarg.py: New! Contains fully specified argument classes previously in basic_trfarg.py
	python/example/ratxml.py: adapted to changes in other python files
	python/example/dc3_evgen_trf.py: using full_trfarg module instead of basic_trfarg (new scheme)
	test/ : New directory to hold tests
	test/testargs_trf.py: New! First TestJobTransform
31-01-2006 Martin Woudstra
	python/trf.py: Removed option -q xmlsig. Functionality now in external python script (example/ratxml.py)
	python/trfutil.py: added function load_transforms() to load all transformations available in a python file.
	python/trfxml.py: some improvements
	python/trfargs/basic_trfargs.py: added classes ListArg and SkipEventsArg
	example/ratxml.py: New! Takes over the functionality of option -q xmlsig, to move the maintainance of this code
	                   to where it belongs: in the RAT group  (since they use this functionality).
30-01-2006 Martin Woudstra
	python/trf.py: - First working version of -q,--query xmlsig
	python/trfutil.py: added some more file classes. Can specify type,contents and suffix.
	python/athena_wrapper.py: added catching of exception AttributeError
	python/trfxml.py: New! Some xml utilities
	python/trfargs/basic_trfargs.py: added some specific input and output classes. Check on filename suffix.
	example/dc3_evgen_trf.py: using new input/output file classes
	example/skeleton.dc3_evgen.py: adapted to use new variable name of input file.
27-01-2006 Martin Woudstra
	python/trf.py: - removed option -r,--run (obsolete)
		       - added option -q,--query xmlsig: print arguments signature in xml format for production system.
	                 Work in progress...
	python/trfutil.py: started to add some specific file type classes
	python/trferr.py: add default for call to trferr.exit()
26-01-2006 Martin Woudstra
	python/athena_wrapper.py: Added catching of exception NameError
	python/trferr.py: added JOBOPTIONS_PYTHON_ERROR
	python/trf.py: - writing argument values in object 'runArgs' instead of global namespace
	               - parameters filename renamed to runargs.<trfname>.py
		       - write out last full command line in file 'last.<trfname>' for easy repeating
		       - easier to read help output (-h option)
		       - some cleanup of code
	python/trfutil.py: removed classes JobOptionsSnippet and JobOptionsFile
	python/trfargs/runargs.py: New! Holds the RunArgument class
	example/dc3_evgen_trf.py: removed the default values
	example/skeleton.dc3_evgen.py: Adapted to use run arguments from runArgs object
24-01-2006 Martin Woudstra
	python/athena_wrapper.py: Catch exception SystemExit
	python/trf.py: some bug fixes
	python/trferr.py: added class OutputFileError
	python/trfargs/basic_trfarg.py: modified Pre/PostRunActions to return message string
	python/trfutil.py: modified Pre/PostRunActions to return message string
	example/dc3_evgen_trf.py: added defaults for ntuple and histo files (for testing)
	example/skeleton.dc3_evgen.py: Fixed bugs. Runs now. Ntuple file missing, Histo file empty.
23-01-2006 Martin Woudstra
	python/trf.py: improve options handling
	python/trfenv.py: add LD_PRELOAD environment to run athena_wrapper.py
	python/athena_wrapper.py: added catching of "DLL not found" error
	example/skeleton.dc3_evgen.py: new! Added to get something to work

20-01-2006 Martin Woudstra
	cmt/requirements: fix package name and make consistent with project builds
	example: now subclassing event generation example transformation
	python/trf.py: Added 'running athena' part, added exit status values, cleanup
	python/trferr.py: added error codes & messages (values to be changed)
	python/trfutil.py: added find_joboptions
	python/athena_wrapper.py: new! Turn Include exception into exit code
	python/trfenv.py: new! Set up some environment info

24-11-2005 Martin Woudstra
	First import of some real code. Only partly working.
Tip: Filter by directory path e.g. /media app.js to search for public/media/app.js.
Tip: Use camelCasing e.g. ProjME to search for ProjectModifiedEvent.java.
Tip: Filter by extension type e.g. /repo .js to search for all .js files in the /repo directory.
Tip: Separate your search with spaces e.g. /ssh pom.xml to search for src/ssh/pom.xml.
Tip: Use ↑ and ↓ arrow keys to navigate and return to view the file.
Tip: You can also navigate files with Ctrl+j (next) and Ctrl+k (previous) and view the file with Ctrl+o.
Tip: You can also navigate files with Alt+j (next) and Alt+k (previous) and view the file with Alt+o.