-
Notifications
You must be signed in to change notification settings - Fork 13
/
ChangeLog.txt
1978 lines (1455 loc) · 48.2 KB
/
ChangeLog.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
Release Notes
* 5/25/2022
* 0.0.6 for containerization
* 2/18/2020
* using use_only_local_data in AtlasPJB
* 9/25/2019
* not to delete empty output datasets for prod
* to use done instead of failed when input has no files
* 7/9/2019
* added CPU abuse check
* 5/30/2019
* using json SW check in AtlasAJB
* 5/2/2019
* added NQUEUED_NUC_CAP
* 4/2/2019
* max attempts/failures for analysis
* 3/25/2019
* task.architecture=cmtconfig@image
* 3/22/2019
* setting task.currentprio=1500 if s or g in gridpref
* 3/18/2019
* added --priority in panda-client
* removed high prio from ready tasks in getTasksTBP
* 3/15/2019
* to automatically fix empty memory requirements
* 3/11/2019
* using fake in specialHandling for WAITING_CO_JUMBO
* 3/5/2019
* added osMatching
* 2/28/2019
* added useZipToPin
* 2/27/2019
* checking nNewJobs when setting exhausted after scouts
* 2/19/2019
* improved AtlasAJB for GPU PQs to request architecture matching
* 2/13/2019
* added toStaging
* 2/5/2019
* using nClosed in AtlasAJB
* 1/28/2019
* added eshigh
* prio boost for tasks with jumbo
* 1/17/2019
* using nEventsReady for jumbo in AtlasPJB
* not to count throttled jobs in nAct in AtlasAJB
* 1/15/2019
* processingType=blah-cont for container based analysys
* added USER_TASKS_MOVE_INPUT
* 12/21/2018
* to use only MCORE for ES scouts
* 12/19/2018
* to use only MCORE for ES with prio>=900
* 12/11/2018
* disabled data locality check in AtlasAJB for taskPrio>=2000
* 12/3/2018
* disabled IO intensity check when no candidate available permanently
* 11/29/2018
* added correction to nPilot for resource shares
* added release and nFiles checks to JumboWatchDog
* 11/26/2018
* using 500kB for analysis outDiskCount
* using 50GB for directIO maxSize
* 11/20/2018
* using TAPE for IO intensity check in AtlasPJB
* 11/19/2018
* added the second look in AtlasAJB to ignore data locality
* set transient for finished+merged+100%
* 11/5/2018
* using new task params for ES
* 10/24/2018
* changed rerefine not to reset some attributes
* 10/22/2018
* added proc_status
* 10/16/2018
* added decAttOnFailedES
* 10/5/2018
* added discardEvents in retryTask
* 9/24/2018
* using resolve_archives=True in list_replicas
* 9/20/2018
* added maxJumboPerSite
* 9/7/2018
* added enableJumbo
* 7/23/2018
* changed getScout to ignore nEvents<10*nCore
* added HS06sPerEventFixed
* 7/19/2018
* changed AtlasAJB to check cmtConfig for nightlies
* 7/17/2018
* added orderByLB
* 7/16/2018
* fixed maxWalltime and maxCpuCount for multiple consumers
* 5/23/2018
* changed AtlasAJB for hospital
* 5/7/2018
* excluding validation from *_AES for es conversion
* 4/25/2018
* make IO intensity cutoff configurable
* 4/19/2018
* using notDiscardEvents for ES tasks by default
* 4/16/2018
* added minCpuEfficiency
* 3/20/2018
* notDiscardEvents
* 3/14/2018
* to reset NG
* 3/1/2018
* added a protection against state change in DB
* 2/28/2018
* changed exhausted messages to INFO
* fixed finished tasks with mutable datasets
* 2/14/2018
* fixed retryTask to go through when input is mutable
* 2/8/2018
* changed not to retry failed child tasks
* 1/26/2018
* added useJumbo=pending
* 1/19/2018
* setting superStatus and T_TASK.status consistently
* 1/17/2018
* added nStandby
* 12/15/2017
* changed to exclude workDirSize from NG
* 12/14/2017
* changing gshare of prod tasks when 95% done
* 12/7/2017
* checking maxattempt when retryTask_JEDI
* 11/29/2017
* setting allowAccessLAN to analysis tasks
* changed InputChunk not to ignore input size for directIO sites if NG is used
* 11/27/2017
* using resurrectConsumers for ES tasks by default
* changed InputChunk to ignore input size for directIO sites
* 11/17/2017
* using NG as tot size of input when pooing input files in child tasks
* 11/15/2017
* added noInputPooling
* 11/14/2017
* fixed double counting of input files in AtlasPJB
* 11/13/2017
* setting prodDBlockToken=local to lib.tgz
* using 60d of lifetime for merged parent datasets
* 11/10/2017
* Changed call to getHS06sec to new mode
* 11/6/2017
* sending analysis tasks with short scouts to exhausted
* 10/25/2017
* fixed AtlasTS to resurrect ES datasets when tasks are retried
* getSURLsFromLFC to also check for gsiftp copies
* change regex to accept se's not ending with /
* 10/16/2017
* unified PQ for analysis
* disable timeout for exhausted when auto finish is disabled
* setting resource_type for buildJob
* fixed AtlasAJB to unified PQ
* 10/10/2017
* adding the offset to nGBPerJob
* 10/09/2017
* added usePrefetcher
* 9/27/2017
* added resurrectConsumers
* 9/26/2017
* removed splitrule check in reactivatePending
* 9/15/2017
* added task.diskIO
* 9/14/2017
* fixed flip-flop of AES flag in refiner
* 9/11/2017
* disabled IO check for event service in AtlasPJB
* 9/6/2017
* added disableAutoFinish
* deleting event service datasets
* fixed respectSplutRule to set nFilesTobeUsed properly
* 8/28/2017
* fixed AtlasAJB/PJB for 3 digtis caches
* 8/23/2017
* using (type=SCRATCHDISK)\notforextracopy=1 for secondary copy of user output
* 8/18/2017
* added registerEsFiles
* 8/14/2017
* added skipShortInput
* 8/11/2017
* added AES_MAXPENDIG
* added SCOUT_CPUTIME_RANK, SCOUT_RAMCOUNT_RANK, SCOUT_RAMCOUNT_MARGIN
* 8/10/2017
* fixed retryTask not to change running files
* 8/9/2017
* fixed AF when tasks are re-refined
* 7/11/2017
* disabled second copy when --destSE
* 7/10/2017
* fixed cloneJobs to set correct ramCount for each PQ
* 7/5/2017
* getting nTasks and nFiles from config
* 7/3/2017
* considering maxNumFiles in getAtom
* added AES_NESCONSUMERS and AES_NESCONSUMERS
* changed AtlasPJB to skip opportunistic sites for high prio jobs
* enabled automatic AES task conversion
* 6/27/2017
* added getNumUnprocessedEvents_JEDI
* 6/26/2017
* skip scouts for ES
* ES without ueRealNumEvt or nEventsPerFile
* WQ-GS alignment
* 5/17/2017
* changed nRunningCap to ignore highPrioWaiting in AtlasPJT
* 5/12/2017
* setting nEsConsumers=1 etc for esFraction
* 5/8/2017
* added nSimEvents
* 5/4/2017
* fixed AtlasAJB for AtnalysisBase
* fixed getAvailableFiles
* 5/3/2017
* added JediDBProxyFiltered
* added inFilePosEvtNum
* 5/2/2017
* changed AtlasPJB for unified PQs
* 4/28/2017
* fixed brokerage for git rel
* 4/25/2017
* fake co-jumbo
* 4/24/2017
* resource_type tagging
* 4/14/2017
* added noExecStrCnv
* 4/11/2017
* added throttleJobsInPausedTasks
* 4/5/2017
* added pseudo output dataset
* 3/27/2017
* changed AtlasAPP to extend lifetime of user output datasets
* 3/2/2017
* added esFraction
* 2/22/2017
* fixed generateBuildJob for nightlies
* changed AtlasTS to make duble copy for users
* 2/20/2017
* changed AtlasPTB for task.nucleus=satellite
* added respectSplitRule
* added indexConsistent
* 2/7/2017
* added DISK_THRESHOLD_xyz in AtlasPTB
* 1/31/2017
* removed dq2 dependency
* changed not to change oldstatus when throttled->paused
* fixed eventRatio for nSecond % nMaster != 0
* 1/13/2017
* introduced GROUPBYATTR in getTasksToBeProcessed
* 1/12/2017
* fixed for incexec with input containers
* 12/20/2016
* changed setScoutJobData to set exhausted when task_ramCount<4GB and scout_ramCount>4GB
* 12/15/2016
* added nQueueCap
* added mergeEsAtOS
* 12/9/2016
* added nRunningCap to AtlasPJT
* disabled setDEL in AtlasPP
* 12/5/2016
* increased scout priority to 901
* 12/1/2016
* changed AtlasPWD to boost task priorities at 95%
* fixed AtlasAJB and JG to support cmake-based nightlies
* 11/28/2016
* fixed setScout to propagate exhausted to DEFT
* 11/24/2016
* changed AtlasPJB not to use baseramcount for pmerge
* disabled transfer backlog check when negative T1 weight in AtlasPTB
* changed thresholds for data availability check configurable in AtlasPTB
* 11/22/2016
* fixed AtlasAJB to use configurator tables instead of schedconfig.space
* 11/16/2016
* changed AtlasAW to use PRESTAGE_LIMIT
* added a hint to setDelFlag
* using 1 for negative ramCount
* fixed AtlasPJB for no-input tasks
* 11/14/2016
* changed outDiskCount not to scale with workDiskCount
* 11/9/2016
* changed AtlasPTB not to ignore job execution ability
* introduced cutoff based on ioIntensity and mbToMove in AtlasPJB
* 11/7/2016
* changed JG not to use compact input params for user's trf jobs
* 10/28/2016
* changed AtlasDDM to work with DN mapped to multiple rucio accounts
* 10/17/2016
* removing multi-byte characters from task params
* added eventRatio
* 10/6/2016
* added addEsParams
* support for reassignTaskToNucleus=None
* 9/30/2016
* changed tobeprocess for ratio in float
* changed ratio of output to check boundaryID
* 9/29/2016
* added support ratio for output datasets
* 9/21/2016
* changed AtlasPTB to use deep in list_dataset_replicas for primary dataset replica lookup
* 9/16/2016
* setting DEL to event ranges
* 9/13/2016
* using maxPSS for analy tasks
* 9/2/2016
* using cloneJobs for ES even with nConsumer=1
* 8/29/2016
* changed analysis to work with containers of containers
* 8/1/2016
* using campaign in WQ
* 7/29/2016
* fixed AtlasPJB to use score queues for pmerge even if site is specified
* 7/25/2016
* fixed IC for cpuEfficiency=0
* tagging of shares
* 7/19/2016
* changed TRB to check boolean value of noWaitParent
* changed IC for NG+NF+LB
* 7/14/2016
* setting old modtime in JG for high prio + running + one or more reset
* using bulk increment in getOutput
* using bulk fetch and insert for files in JG
* 7/12/2016
* changed AtlasTS and AtlasPPP to register and delete ES datasets
* 7/11/2016
* added jumbo job stuff
* added requestType
* added getOldMergeJobPandaIDs
* 7/7/2016
* added removeTags to AtlasAPP
* 7/5/2016
* added rescueUnLockedTasksWithPicked
* 7/4/2016
* changed ToBeProcessed to check done datasets as well
* 6/30/2016
* added ignoreMissingInDS
* using bulk fetching for JEDI_DATASET_CONT_FILEID_SEQ
* 6/28/2016
* resetting transferType for pmerge
* using lfn sort for pmerge even if reuseSecOnDemand is used
* fixed AtlasPWD for deleted datasets
* fixed TC for paused
* 6/24/2016
* enabled history-aware brokerage for analysis
* added splitByFields
* 6/23/2016
* using longer lifetime for unmerged datasets when AOD merge is not very successful
* added useFileAsSourceLFN
* added forceAvalanche
* put logrotate file to etc/panda to avoid automatic installation on test nodes
* 6/20/2016
* using _tid\d+ as a deliminator for NthField
* setting ramCount=2000 for anal tasks by default
* using eventService=2 for tasks with JC
* enabled space check in AtlasPJB
* 6/13/2016
* changed not to reset errorDiag until next JG attempt
* fixed 24h timeout in AtlasPJB to send pmerge to remote sites
* 6/11/2016
* changed TR not to update the same task simultaneously
* 6/8/2016
* increased space limit to 100TB in AtlasPTB
* 6/6/2016
* using rucio in registerDatasetSubscription
* fixed AtlasPWD to use Production Output for subscriptions
* 6/2/2016
* removed scope from tmpin
* 5/25/2016
* changed AtlasPTB to send more log messages
* added writeInputToFile
* set nqueuelimit to 10k for opportunistic
* 5/24/2016
* changes for failedAttempt and hs06sec in job
* 5/18/2016
* removed maxinputsize from JG
* 5/16/2016
* fixed ZeroDivision error in AtlasPTB
* changed AtlasPTB to skip data locality check when no nucleus has data
* 5/12/2016
* changed t1weight based on IO intensity
* 5/11/2016
* added getFailureCountsForTask_JEDI
* changed lifetime of um.log to 365 days
* 5/6/2016
* changed JG not to lock when nQueued<nQueueLimit
* set maxNumJobs=nQueueLimit/10 when nQueued<nQueueLimit
* 5/4/2016
* fixed lockedBy IS NULL in getTaskDatasetsWithID_JEDI
* changed lifetime of transient datasets to 14 days
* 5/3/2016
* changed job brokerages to use mergeRamCount for pmerge
* 5/2/2016
* updated to second version of network brokerage
* changed AtlasPJB to ignore t1w for pmerge
* avoid assigning tasks to nuclei with large transfer backlogs
* fixed taskRW in AtlasPTB
* 4/28/2016
* changed retryTask to update t_task.start_time
* 4/27/2016
* increased nQueueLimit for group to 40k
* changed JG not to lock counter for pmerge or nQueued<nQueueLimit
* fixed JG for uniform tasks with ES and non-ES
* 4/26/2016
* resetting walltimeunit when re-refining tasks
* updates/protections for new network metrics sources
* 4/25/2016
* changed AtlasPTB to respect dst: when input is unavailable
* 4/20/2016
* fixed unlockTasks to release picked files
* introduced 5% of tolerance in minRamCount
* introduced MBPerCoreFixed
* changed JG to set maxWalltime=24h to scouts with cpuTime=0 or None
* 4/19/2016
* fixed getScoutJobData to use jobs with nEvents<10 but wallTime>6h
* 4/18/2016
* using 95th percentile for cpuTime
* fixed dst:XYZ to use nucleus instead of PQ
* 4/14/2016
* using smaller minTotalWalltime for WQ with site in AtlasPJT
* 4/13/2016
* changed resume/pauseTask to activate/throttle jobs
* fixed JG for pending tasks to release lock
* fixed dump in ThreadUtils
* imporved AtlasAJB to work with includedSite=''
* improved unlockTasks to release tasks locked by dead processes
* fixed oldPandaIDs in JG for job cloning and multiple consumers
* 4/12/2016
* added splitRule to mapper attributes
* 4/10/2016
* fixed sortParallelJobsBySite for oldPandaIDs
* 4/7/2016
* added eventService to WQ
* 4/5/2016
* fixed AtlasAJB to works with strings for excluded/includedSites
* changed AtlasPPP to pause tasks when duplication is found
* 4/4/2016
* added fractionFreeSpace to AtlasPTB
* changed AtlasPTB to take fullRW into account for space estimation
* 3/31/2016
* fixed AtlasAJB for ath releases
* disabled to send scouts to non-short queues if site is set
* fixed AtlasAJB for remote access not to ignore resource matching
* 3/30/2016
* fixed prepareTasksToBeFinished_JEDI for scouting+noWaitParent
* using assessmentTime for early avalanche
* 3/24/2016
* fixed scoutJobData for ES
* fixed retryTask for active exhausted tasks
* 3/23/2016
* fixed JG to record retry history for multiple consumers
* fixed JG for ddd:
* 3/21/2016
* changed JS to take job multiplicity into account
* fixed JRB to set implicit parameters for job cloning
* 3/17/2016
* removed quote from error message for upstream
* changed fatal errors in JG to temporary
* added skipUnavailableFiles to InputChunk
* 3/16/2016
* added rerefine to setOnHold
* added putLogToOS
* fixed AtlasTR to correctly set/unset transient to datasets
* 3/15/2016
* set cloudToTask for tasks with no outputs
* fixed updateByCF for tasks with no outputs
* added isDistributedDataset
* changed AtlasPJB to take into account the number of files to move
* using dict instead of list in AGIS json
* 3/11/2016
* changed AtlasPJB to take EC into account when walltime estimation
* changed getTasksToRefine to delete leftover
* 3/10/2016
* changed AtlasDDM to support registration of multiple locations
* 3/8/2016
* fixed InputChunk for distributed datasets
* 3/7/2016
* changed AtlasDDM and AtlasAJB to support distributed datasets
* fixed AtlasPJB to correctly get sites associated to a nucleus
* 3/4/2016
* changed CF to lookup for assigning tasks
* 3/3/2016
* fixed maxEvents for zero event files
* 2/25/2016
* added allowInputLAN
* 2/23/2016
* changed AtlasPJB to support allowInputWAN
* removed some AGIS dependency from JG
* 2/22/2016
* changed to update superstatus when scouting->exhausted
* 2/18/2016
* changed hasZeroShare to work with processingType=validation in non-validation WQ
* added doForPreStaging to AtlasAWD
* 2/16/2016
* changed getScoutJobData to use jobs with nEvents<10 and wall>6h too
* 2/15/2016
* added workQueueName
* aded mcore_repro
* added altStageOut
* 2/10/2016
* fixed retryTask to correctly set status of secondary datasets
* fixed to register final datasets when internal merge is used
* removed T1 constraint from AtlasPJB for high prio or scouts
* 2/9/2016
* added nokill to reassignTask
* 2/8/2016
* added network weights to brokerage
* 2/4/2016
* fixed ToBeProcessed to correctly read secondary files when multiple RAM requirements
* changed finishTask to ignore failGoalUnreached
* setting nucleus and eventService in jobSpec
* 2/3/2016
* fixed MIDDLENAME for multi fileds of addNthFieldOfInDSToLFN
* changed AtlasPTB to use any nuclei where input is available when no nuclei can run jobs
* 2/1/2016
* added reassignTaskToNucleus
* fixed insertFilesForDataset_JEDI for ready->defined in scout
* 1/30/2016
* changed log messages of AtlasPTB to be compatible with bamboo
* 1/27/2016
* changed AtlasPTB to get messages from AtlasPJB
* 1/25/2016
* changed addNthFieldToLFN to allow multiple fields
* 1/19/2016
* changed AtlasPP to set lifetime to failed log datasets
* fixed getAnalSitesWithData to ignore STAGING
* 1/18/2016
* changed getScoutJobData to reduce baseWalltime when calculating cpuTime
* changed TR to respect the boolean vale for failWhenGoalUnreached
* 1/15/2016
* changed message type for WORLD taskbrokerage
* setting maxWalltime to maxCpuCount for HS06
* changed AtlasPJB to support 3 digits base releases
* 1/14/2016
* changed setScoutJobData so that tasks go to exhausted when execution time is unexpectedly too long
* removed TiersOfATLAS from AtlasDDM
* 1/5/2016
* changed restartTasksForCF to restart ready tasks with defined DSs
* 12/31/2015
* fixed CF to use nFilesForScout for toupdate in scouting
* 12/16/2015
* changed restartTasksForCF to restart defined tasks with only ready DSs
* 12/15/2015
* using roundi up for the brokerage as well
* added protection against nFilesPerJob=0 and nEventsPerJob=0
* 12/9/2015
* rounding up job.minramcount with 500MB chunks
* 12/8/2015
* fixed PP to go finished instead of exhausted if goal is reached
* changed AtlasPJB to send pmerge to anywhere after 24h
* 12/7/2015
* removed validationrelease from AtlasPJB
* enabled duplication check for production
* 11/27/2015
* fixed for reassignTask not to make subscriptions for distributed datasets
* removed offet for zero events in getEffectiveNumEvents
* 11/23/2015
* added updateJobMetrics
* fixed nEventsPerMergeJob
* 11/19/2015
* changed AtlasPP to delete empty datasets
* fixed AtlasPTB for WORLD + no output
* changed getScout to ignore jobs with nEvents<10
* 11/18/2015
* fixed AtlasPTB for WORLD
* 11/16/2015
* changed getScout to fill IO info
* 11/13/2015
* fixed AtlasPTB for negative t1Weight in WORLD
* changed AtlasPTB for all prod jobs to respect fairsharepolicy
* 11/11/2015
* fixed AtlasPTB for missing nuclei in RW
* added protection to RW calculation against cpuTime=NULL
* changed AtlasAJB to take wansinklimit into account for WAN weight
* 11/10/2015
* changed prod/anal prokerages to use max/minrss
* 11/9/2015
* changed JG to avoid passing parallelOutMap to TB
* 11/6/2015
* removed T1 constraint for ES job brokerage
* changed AtlasPTB to use confgurator tables
* using jobseed=std by default
* changed AtlasPTB to take size of expired secondaries into account
* 11/5/2015
* changed jobseed=all when switchEStoNormal is not used
* 11/4/2015
* changed JG to remove job parameters for empty streams
* 11/2/2015
* giving a smaller weight to tape for analy
* 10/27/2015
* tweaked freezeDataset to avoid a rucio bug
* fixed setCloudToTask to respect distributed datasets
* 10/26/2015
* fixed for internal merge + zero suppression
* added checkDuplication
* avoiding duplication for non-zero RAM chunk
* 10/22/2015
* changed insertFiles to separate FID retrieval
* changed AtlasPJB to cap based on nActivated+nStarting when data is available
* 10/21/2015
* changed getScoutJobData to ignore jobs with nEvents<10
* using bulk insert in insertFiles
* fixed slow insertFiles due to givenFiles
* 10/20/2015
* fixed getMaxAtom for pmerge to use getNum*PerMergeJob
* added allowNoOutput
* 10/19/2015
* changed AtlasTPP to set lifetime to transient datasets
* 10/16/2015
* fixed resetUnused to take ramcount into account
* 10/14/2015
* added dynamicNumEvents stuff
* added maxNumEventRanges
* 10/12/2015
* added dynamic nQueueLimit
* added timeoutForPendingVoLabel
* 10/8/2015
* ignoring TEST when converting token to RSE
* 10/7/2015
* added a margin to the multiplier of disk count estimation for pmerge
* fixed CF for nEventsPerRange
* 10/6/2015
* changed JG to release the lock as soon as tasks are taken when nQueued<nQueueLimit
* fixed lackOfJobs
* 10/2/2015
* fixed JG to reset destinationDBlock when the site doesn't have endpoint for ddd:
* more fixed CF for seq_num+skipFilesUsedBy
* 10/1/2015
* fixed CF for seq_num+skipFilesUsedBy
* changed JG to lock just before final task update
* 9/30/2015
* added ddd:
* changed AtlasPJB to work with cpuTime=0
* 9/29/2015
* fixed setScoutJobDataToTasks_JEDI to take young tasks into account
* suppressed files in JediDataset.str
* 9/28/2015
* added retrial module
* 9/27/2015
* fixed AtlasProdWatchDog for reassigning
* 9/25/2015
* added protection to TC for empty commentStr
* 9/24/2015
* changed APJB to take priority-based shares into account
* 9/22/2015
* removed amiFlag
* 9/21/2015
* fixed failWhenGoalUnreached for goal=100.0
* fixed rescuePicked to lock tasks
* 9/11/2015
* added setScoutJobDataToTasks
* 9/9/2015
* added WORLD
* 9/1/2015
* added skipFilesUsedBy
* 8/27/2015
* changed AtlasPJB to skip share check for test/validation
* 8/25/2015
* setting activity for registerDatasetLocation
* changed task.outDiskCount to include size of intermediate files
* changed task.outDiskCount to scale with events when outDiskUnit has PerEvent
* changed AtlasPJB so that merge jobs avoid SHORT
* 8/24/2015
* modified retryChildTask not to reactivate children if they were aborted/broken
* 8/21/2015
* changed prepareTasksToBeFinished so that scouting can be kicked
* increasing taskprio by 1 for ready/scout so that scouts don't wait behind the bulk
* 8/20/2015
* fixed getTasksToBeProcessed to respect priorities of prod tasks
* changed the limit to avoid inactive sites to prio=800
* 8/18/2015
* increased the limit on how many times dataset is duplicated in a single cycle
* 8/11/2015
* removed RRC-KI-T1 from slow site list
* 8/6/2015
* added a mechanism so that a single task can generate both ES and normal jobs
* 8/5/2015
* changed PJB for zero-walltime to avoid short queues
* 8/4/2015
* changed locCheck to cloudLocCheck
* 8/3/2015
* changed PP so that child tasks immediately go to finished w/o exhausted when all inputs are successful
* added cloud locality check
* added failWhenGoalUnreached
* 7/30/2015
* fixed AtlasPJB to check inactive sites with nActivated>0
* 7/30/2015
* changed AtlasPJB to avoid inactive sites for job.prio>=900
* 7/16/2015
* changed AtlasPJB to avoid slow sites for job.prio>=900
* 7/14/2015
* sending scouts to sites where walltime is more than 24h
* 7/13/2015
* fixed getSubChunk to correctly take secondary files into account
* 7/10/2015
* changed cpuTime for scouts to always update the value
* changed walltime messages more informative
* 7/9/2015
* added job cloning
* added message to errorDialog when goal is reached
* 7/8/2015
* fixed retryTask to reset picked files
* setting amiFlag
* changed goal to have the first decimal place
* changed child tasks not to go done when parents are not done
* fixed reactivatePendingTasks to timeout child tasks correctly
* increased upper limit of outdiskcount to 10MB
* 7/7/2015
* added goal
* added transient flag to datasets
* 7/6/2015
* added size check in taskParam copy from DEFT
* 6/26/2015
* added 50% of safety mergin to cpuTime
* added more message for input file shortage
* fixed maxWalltime for CPU efficiency
* added stayOutputOnSite
* 6/24/2015
* decomposed INSERT+SELECT to copy task params from DEFT
* changed tgtNumEventsPerJob to ignore file boundaries
* 6/23/2015
* added reactivateTasks for high prio tasks
* reduced delay for reassignTask
* added traceback to dumpErrorMessage