09/10/18
Run data vs MC comparison:
git clone https://:@gitlab.cern.ch:8443/VBFInv/STPostProcessing.git source
mkdir build run
cd build
acmSetup AthAnalysis,21.2.45
acm compile
# everytime: acmSetup # compile: acm compile
cd ../run
# create file "list" with the list of directories to the samples of interest
ls -d /nfs/dust/atlas/user/othrif/samples/MiniNtuples/v11/user.jvonahne.v11.* > list
python ../source/VBFAnalysis/util/writeFileMapLS.py
submitVBFAnalysisCondor.py -l myMap.p -n -f $TestArea/x86_64-slc6-gcc62-opt/data/VBFAnalysis/f_out_total_v13Loose.root
08/10/18
Week goals:
- machine shop drawing
- fit model and treatment of uncertainties in the CMS paper
- theory uncertainties and fit model with mono-jet and Jonas
- EWK 2L0J Int note
ITK:
- got .iges drawings of the bridge and bridgeless tools
- need the inner frame and side mounts for camera and glue dispenser
28/09/18
Typical submission:
kinit othrif@CERN.CH
cd /nfs/dust/atlas/user/othrif/vbf/myVBF/STAnalysisCode
git add .
git commit -m "Preparing v11 submission"
git tag -a v11-AB21p2p45 -m "Tag version v09"
git push origin v09-AB21p2p42
cd versions_myVBF/
mkdir v11 && cd $_
git clone https://:@gitlab.cern.ch:8443/VBFInv/STAnalysisCode.git
cd STAnalysisCode
git checkout tags/v11-AB21p2p45
cd ..
source STAnalysisCode/VBFInvAnalysis/scripts/setupRelease.sh
cd ../STAnalysisCode/VBFInvAnalysis/arguments/
emacs v11_args.txt # change name of version, and --doSyst
cd ../../../../run/
runVBF.sh v11 pilot
nohup runVBF.sh v11 mc16a_EXOT5 > nohup_mc16a.out &
nohup runVBF.sh v11 data15_EXOT5 > nohup_data15.out &
nohup runVBF.sh v11 data16_EXOT5 > nohup_data16.out &
nohup runVBF.sh v11 data17_EXOT5 > nohup_data17.out &
Useful dealing with tags:
# create tags
git tag v11-AB21p2p45
git push origin v11-AB21p2p45
# delete tags
git tag -d v11-AB21p2p45
git push origin :refs/tags/v11-AB21p2p45
27/09/18
Theory talk:
- sensitive to cuts? and corrections... -
Vincent talk:
- drop mht cut for both
- tool to see if it is pileup or not
- plot jvt for pt<60GeV
- check jvt configuration
Muons:
- calo tag muons?
- combined muons: try Loose (which includes calo-tagged muons) https://gitlab.cern.ch/VBFInv/STAnalysisCode/blob/master/VBFInvAnalysis/share/SUSYTools_VBF.conf#L26
26/09/18
To do:
- Check Isolation
- Launch full production with minimum events in data and MC
20/09/18
Instruction for release comparison plotting:
Plot the same event from rel20 and rel21 in a 2D plane
# Pick the common runNumbers and eventNumbers between the releases in processed_data_rel2*.root python diff_two_trees.py # Create histograms from trees python ntuplesToHists.py input/diff_vbf_rel20_rel21.root --config diff_hists_config.json --treename nominal --eventWeight "1" --newOutputs # plot the file in input/hists_diff_vbf_rel20_rel21.root python diff_make_plots.pyPlot all events from rel20 and rel21 in a 1D plane:
python extract_all.py # For all events python extract_common.py # For common only python ntuplesToHists.py input/sel/Incl_vbf_rel2*.root --config hists_config_Nm1.json --treename nominal --eventWeight "1" --newOutputs python ntuplesToHists.py input/sel/Com_vbf_rel2*.root --config hists_config_Nm1.json --treename nominal --eventWeight "1" --newOutputs root plot.CRun cutflow comparing rel20 and rel21:
Prepare the processed files in rel20 and rel21:
cd /nfs/dust/atlas/user/othrif/vbf/myVBF/run runVBF.py -i file cd /nfs/dust/atlas/user/othrif/scratch/myPostProcessing/run ./run.sh -1 ./user.othrif.vXX.308276.SAMPLE.root cp ~/dust/vbf/myPostProcessing/run/VBFH125Nominal308276.root input/raw_vbf_rel21.root root rel21.C > rel21 t > t.Loop()
19/09/18
Don't really have a resolution on muon isolatino WP as i dont' see a change in results I need to figure out how to keep two cleaning separate for PFlow and EMTopo.
Matlab lisence: https://confluence.desy.de/display/IS/Matlab+Mac+OSX+Installation
Running QuickVBF code:
cd /nfs/dust/atlas/user/othrif/scratch/quickAna/QuickVBF
source rcSetup.sh
# change the file list in submitGrid.C
root -l -b -q $ROOTCOREDIR/scripts/load_packages.C submitGrid.C
Rel20 lepton definition:
- Electron: id=tight, iso=gradient, z0Sin<0.5, d0/sig<5
- Muon: id=Medium, iso=gradient, z0Sin<0.5, d0/sig<3
For Rel21:
- Electron: id=TightLHElectron, iso=Gradient,
To do:
- Write talk VBF+MET
- Implement tight cleaning for PFlow jets
- Fix lepton requirements
- Check Overlap removal
- Release comparison update
- Andy
Chat with Michaela and Christian regarding 3rd jet veto:
- Non-closure of jet calibration in low pT region improved in rel21. Meaning: if i have a truth jet pt of 20GeV. In rel20, the calibration is 1.2 and in rel21, it is 1.1. Which means that more jets will have migrated from <25gev to="">25GeV in rel20 than in rel21. This translates to having less events in rel21 at > 25GeV. Of course these events are coming from the <25GeV slice which are the events we are keeping in the analysis. So we keep more events, thus higher ratio. This is at least what we are seeing.
Chat with Priscila:
- MET+2b: figure 16 of https://cds.cern.ch/record/2314002/files/ATL-COM-PHYS-2018-409.pdf similar to VBF diagram, might improve sensitivity. Can get signals to run on in rel20
- Study dark matter summary paper
- Look at draft white paper in email
- bibile for 2HDM: https://link.springer.com/content/pdf/10.1007%2FJHEP05%282017%29138.pdf
- problem is coupling to bosons, in summary paper is set to 0
- try to answer the question of why varying the Higgs mass is important?
18/09/18
Two commands to run:
python ntuplesToHists.py input/sel/Inclall_vbf_rel2*.root --config hists_config_Nm1.json --treename nominal --eventWeight "1" --newOutputs
root plot.C
17/09/18
Problem found with release comparison: MET trigger. I think i am doing something wrong since I have to treat the different years separately.
To do:
- Run quickAna on VBF and data
- Run rel comparisons on data
- EXOT5 derivation
- Send email to Jonas
Figure out differences between MC
Beate: Lower the MET cut
Rel20 normalization: w = lumi xs 1000 w_event filter_eff /nevents; w_event = weightMC weightReco weightNJet * weightPileup; Xsec = 3.782, filter eff = 0.22285, lumi = 36.1, nevents = 7.442760e+05
Running QuickAna Postprocessing code:
cd /nfs/dust/atlas/user/othrif/scratch/QAPostProcess/QAPostProcess
source rcSetup.sh
./SkimPackage/scripts/build_sh_othrif.py
python ./SkimPackage/scripts/hist_maker_loop_prova.py --runDir sh_met100_201516_dec_jetClean_nominal --submitDir submit_othrif --treeSuffixName NONE
Run numbers in data to process that have most events in SR:
310634 102
304128 98
302872 91
307732 88
308047 84
302393 79
307454 78
303208 77
303846 77
303304 76
301973 75
(code in get_rel21_important_runs.py of relComp_230818)
Cutflow comparison between rel20.7 and rel21:
Count Rel. 20 Rel 21
-------- -------- --------
All events 169042 25760.93 161238 24212.57
Jet cleaning 163831 24946.50 0.97 156992 23575.22 0.97
MET trigger 151332 23010.43 0.92 153128 23007.85 0.98
j1_pT>80GeV 140627 21387.55 0.93 143069 21493.52 0.93
j2_pT>50GeV 87328 13297.43 0.62 90927 13674.75 0.64
n_jet>=2 87328 13297.43 1.00 90927 13674.75 1.00
n_lep=0 87297 13292.94 1.00 90820 13658.98 1.00
N3rdjet=0 47131 7180.84 0.54 52059 7887.25 0.57
j1_eta*j2_eta<0 41167 6266.95 0.87 45562 6901.62 0.88
dEta(jj)>4.8 16428 2493.09 0.40 18522 2803.71 0.41
dPhi(jj)<1.8 13230 2004.43 0.81 14939 2263.50 0.81
dPhi(j1,MET) 13230 2004.43 1.00 14939 2263.50 1.00
dPhi(j2,MET) 13230 2004.43 1.00 14939 2263.50 1.00
MET>180GeV 6276 948.20 0.47 6776 1026.72 0.45
HTmiss>150GeV 6203 937.14 0.99 6737 1021.04 0.99
M(jj)>1TeV 6151 929.81 0.99 6696 1014.88 0.99
After fixing the typo of met, and common skimming applied:
Summary Rel. 20 Rel 21
-------- -------- --------
All events 98226 15016.30 99395 14946.78
Jet cleaning 95151 14534.61 0.97 96784 14555.13 0.97
MET trigger 89930 13717.71 0.95 95011 14295.77 0.98
j1_pT>80GeV 83168 12691.02 0.92 87369 13145.65 0.92
j2_pT>50GeV 67507 10304.62 0.81 71030 10682.69 0.81
n_jet>=2 67507 10304.62 1.00 71030 10682.69 1.00
n_lep=0 67486 10301.92 1.00 70966 10673.13 1.00
N3rdjet=0 40934 6232.95 0.61 45378 6871.44 0.64
j1_eta*j2_eta<0 38573 5868.88 0.94 42746 6473.75 0.94
dEta(jj)>4.8 16362 2482.55 0.42 18451 2792.84 0.43
dPhi(jj)<1.8 13214 2002.04 0.81 14917 2260.47 0.81
dPhi(j1,MET) 13214 2002.04 1.00 14917 2260.47 1.00
dPhi(j2,MET) 13214 2002.04 1.00 14917 2260.47 1.00
MET>180GeV 6276 948.20 0.47 6776 1026.72 0.45
HTmiss>150GeV 6203 937.14 0.99 6737 1021.04 0.99
M(jj)>1TeV 6151 929.81 0.99 6696 1014.88 0.99
Adding yet another fix for the MET trigger strategy:
All events 98226 15016.30 99395 14946.78
Jet cleaning 95151 14534.61 0.97 96784 14555.13 0.97
MET trigger 89930 13717.71 0.95 90416 13638.65 0.93
j1_pT>80GeV 83168 12691.02 0.92 83904 12650.62 0.93
j2_pT>50GeV 67507 10304.62 0.81 68448 10314.48 0.82
n_jet>=2 67507 10304.62 1.00 68448 10314.48 1.00
n_lep=0 67486 10301.92 1.00 68387 10305.32 1.00
N3rdjet=0 40934 6232.95 0.61 43706 6631.60 0.64
j1_eta*j2_eta<0 38573 5868.88 0.94 41191 6251.11 0.94
dEta(jj)>4.8 16362 2482.55 0.42 17664 2680.23 0.43
dPhi(jj)<1.8 13214 2002.04 0.81 14332 2176.74 0.81
dPhi(j1,MET) 13214 2002.04 1.00 14332 2176.74 1.00
dPhi(j2,MET) 13214 2002.04 1.00 14332 2176.74 1.00
MET>180GeV 6276 948.20 0.47 6763 1024.89 0.47
HTmiss>150GeV 6203 937.14 0.99 6724 1019.22 0.99
M(jj)>1TeV 6151 929.81 0.99 6683 1013.05 0.99
At derivation EXOT5, we apply the skimming:
LeadingJetPtCut = 40000.,
SubleadingJetPtCut = 40000.,
DiJetMassCut = 150000.,
VBFJetThresh = 40000.,
DiJetMassMaxCut = 150000.,
DiJetDEtaCut = 2.5
For v09 MiniNtuples, i apply the skimming:
VBF.pt1Skim: 20000
VBF.pt2Skim: 20000
VBF.metSkim: 100000
VBF.mjjSkim: 200000
VBF.detajjSkim: 2.5
A conservative skimming is then:
VBF.pt1Skim: 40000
VBF.pt2Skim: 40000
VBF.metSkim: 100000
VBF.mjjSkim: 200000
VBF.detajjSkim: 2.5
Interesting to look at this trigger code: checkXETrigger in Triggers.cxx in QuickAna.
14/09/18
Uncertainty on TF: (1+unc_SR)/(1+unc_CR) -1 where unc=(nom-up)/nom
To do:
- Run quickAna
- run on data with postprocessing
- run with systematics in rel21
- setup HF
- send slide to kurt done
- send slide to sarah done
running quickAna
rcSetup Base,2.4.33
rc find_packages
rc clean
rc compile
root -l -b -q $ROOTCOREDIR/scripts/load_packages.C runLocal.C'("submitDir")'
Next time you set up:
source rcSetup.sh
Check talk about alternative MET defintions to deal with fake MET:
https://indico.cern.ch/event/750247/contributions/3138641/attachments/1716342/2769096/JDMmetmethods.pdf
Jet and MET workshop:
https://indico.cern.ch/event/711895
PFlow:
https://indico.cern.ch/event/751945/contributions/3116024/attachments/1705821/2748672/StevenSchramm-PFlow.pdf
Educate yourself on MET:
https://indico.cern.ch/event/711895/contributions/3000791/attachments/1714292/2764970/IntroToMET_HCW2018.pdf
13/09/18
To do:
- Rel comparisons:
- Plot rel20 vs rel21 before MET and TJV
- Plot SR, SR-MET, SR-JVT, SR-(MET+JVT)
- Plot SR with weight
- Plot SR with pileup windows
- Removed photons from MET calculation
no impact
- Plot for all events as well as common ones
- Plot 2D the diff
- Plot rel20 vs rel21 before MET and TJV
- EXOT5 derivation
- Kurt: VBF and Module
To run the full chain of release comparison:
cd /nfs/dust/atlas/user/othrif/vbf/myVBF/run
runVBF.py -i file
cd /nfs/dust/atlas/user/othrif/vbf/myPostProcessing/run
./run.sh -1 ./user.othrif.vXX.308276.SAMPLE.root
cp ~/dust/vbf/myPostProcessing/run/VBFH125Nominal308276.root input/raw_vbf_rel21.root
root rel21.C
> rel21 t
> t.Loop()
python extract_all.py # For all events
python extract_common.py # For common only
python ntuplesToHists.py input/sel/Inclall_vbf_rel2*.root --config hists_config_Nm1.json --treename nominal --eventWeight "1" --newOutputs
python ntuplesToHists.py input/sel/Allcommon_vbf_rel2*.root --config hists_config_Nm1.json --treename nominal --eventWeight "1" --newOutputs
root plot.C
12/09/18
Change1: remove the "Data-nominal" line from xml files
Change2: runAsymntotic, only change obsData to asimovData, 'asimovData_0' must stay
check3: Open the main root file (like hvbfhi_125_2016_combined_allsys_model.root), and do combined->Print() then search for datasets, that's where the obsData or asimovData defined
11/09/18
To do:
- EXOT5 derivation
- Cutflow check
- Compile code
- ITK code
10/09/18
Rel20 analysis: VBF signal has 135400 weighted events in the paper. AMI has 199000 MC events. Cross section: 3.736 pb, Efficiency: 0.22266, Luminosity: 36/fb Normalization factor: 3.736 0.22266 36 * 1e3 / Ngen
07/09/18
To do:
- Cutflow check
- ITK code
- compile analyzer code
06/09/18
To do:
- Send slide to kurt
- Prepare Talk
Run code ITK
VBF parton filter in MadGraph and Powheg, extract the partons and emulate the filter and see how efficient it is
- Can we generate LO large sample in PTV, quantify the fluctuation in every variable you want to cut in, how much faster the generator should be for the strategy works > several issues -
Commands:
python ntuplesToHists.py input/processed_data_rel20.root --config hists_config.json --treename nominal --eventWeight "1" --newOutputs
05/09/18
To do:
- check met in vbf with new met trigger
- simplify analysis code
- check rel21 vs. rel20
Stat code:
- https://gitlab.cern.ch/VBFInv/StatsTools/tree/master/HistFactoryTool/input/2018-May15-separate/data -
Typical submission:
kinit othrif@CERN.CH
cd versions_myVBF/
mkdir v09 && v09
git clone https://:@gitlab.cern.ch:8443/VBFInv/STAnalysisCode.git
source STAnalysisCode/VBFInvAnalysis/scripts/setupRelease.sh
cd STAnalysisCode/VBFInvAnalysis/data/
mv v05_pilot.txt v09_pilot.txt
mv v05_mc16a_EXOT5.txt v09_mc16a_EXOT5.txt
mv v05_data16_EXOT5.txt v09_data16_EXOT5.txt
mv v05_data17_EXOT5.txt v09_data17_EXOT5.txt
cd ../arguments/
mv v05_args.txt v09_args.txt
emacs v09_args.txt # change name of version, and --doSyst
cd ../../../../run/
runVBF.sh v09 pilot
nohup runVBF.sh v10 mc16a_EXOT5 > nohup_mc16a.out &
nohup runVBF.sh v10 data15_EXOT5 > nohup_data15.out &
nohup runVBF.sh v10 data16_EXOT5 > nohup_data16.out &
nohup runVBF.sh v10 data17_EXOT5 > nohup_data17.out &
cd ../STAnalysisCode
git add .
git commit -m "Preparing v09 submission"
git tag -a v09-AB21p2p42 -m "Tag version v09"
git push origin v09-AB21p2p42
04/09/18
To do:
- EXOT5 VBF triggers
- Develop analysis code
- Plot with systematics
Promised:
- Check Powheg emulation filter with Mjj from SM group
Sherpa jobOptions:
- In
https://svnweb.cern.ch/trac/atlasoff/browser/Generators/MC15JobOptions/trunk/share/DSID364xxx/MC15.364262.Sherpa_222_NNPDF30NNLO_Znunu_MAXHTPTV140_280_MJJ0_500.py, they changed
NJET:=3; LJET:=2,3,4; QCUT:=20.;toNJET:=3; LJET:=0; QCUT:=20.;to get rid of the NLO part. The rest of the files are in: - https://svnweb.cern.ch/trac/atlasoff/browser/Generators/MC15JobOptions/trunk/common/Sherpa/Sherpa_2.2.2_NNPDF30NNLO_Common.py
- https://svnweb.cern.ch/trac/atlasoff/browser/Generators/MC15JobOptions/trunk/common/Sherpa/Sherpa_2.2.4_Base_Fragment.py
03/09/18
Plot with systematics
Checked in 21.2.41 but have problems with JetUncertainties, skyped Keisuke
done
I figured out the problem about cutflows: -Events that pass SR in both rel20 and rel21 -rel20 SR: 6151 -rel21 SR: 6664 -overlap: 3922 -r21 events corresponding to rel20 events that passed SR -rel20 SR: 6151 -rel21 SR: 6090 -overlap: 6090
- Size of v05tmp is 1.4TB are ready
- v04Truth:
nohup rucioget.sh user.othrif v04TruthRetry083118*MiniNtuple.root > nohup0.out &being downloaded and i also re-submitted some 'finished' state. NEED to update again - duplicate v08:
ruciolist.sh user.rzou v08.*MiniNtuple.root | gridDuplicate.sh
To do:
- cutflow checks
done
- Vincent PFlow with analysis code
done
- Plotting code for Janik with systematics
- v08 download
- truth samples
- launch v08Syst
- Review material ITK: Scott and Daniele
- write email to group
- Request derivations and add VBF MET trigger
Week to do:
- Rel comparisons
- Vincent PFlow with analysis code
- Janik with HistFitter
- data16 vs data17 vs data18
- FJvt with truth vertex and truth MET
- Figure out why grid link doesn't work
31/08/18
Duplicating v08: ruciolist.sh user.rzou v08.*MiniNtuple.root | gridDuplicate.sh
Download v05Syst for missing files after grid credentials expired: nohup rucioget.sh user.othrif v05Syst*MiniNtuple.root > nohup2.out &
To do:
- Launch Truth samples for Janik
done
- Figure out difference in cutflow
- HFitter config
- Analysis code + Plotting to use
For next week:
- Vincent PFlow with analysis code
- Janik with HistFitter
- Rel comparisons
- data16 vs data17 vs data18
- FJvt with truth vertex and truth MET
- Figure out why grid link doesn't work
Tryuing to match to the numbers below from Rui using v08 for rel21:
all 6090
trig 6076
n_mu 6075
n_el 6073
met_tst_et 5196
met_cst_jet 5175
met_tst_j1_dphi 5128
met_tst_j2_dphi 5126
passJetCleanTight 5032
n_jet 4138
j1 4138
j2 3952
mjj 3944
dphijj 3939
hemi 3939
deta 3922
git tag -a v04Truth-AB21p2p3 -m "tag v04Truth code for truth studies"
git push origin v04Truth-AB21p2p3
30/08/18
To do:
- Run on MiniNtuples to produce output and plot quickly
- Fix PFlow for Vincent
- Janik plotting
- Create list of new EXOT5 derivations for MC with p3596
- Add VBF MET to derivation
29/08/18
To do:
- PRW file
done
- Rel20 match
- Add new MET definitions ST
athena VBFAnalysis/VBFAnalysisAlgJobOptions.py --evtMax 10 --filesInput /nfs/dust/atlas/user/othrif/samples/MiniNtuples/v04tmp/user.othrif.v04.00276954.physics_Main.r9264_p3083_p3576_MiniNtuple.root/user.othrif.14790528._000001.MiniNtuple.root - --currentVariation Nominal
athena VBFAnalysis/VBFAnalysisAlgJobOptions.py --evtMax 10 --filesInput - --currentVariation Nominal
23/08/18
Look at Cutflow in Rel20 and Rel21
22/08/18
Data numbers close to the paper, but have a factor of 2 less in Wenu and i haven't implemented mht cut.
''' bash python ntuplesToHists.py input/processed_data_rel20.root --config hists_config.json --treename nominal --eventWeight "1" --newOutputs python ntuplesToHists.py input/common_data_rel20.root input/common_data_rel21.root --config hists_config.json --treename dataNominal --eventWeight "1" --newOutputs '''
21/08/18
My Code struct ure:
- Ntuple production to MiniNtuples
- Postprocess to MicroNtuples
- Analyze trees to make histograms and cutflows
- Plot _ Input to HistFitter
To do:
- Region selection
done
- data17 lunch
done
- ingrid order: vacuum terminal and controller
done
20/08/18
To do:
- Rel20.7 vs Rel21 eventNumbers/runNumbers for data
done
- Submit travel forms
done
- Order HW for module loading: PC, vacuum
discussed with Sergio, talk to David tomorrow
- Check systematics variation status
done
Steps to reproduce release comparisons:
cd /Users/othmanerifki/scratch/gstark/myR21Scrutiny
# get processed_data_rel2X.root from raw_data_rel2X.root:
root rel21.C
root > rel21 t
root > t.Loop()
root rel20.C
root > rel20 t
root > t.Loop()
# get common_data_rel2X.root from processed_data_rel2X.root
python get_rel21_runsANDevents_used.py
# get histograms from trees
python TakeOverTheWorld/NTuplesToHists/YourMacrosAreBadAndYouShouldFeelBad.py vbf/common_data_rel20.root vbf/common_data_rel21.root --config hists_config.json --treename dataNominal --eventWeight "1" --newOutputs
# plot
root read.C
# plots in plots/
18/08/18
python extract_rel20_by_runs.py
python TakeOverTheWorld/NTuplesToHists/YourMacrosAreBadAndYouShouldFeelBad.py vbf/common_data_rel20.root vbf/common_data_rel21.root --config hists_config.json --treename dataNominal --eventWeight "1" --newOutputs
python TakeOverTheWorld/totw.py --config plots_config_rel20rel21.yml --file-ext pdf
17/08/18
Next week:
- Meet with Ingrid and Sergio to discuss equipment:
- Tell about Joystick that is included
- PC ordering
- Vacuum
Today:
- Rel20.7 vs Rel21 eventNumbers/runNumbers for data
- Histfitter setup with dictionaries
- Submit travel forms
- LO vs. NLO, contact Ben, Frank
done
16/08/18
- PC specs
done
- Hardware list to ingrid
done
- Rel20.7 vs Rel21 eventNumbers/runNumbers for data
- Plotting code
- Histfitter setup with dictionaries
- Submit travel forms
- LO vs. NLO, contact Ben, Frank
- Ping JIRA requests
15/08/18
- Documenting how i make the Ntuple production.
- Now I have all Rui's files here: /nfs/dust/atlas/user/othrif/vbf/myPostProcessing/run/run_rui
- Time to use MonoJet plotting code to make plots!!!
Missing container from DESY-HH_LOCALGROUPDISK: user.othrif.v04.00303846.physics_Main.r9264_p3083_p3576_MiniNtuple.root
Requested a duplication
14/08/18
Goals for the week:
- PFlow jets vincent
done
- Systematic submit
done
- Data vs. MC
- Rel21 data
- Order list
- Submit travel forms
- NLO Sherpa vs. LO, why adding extra partons we still see this problem of high Mjj
Talk with James about defining a new PV. only change is the JVT scale factors.
08/08/18
How to properly find your DSIDs:
- Give it: DSID, derivation, ptag
Gives me: EVNT, DAOD_EXOT5 (data15,datat16,data17,data18,mc16a,mc16c,mc16d), TRUTH lists
06/08/18
Goals of the week:
- Theory corrections
- PFlow jets
- Systematics Ntuples
- Data vs. MC
- HistFitter fit
- Sherpa LO
JES Uncertainties:
Code to run https://gitlab.cern.ch/atlas/athena/blob/21.2/Reconstruction/Jet/JetUncertainties/util/MakeUncertaintyPlots.cxx
Script to execute https://gitlab.cern.ch/atlas/athena/blob/21.2/Reconstruction/Jet/JetUncertainties/util/MakeUncertaintyPlots.cxx
Basically, make a fake jet and give it a pt and eta of whatever bit of that you want to look at then retrieve the uncertainties for that jet like this with the tool and then you can define how big the uncertainty is
01/08/2018
Preparing a new production:
getDAODfromAOD.sh vbf/myVBF/STAnalysisCode/VBFInvAnalysis/data/samples/v04/mc16a_AOD.txt mc16_13TeV EXOT5*r9364_r9315_p3575
31/07/18
Launched v04Truth with a reduced jet pt threshold to 5GeV:
Command to run: runVBF.py -a VBFInvTruth -v v04Truth -d prun -l listProduction/v04/TRUTH3.list
Fixed the list of TRUTH3 samples here:
Code development:
Done >
Fix triggers
Fix cross section
Disable PDFs
Standard cleaning in one flag: flag_lar, etc. > pass_event
Jet cleaning in tight and loose
Ele and muon trigger flags, keep MET seperate.
Remove GenMET, check with christian
Get rid of bare definition,
Get rid of boson stuff
Fix jj_variables, you don’t really need to have them in content holder….
Save vector of truth jets
Get rid of etx and ety
Styles plotting out of post processing
Fix Remove doDetail, not needed just use doTrim flag
Truth leptons but not for systematics
Leptons status not needed
Replace the MET-noel/nomu to MET-nolep
MET>100GeV
DEta>2.5
Minimal list
To be done >
Run with systematics
PDF uncertainties in current code
Theory weights: Reno, fact, max of all PDF variations
Jet moments
JVT Loose, tight, medium configuration throughout, consistently throughout
Tau study and truth taus
Trimming with doTrim:
- Default > all > 607K
- doSyst > basic > 138K
- doXXXDetail -> basic + XXX detail
24/07/18
Back taking notes...
Analysis preservation, checkout:
- event selection: https://gitlab.cern.ch/MultiBJets/MBJ_Analysis
- histfitter: https://gitlab.cern.ch/MultiBJets/MBJ_HistFitter
28/06/18
To do:
- update GRL: https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/GoodRunListsForAnalysisRun2#2017_13_TeV_pp_data_taking_summa
- 2018 GRL
PRW email from Sam in 02/07/2018: Plot mu_avg(data) compared to mu_avg(MC) without PU weights Plot mu_avg(data,scaled/corrected) compared to mu_avg(MC) without PU weights Plot mu_avg(data) compared to mu_avg(MC) with PU weights Plot mu_avg(data,scaled/corrected) compared to mu_avg(MC) with PU weights Plot mu_actual(data) compared to mu_actual(MC) without PU weights Plot mu_actual(data,scaled/corrected) compared to mu_actual(MC) without PU weights Plot mu_actual(data) compared to mu_actual(MC) with PU weights Plot mu_actual(data,scaled/corrected) compared to mu_actual(MC) with PU weights Plot NPV(data) compared to NPV(MC) without PU weights Plot NPV(data) compared to NPV(MC) with PU weights - Don’t expect this to agree. Agreement here is not a confirmation that things are being done correctly.
Check that these samples are included:
mc16_13TeV.364172.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV0_70_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364175.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV70_140_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364178.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV140_280_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364181.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV280_500_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364158.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV0_70_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364164.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV140_280_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364167.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV280_500_BFilter.recon.AOD.e5340_s3126_r9364
mc16_13TeV.364119.Sherpa_221_NNPDF30NNLO_Zee_MAXHTPTV70_140_BFilter.recon.AOD.e5299_s3126_r9364
mc16_13TeV.364122.Sherpa_221_NNPDF30NNLO_Zee_MAXHTPTV140_280_BFilter.recon.AOD.e5299_s3126_r9364
mc16_13TeV.364125.Sherpa_221_NNPDF30NNLO_Zee_MAXHTPTV280_500_BFilter.recon.AOD.e5299_s3126_r9364
mc16_13TeV.364102.Sherpa_221_NNPDF30NNLO_Zmumu_MAXHTPTV0_70_BFilter.recon.AOD.e5271_s3126_r9364
mc16_13TeV.364105.Sherpa_221_NNPDF30NNLO_Zmumu_MAXHTPTV70_140_BFilter.recon.AOD.e5271_s3126_r9364
mc16_13TeV.364111.Sherpa_221_NNPDF30NNLO_Zmumu_MAXHTPTV280_500_BFilter.recon.AOD.e5271_s3126_r9364
mc16_13TeV.342195.MadGraphPythia8EvtGen_A14NNPDF23_QCDZbbjj_Incl.merge.AOD.e4191_s3126_r9364_r9315
mc16_13TeV.342196.MadGraphPythia8EvtGen_A14NNPDF23_EWKZbbjj_Incl.merge.AOD.e4191_s3126_r9364_r9315
25/06/18:
- Fix electron pt in data
- Add jet cleaning
- Investigate why electrons are killed
- Submit request for EWK V+jets extension, QCD filters
- New production
Code developments:
- PMGTool for LHEWeights: CentralDB
- https://gitlab.cern.ch/SusySkimAna/SusySkimDriver/commit/3fda81b0bab735f8a722f8df18ffa787251dc621 and https://gitlab.cern.ch/SusySkimAna/SusySkimMaker/commit/7b69a3e4743bef27ed33c09022df2f50690b78c7 -
22/06/18
Checsk for framework:
- Reimplement jet cleaning
- Check with other types of electrons, baseline, signal to understand the double peak
- Maybe overlap removal?
- Prepare for the new production
21/06/18
To do:
- Derivation request in 21.2.33.0
done in:
/nfs/dust/atlas/user/othrif/scratch/requests/EXOT5_cache21p2p33p0/listsI develped a script in - Ingrid specs for camera
done
- List of samples for derivaitons
done
- Request with filter in strong V+jets extension
- Request EWK V+jets with filtering
- Ntuple development: naming convention, Christian's improvements
- Run code and make plots
- Release 20.7 vs. rel21
- review SUSY EDborad: https://docs.google.com/presentation/d/1F0E4cbpHbh7rlY9OVl-FoC215CYFlY02diUa6Brd1Yw/edit#slide=id.g3c4bc60619_2_75
- EVNT-EVNT transform on Zvv
Questions:
- Why is ptV better than max ptv or HT?
- wasn't aware of PTV slices
don't need them as they are high the boson pt
VBF+MET meeting:
- calculate MET using the correct vertex
- solve the problem with the JetEtmiss group
- PFlow lower pt 20-30GeV
- Cutflow with PFlow and EMTopo
- OTP in fast simulation
- Double ratio method
- Janik: jet veto at arbitrary small thresholds
20/06/18
Preparing talk for V+jets on statistics issues:
Commands:
for i in $(cat ext.list); do ruciolist.sh mc15_13TeV $i* | grep EVNT; done
V+jets meeting: https://indico.cern.ch/event/737316/
19/06/18
Vincent PFlow vs. Topo: https://indico.desy.de/indico/event/20844/
- fix x-axis label
- split into light flavor and heavy flovour
- DeltaR 0.15
- poor tracking at high pT, split into intermediatry pt region to see the transition
- multiple effects in the plots, split into eta regions and pt regions, especially for Mjj
- increase in low pt of jet3, pt threshold in calibration, uncalibrated PFlow jet has higher pt than the calo jet > check with Michaela, check for uncalibrated jets
- see a shift on all pt distributions, possibly DeltaR is responsible
- why we expect more truth pflow jets than calo jets?
- MET perofrmance
- Apply a selection close to SR and CR
Chrsitian:
- showing events where a pileup jet has the largest sum of track pts (3rd jet in slide3) but outside of tracker therefor not jvt removed. will fjvt fix this issue? the problem is that truth match jets are tagged as pileup jets since they are not from index0
- vertex defined as sum pt, order vertices, all called primary vertices, can be up to 60
- s7: probably to be tagged as pileup by jvt, fraction of events having jvt < 0.59
Yu-Hen:
- check JVT scale factros for Data and MC in the JetEtmiss group
Ben Carlson on VBF summary: https://indico.cern.ch/event/737662/
- Open up the selection with systematics -
Nice artilce on TLA: http://cerncourier.com/cws/article/cern/71527
18/06/18
For derivation, i need to run it on 21.2.33.0 whenever it is built!
/eos/atlas/atlasscratchdisk/rucio/user/rzou/aa/5e/VBFHiggsInv_Ztot_bkg_fixednom.root /eos/atlas/atlasscratchdisk/rucio/user/rzou/cc/01/VBFHiggsInv_Wtot_bkgcomplete_fixednom.root /eos/atlas/user/m/mperego/HF_ntuples/ntuples_7dic2017/VBFHiggsInv_topbkg.root /eos/atlas/user/m/mperego/HF_ntuples/ntuples_7dic2017/merge_data.root /eos/atlas/user/m/mperego/HF_ntuples/ntuples_7dic2017/VBFHiggsInv_signals.root
xrdcp root://eosatlas.cern.ch//eos/atlas/atlasscratchdisk/rucio/user/rzou/aa/5e/VBFHiggsInv_Ztot_bkg_fixednom.root . & xrdcp root://eosatlas.cern.ch//eos/atlas/atlasscratchdisk/rucio/user/rzou/cc/01/VBFHiggsInv_Wtot_bkgcomplete_fixednom.root . &
13/06/18
Tasks for next week:
- Prepare plots for Thursday
- apply isolation
- data15/16 vs. mc16a
- do rel20.7 vs. rel21 comparisons
- Ntuple maker development for v02:
- forward JVT value and bool
- Trigger strategy implemented
- CR preselection
- SR preselection
- Chrsitian vertex index study
- fix naming of long container names (see SUSYSkim)
- gitlab CI run over samples
Status:
- logistics for vancouver
- Janik setup to run over all data and mc
12/06/18
PFlow vs Topo (Vincent):
- Split by jet flavor
- extend to forward jets 2.5 < eta < 4.8
- Jet Eta, Phi and Delta Phi not diveded by truth
- Pflow and calo are treating similarly for overlap removal
- how skewed distributions are
- truth jets vs (topo or pflow)
- are they calibrated to the same truth jet?
- how are muons treated in pflow, should they be added back
- grid direct in SampleHandler
11/06/18
To do:
- Fix long names looking at example from: https://gitlab.cern.ch/SusySkimAna/SusySkimDriver/blob/master/util/run_xAODNtMaker.cxx
- fix prw for files below
- resubmit jobs below.
- Writing DM talk
- Contacted PMG
- Talked to Guiliano:
- resubmit grid jobs
Grid babysitting:
Broken due to ANALY_AGLT2_SL7-condor:
mc16_13TeV.364129.Sherpa_221_NNPDF30NNLO_Ztautau_MAXHTPTV0_70_CFilterBVeto.deriv.DAOD_EXOT5.e5307_e5984_s3126_r9781_r9778_p3480
mc16_13TeV.364106.Sherpa_221_NNPDF30NNLO_Zmumu_MAXHTPTV140_280_CVetoBVeto.deriv.DAOD_EXOT5.e5271_e5984_s3126_r9781_r9778_p3480
mc16_13TeV.363356.Sherpa_221_NNPDF30NNLO_ZqqZll.deriv.DAOD_EXOT5.e5525_s3126_r9781_r9778_p3480
mc16_13TeV.361026.Pythia8EvtGen_A14NNPDF23LO_jetjet_JZ6W.deriv.DAOD_EXOT5.e3569_e5984_s3126_r10201_r10210_p3480
mc16_13TeV.426008.PowhegPythia8EvtGen_A14_NNPDF23LO_CT10ME_jetjet_JZ8.deriv.DAOD_EXOT5.e3788_s3126_r10201_r10210_p3371
mc16_13TeV.364253.Sherpa_222_NNPDF30NNLO_lllv.deriv.DAOD_EXOT5.e5916_e5984_s3126_r10201_r10210_p3371
mc16_13TeV.364200.Sherpa_221_NN30NNLO_Zmm_Mll10_40_MAXHTPTV70_280_BVeto.deriv.DAOD_EXOT5.e5421_e5984_s3126_r10201_r10210_p3371
mc16_13TeV.364147.Sherpa_221_NNPDF30NNLO_Znunu_MAXHTPTV70_140_BFilter.deriv.DAOD_EXOT5.e5308_e5984_s3126_r10201_r10210_p3480
resubmit with 'retry1' Commands to
retry1:runVBF.py @/nfs/dust/atlas/user/othrif/myBookKeeping/ntupleProduction/argsParser/v01/args_v01.txt --inputRucioLists=/nfs/dust/atlas/user/othrif/myBookKeeping/ntupleProduction/listProduction/v01/list_v01_EXOT5_retry1.txt -s submitDir_retry1 --version v01retry1 2>&1 | tee log_retry1.txt
Broken due to PRW: mc16_13TeV.364224.Sherpa_221_NNPDF30NNLO_Wmunu_PTV500_1000.deriv.DAOD_EXOT5.e5626_e5984_s3126_r9364_r9315_p3500 mc16_13TeV.364225.Sherpa_221_NNPDF30NNLO_Wmunu_PTV1000_E_CMS.deriv.DAOD_EXOT5.e5626_e5984_s3126_s3136_r9364_r9315_p3500 mc16_13TeV.364227.Sherpa_221_NNPDF30NNLO_Wenu_PTV1000_E_CMS.deriv.DAOD_EXOT5.e5626_e5984_s3126_s3136_r9364_r9315_p3500 mc16_13TeV.364226.Sherpa_221_NNPDF30NNLO_Wenu_PTV500_1000.deriv.DAOD_EXOT5.e5626_e5984_s3126_s3136_r9364_r9315_p3500 mc16_13TeV.364229.Sherpa_221_NNPDF30NNLO_Wtaunu_PTV1000_E_CMS.deriv.DAOD_EXOT5.e5626_e5984_s3126_r9364_r9315_p3500 mc16_13TeV.364224.Sherpa_221_NNPDF30NNLO_Wmunu_PTV500_1000.deriv.DAOD_EXOT5.e5626_e5984_s3126_r10201_r10210_p3371 mc16_13TeV.364226.Sherpa_221_NNPDF30NNLO_Wenu_PTV500_1000.deriv.DAOD_EXOT5.e5626_e5984_s3126_r10201_r10210_p3371 mc16_13TeV.364225.Sherpa_221_NNPDF30NNLO_Wmunu_PTV1000_E_CMS.deriv.DAOD_EXOT5.e5626_e5984_s3126_r10201_r10210_p3371 mc16_13TeV.364228.Sherpa_221_NNPDF30NNLO_Wtaunu_PTV500_1000.deriv.DAOD_EXOT5.e5626_e5984_s3126_r10201_r10210_p3371
Not clear what's wrong with these jobs: https://bigpanda.cern.ch/task/14360392/ https://bigpanda.cern.ch/task/14361068/ https://bigpanda.cern.ch/task/14361111/ https://bigpanda.cern.ch/task/14359530/ https://bigpanda.cern.ch/task/14360399/ https://bigpanda.cern.ch/task/14361130/ https://bigpanda.cern.ch/task/14361117/ https://bigpanda.cern.ch/task/14361131/
and these but seems a different problem: https://bigpanda.cern.ch/task/14360325/ https://bigpanda.cern.ch/task/14360221/
08/06/18
- cross section in files wrong
- My v01 MiniNtuple submission failed, now resubmitting MC16a only from
/nfs/dust/atlas/user/othrif/myBookKeeping/ntupleProduction/listProduction/v01/list_v01_EXOT5_missingMC16a.txt To run production, do:
runVBF.sh <version> <DAOD_Description>like:runVBF.sh v01 EXOT5_mc16cFast sim for forward jets: https://twiki.cern.ch/twiki/bin/view/AtlasComputing/FastCaloSimNewParametrization
07/06/18
To do:
EWK request
Exotics PMG contact? June 18th
VBF+MET meeting:
- Take events from MC16a, apply VBF, put in MC16d,e.
- pick up filter from jira of the large extension
06/06/18
To do:
- EWK request
- Plot rel21
- Dark matter talk
latex:\beta = \frac{\sqrt{\sum_i{w_i^2}}}{\sqrt{N_{\text{data}}}}\frac{L_{\text{data}}}{L_{\text{MC}}}
Prepared description of the problem i encountered with MC16 campaign names: working area is /nfs/dust/atlas/user/othrif/scratch/requests/310518
05/06/18
To do:
- MC statistics issue
done problem with the sub-campaign not labelled properly. Dave said ignore it for now.
- EWK sample size
- Vincent PFlow
- Janik run over Ntuples
- Fix triggers in ST
Jet energy scale and resolution: CMS:
- central jets 15–20% at 30 GeV, about 10% at 100 GeV, and 5% at 1 TeV
- uncertainties on the jet energy scale are below 3%
- barrel uncertainty 1% for pT > 30 GeV
04/06/18
Fix the single top samples where i have the AODs in scratch/requests/310518/tmp to be converted to EVNT: mc15_13TeV:mc15_13TeV.410642.PhPy8EG_A14_tchan_lept_top.evgen.EVNT.e6536
done
test area of janik:
/nfs/dust/atlas/user/ahnenjan/new_ntuples/Condorjobs/Condorout/temp/test
02/06/18
To read:
01/06/18
To do:
- summarize existing statistics
- auto config of PRW tool
- run tests
- submit all Data and MC
- Figure out statics needed to reduce EWK samples to 0.1 uncertainty
- propose to start using meeting room at DESY for the meeting
2L0J FAR:
- VRtop mT2 where it shows an excess, this is the region of interest for the anlaysis and it shows a mismodelling
- DF VR with 1 jet which get more ttbar but not lot of information about WW which you are trying to validate,
a large amount of Z+jets in the SF
Split to 0 and 1 jet
- You have a lot of events in the VRs and CRs, can't you move them closer to the SR.
- Try to get the CR and VR much closer SR, or you have to validate the extrapolation you do in several variables: mt2, met, met significance, ... in addition to pileup, new MC, release.
- Fakes validation, which fake rates are applied in data15,16,17,
To do, implement CI following this example: https://gist.github.com/kratsg/0e3f6382c3f9a59c652fe87e2b1cd7eb
- Run sample code in different MC campaigns, data
- Do cutflow comparisons
31/05/18
- Added rule: add-rule
mc16_13TeV:DAOD_EXOT5.13461026._000713.pool.root.1
Derivation Merge Request !11561 was merged on May25. Currently latest nightly is May23.
I can only run on MC16a and MC16d! no MC16c handling in my code!
- MC16a:
rucio list-dids 'mc16_13TeV:mc16_13TeV.*.*.merge.AOD.e*_*_r9364_r9315' - MC16d:
rucio list-dids 'mc16_13TeV:mc16_13TeV.*.*.merge.AOD.e*_*_r10201_r10210'
In summary we have:
r9364_r9315for MC16a,r9781_r9778for MC16c, andr10201_r10210'for MC16d- p3480 (skim), p3481 (data), p3482 (no skim)
Summary of discussion with Keisuke about Lepton Scale Facgtors
- See definition of Trig.Singlelep20XX in SUSYObjDef_xAOD.cxx
- You can modify it in the ST config under Trigger SFs configuration
For single trigger SF per object:
- You need to call GetTotalElectronSF
- https://gitlab.cern.ch/atlas/athena/blob/21.2/PhysicsAnalysis/SUSYPhys/SUSYTools/SUSYTools/ISUSYObjDef_xAODTool.h#L307
- Nothing to do other than
trigExpr="singleLepton"it will pick up the default in: https://gitlab.cern.ch/atlas/athena/blob/21.2/PhysicsAnalysis/SUSYPhys/SUSYTools/Root/SUSYObjDef_xAOD.cxx#L1049-1051 - Otherwise you can add the configuraiton to the ST config file if you want to remove a leg like this:
For di-lepton trigger SF per event:Trig.Singlelep2015: e24_lhmedium_L1EM20VH_OR_e60_lhmedium_OR_e120_lhloose || mu20_iloose_L1MU15_OR_mu50 Trig.Singlelep2016: e26_lhtight_nod0_ivarloose_OR_e60_lhmedium_nod0_OR_e140_lhloose_nod0 || mu26_ivarmedium_OR_mu50 Trig.Singlelep2017: e26_lhtight_nod0_ivarloose_OR_e60_lhmedium_nod0_OR_e140_lhloose_nod0 || mu26_ivarmedium_OR_mu50 - You only need to call GetTriggerGlobalEfficiencySF
- https://gitlab.cern.ch/atlas/athena/blob/21.2/PhysicsAnalysis/SUSYPhys/SUSYTools/SUSYTools/ISUSYObjDef_xAODTool.h#L300
- https://gitlab.cern.ch/atlas/athena/blob/21.2/PhysicsAnalysis/SUSYPhys/SUSYTools/Root/Trigger.cxx#L368
- It will calculate an event level SF which handle properly an OR between single and di-lepton SF
- In this case, don't call
30/05/18
A lot of work in getting the Ntuple maker ready:
- added R&S stuff
- electron and muon scale factors
- cross sections
- fixed the ST config file
- and other things ...
Now submit a pilot job in data and MC:
I see these errors:
ToolSvc.METMaker_SUSYT...WARNING Object is not in association map. Did you make a deep copy but fail to set the "originalObjectLinks" decoration?
ToolSvc.METMaker_SUSYT...WARNING If not, Please apply xAOD::setOriginalObjectLinks() from xAODBase/IParticleHelpers.h
ToolSvc.METMaker_SUSYT...WARNING Missing an electron from the MET map. Included as a track in the soft term. pT: 7519.36
`VBFInv.ST.ORToolBjet.M...FATAL /build1/atnight/localbuilds/nightlies/21.2/athena/PhysicsAnalysis/AnalysisCommon/AssociationUtils/Root/MuJetOverlapTool.cxx:221 (const Vertex ORUtils::MuJetOverlapTool::getPrimVtx() const): No primary vertex in the PrimaryVertices container!
VBFInv.ST.ORToolBjet.M...ERROR /build1/atnight/localbuilds/nightlies/21.2/athena/PhysicsAnalysis/AnalysisCommon/AssociationUtils/Root/MuJetOverlapTool.cxx:157 (virtual StatusCode ORUtils::MuJetOverlapTool::findOverlaps(const MuonContainer&, const JetContainer&) const): Failed to call "vtx != nullptr"
VBFInv.ST.ORToolBjet.M...ERROR /build1/atnight/localbuilds/nightlies/21.2/athena/PhysicsAnalysis/AnalysisCommon/AssociationUtils/Root/MuJetOverlapTool.cxx:136 (virtual StatusCode ORUtils::MuJetOverlapTool::findOverlaps(const IParticleContainer&, const IParticleContainer&) const): Failed to call "findOverlaps(static_cast
`
29/05/2018
To do:
- Get Ntuple code running
- Update code and submit new Ntuple production
27/05/2018
Now log commands everytime you send jobs to the grid
Truth produciton of Znn
vXXretry:
runVBF.py @./argsParser/args_vXXretry_Truth3_Znunu.txt 2>&1 | tee log_vXXretry_Truth3_Znunu.txt- Now i have a script, prepare the args and list files, then run:
runVBF.sh vXXretry_Truth3_Znunu
Create a new repostiroy to keep my book keeping easier and available in git: https://gitlab.cern.ch/othrif/myBookKeeping
26/05/18
- My work area for the Ntuple making is:
/nfs/dust/atlas/user/othrif/vbf/myVBF - Test files to run the code is:
/nfs/dust/atlas/user/othrif/samples/myRunTests - Bookkeeping area for ntuple production, sample lists, mc stats:
/nfs/dust/atlas/user/othrif/bookKeeping
25/05/18
Still need to do:
- read v5 WW SUSY
- reply to Valerie
fix truth code
correct high weight events in sherpa: set to 1 for
|w|>100done
- Send code to team
done
- document in twiki
done
- Modify code to include cleaning
decided not to do this and leave it for the analyzers
- compile MC stats in EVNT, TRUTH, MC16x
done
Currently studying: mc15_13TeV.364176.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV140_280_CVetoBVeto.evgen.EVNT.e5340
Some latex stuff:
\alpha\sum_i{w_i} \pm |\alpha|\sqrt{\sum_i{w_i^2}}
\alpha = \frac{L_\text{Data}}{L_\text{eff}}
N_\text{data} \pm \sqrt{N_\text{data}}
L_{eff} = \frac{\sum_i{w_i}}{\sigma\epsilon}
\frac{|\alpha|\sqrt{\sum_i{w_i^2}}}{\sqrt{\sigma\epsilon L_{\text{data}}}}
24/05/18
- Add flags to use at Cutflow level and add it to a spreadsheet:
https://docs.google.com/spreadsheets/d/1f3oW_cepPiYlOh_BJNviGTWO6lgxOzUfLOOB-cvS1B8/edit?usp=sharing
- Get EVNT, TRUTH, mc16a,c,d,e
Process new samples for Zvv
done
Make sure to tag the code in every production:
AB21p2p30_vXX.MC16aAB21p2p30_vXX.MC16dAB21p2p30_vXX.dataYYfile not complete at DESY, still running: user.othrif.vXX.366002.Sherpa_221_NNPDF30NNLO_Znunu_PTV100_140_MJJ0_500.e6669_p3135_MiniNtuple.root
Start production using JIRA to keep track of things: https://its.cern.ch/jira/browse/VBFINV-32
Maybe worth adding a table with details
23/05/18
- Why can't i run over full samples? showed up initially in the grid but present locally as well.
- tau's implementation not working, crashes in ST, implement later later
- Xsection tool in VBFInvTruth is not working
- Janik area:
/nfs/dust/atlas/user/ahnenjan/new_ntuples/Condorjobs/Condorout/temp
Remember when updating the AnalysisBase for docker, you change make a change in two locations: Dockerfile and .gitlab-ci.yml
22/05/18
To do:
- send with skim
- resolve exot5 pflow
- run on susy1
- implement CP groups recommendations
- advertise framework
Progress:
- duplicated EXOT5 that failed in the grid to DESY localgroup disk
- duplicated SUSY1 to local to test with
Done:
- Fixed EXOT5 adding PFlow b-tagging, skimming based an OR between Topo and Pflow, and extra PFlow variables
- Fixed problem with code crash in data (forgot to protect with
m_isMC) -
21/05/18
Problems to solve:
- PFlow in derivation not working
- Jet skimming based on Pflow and EMTopo
- Produce ntuples
- Jet cleaning variable for both PFlow and EMTopo
Problem with derivation and EMPFlow: See: https://twiki.cern.ch/twiki/bin/viewauth/AtlasProtected/HowToCleanJets2017#Derivation_Framework_Decorations I did:
static SG::AuxElement::Accessor<char> acc_eventClean("DFCommonJets_eventClean_LooseBad");
ANA_MSG_INFO ("Event cleaning: eventClean=" << acc_eventClean(*eventInfo) );
I get: ```bash myAnalysis INFO Event cleaning: eventClean=
you should do ``` bash
const xAOD::EventInfo *eventInfo = nullptr;
ANA_CHECK (evtStore()->retrieve (eventInfo, "EventInfo"));
static SG::AuxElement::Accessor<char> acc_eventClean("DFCommonJets_eventClean_LooseBad”);
ANA_MSG_INFO ("Event cleaning: " << bool acc_eventClean(*eventInfo) );
20/05/18
Time to wrap up the code implementation and produce ntuples.
19/05/18
Tried fixing the problem with PFlow derivation but without sucsses. Sent an email to Francesco.
18/05/18
Ntuple making:
- JETS:
Select jets with pt>25GeV and eta<4.5 and not bad, decorate them with: passJvt, passOR
JVT: • JVT cuts recommended are 0.11, 0.59 and 0.91 for loose, medium (default) and tight working points for EM Topo and LC Topo jets. • JVT cut recommended for EMPFlow jets is 0.20 for medium and 0.50 for tight working points. https://twiki.cern.ch/twiki/bin/view/AtlasProtected/JVTCalibration
Having problems switching between EMTopo and PFlow jet collections in Jet.InputType of ST
with the help of keisuke, we figured out that Jet.InputType=9, and that the derivation i am using have something wrong, just lunched a new derivation in the grid and also duplicating a susy derivaiton to check
weights that need to be implemnted: mc, pu, btag, elec, elec_trig, muon, muon_trig, jvt
For electrons/muons scale factors, for now i implement a generic function from ST, i am not dividing with respect to isolation, trigger, id , reco
For systematics:
The idea is to have the final objects outputed in the systematics tree. For now, we keep all the details for nominal studies.
11/05/18
Ntuple making:
- fix trigger info
- add cutflow
- revise impact parameters defined in monojet
done
- implement scale factors for triggers, leptons, jets
- isolation and reconstruction scale factors
- add met computed with invisible el, mu, ph
- add truthjets L1251
- add isBjet
- problem with trackmet
- Overlap removal with isolated objects
- MET calculation with define as invisible
- MET calculation and OR in ST
Now testing with
mc16_13TeV.410470.PhPy8EG_A14_ttbar_hdamp258p75_nonallhad.deriv.DAOD_EXOT5.e6337_e5984_s3126_r9781_r9778_p3480
since it has a good number of leptons and jets.
08/05/18
- Still ntuple making
07/05/18
Ntuple making code:
Pileup reweighting
- GRL, actualMu, lumicalc: http://atlas.web.cern.ch/Atlas/GROUPS/DATABASE/GroupData/GoodRunsLists/data17_13TeV/20180309/
- mc16d: https://indico.cern.ch/event/712774/contributions/2928042/attachments/1614637/2565496/prw_mc16d.pdf
- pileup studies: https://twiki.cern.ch/twiki/bin/view/AtlasProtected/PileupModellingInitiative
- Normalizing MC with different weights: https://indico.cern.ch/event/679306/contributions/2784652/attachments/1556491/2447942/PRWForMC16.pdf
- save weight for each event:
tool->expert()->GetPeriodWeight( eventInfo->runNumber(), eventInfo->mc_channel_number() );
04/05/18
Priorities for next week:
- Ntuple maker for VBF
- Produce Ntuples for the team
- Follow up on Statistics issues
- Study rel20.7 vs. rel21 comparisons
- Send comments on SUSY analysis
Status of projects:
Code development for the VBF Ntuple Maker:
- Location: /nfs/dust/atlas/user/othrif/scratch/myNewNtupleMaker/
- git: https://gitlab.cern.ch/othrif/STAnalysisCode
- Got to the point where I need to implement PRW and understand MC campaigns
Ntuple analyzer is setup:
- git: https://gitlab.cern.ch/othrif/myVBFAnalysis
- I need to work a bit more in implementing functions and methods from my old framework located in: https://gitlab.cern.ch/othrif/mySS3LAnalyzer
Monte Carlo strategy:
- What will be reconstructed from V+jets samples in MC16a/c/d/e/f
- Extend EWK samples by a factor x10
- There are two types of filter, the parton level filter and the truth jet filter. If we have a large sample of W or Z events that have not been reconstructed, we use the truth jet filter. If we have to start over, we use the parton filter.
Rel21 vs. Rel20.7 comparisons:
- Using xAH:
- rel21: working area is here
/nfs/dust/atlas/user/othrif/vbf/rel21/170418 - To run:
xAH_run.py --file $ASG_TEST_FILE_MC --config ${AnalysisCode_DIR}/data/VBFInv/vbf21.py --nevents=100 --submitDir=submitDir direct - To submit to grid:
xAH_run.py --files samples.list --inputRucio --inputList --config ${AnalysisCode_DIR}/data/VBFInv/vbf21.py --submitDir=submitDir prun --optGridOutputSampleName=user.%nickname%.%in:name[2]%.%in:name[3]%.%in:name[6]%.%in:name[7]%_xAH_mar14 --optRemoveSubmitDir=1 - rel20: area is here
/nfs/dust/atlas/user/othrif/vbf/framework_rel20p7/230118/AnalysisCode - To submit xAH to the grid use:
nohup ./cmd_xAH.sh unskim_rel20.list apr18_rel20_unskim.v0 apr18_rel20_unskim.v0 2>&1 | tee unskim.log - To run xAH on data for rel20:
xAH_run.py --file $disk/samples/EXOT5/data15_13TeV.00284285.physics_Main.merge.DAOD_EXOT5.r7562_p2521_p2950/DAOD_EXOT5.10298488._000347.pool.root.1 --config=$ROOTCOREBIN/user_scripts/VBFInv/vbf20p7_data.py --mode "athena" direct - To run xAH on MC for rel20:
xAH_run.py --file $disk/samples/EXOT5/mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.DAOD_EXOT5.e3909_s2608_s2183_r7772_r7676_p2949/DAOD_EXOT5.11691357._000001.pool.root.1 --config=$ROOTCOREBIN/user_scripts/VBFInv/vbf20p7_mc.py --mode "athena" direct - Using Giordon's tools:
- Located in my mac here:
/Users/othmanerifki/scratch/gstark/Rel21ScrutinyEffort
03/05/18
Notes on code development of VBFInvAnalysis:
- METDoTrkSyst vs. METDoCaloSyst in SUSYTools obj initialization -
02/05/18
I have a new repo for the Ntuple making code in: https://gitlab.cern.ch/othrif/STAnalysisCode
Nice read about regular expressions: https://docs.python.org/3/howto/regex.html
01/05/18
I have my framework to analyze Ntuples setup in: https://gitlab.cern.ch/othrif/myVBFAnalysis I need to work a bit more in implementing functions and methods from my old framework located in: https://gitlab.cern.ch/othrif/mySS3LAnalyzer
30/04/18
Bing's framework:
To run: analyze $myVBF_DIR/data/myVBFAnalyzer/inputlist $myVBF_DIR/data/myVBFAnalyzer/globalSetup.txt $myVBF_DIR/data/myVBFAnalyzer/histograms.txt Inclusive
For the new framework: run_hto4bLLP inputlist ../src/Hto4bLLPAnalysis/share/globalSetup.txt ../src/Hto4bLLPAnalysis/share/histograms.txt Inclusive SingleLepton DiLepton
28/04/18
- Process Ntuples to produce histograms > analyze
- runAllData.py:
- Submission with
pif you don't have filelist otherwise withoutp, then run it again with Hadd (merging) withm - Ntuples have to be in a specific location
- periods will be backgrounds in the case of MC
- mergeOutput.py:
- run after the merge
- here you normalize your samples
See localMergeConfig.py
Skim the tree to looser than you need and increase the speed to run over
27/04/18
To do for next week:
- Statitcal power of the current MC in SR, CRZ, CRW for Z+jets, Zvv+jets, W+jets for QCD/EWK
- Write the SUSYTools framework
- Produce first set of ntuples for the team, cutflow challenge
- Summarize the MC16 strategy and available samples
- Run Bing's analyzer code
- Rel20.7 vs Rel21:
- make sure it is the same events in data
- Test in Zvv, Wjets, Z+jets in SR/CRW/CRZ
25/04/18
Notes on the new framework for VBF.
Mono-jet is well written, so i will start from it and make my modifications. Some of the requirements i need are:
- ability to skim the ntuples on jet pt's, Deta, Dphi, Mjj, met, and so on
- wirebond potting
Notes on Module Loading:
- ITK week in september @ Oxford
21/04/18
Setup python correctly on my mac
19/04/18
Do rel20 vs. rel21 comparisons:
- Use xAH out of the box
- Write simple ntuple maker to do it step by step
18/04/18
Command for checking NTUP:
for i in $(cat unskim.list); do id=`echo $i | awk -F"." '{print $2}'`; echo "${id} `rucio list-dids mc16_13TeV:mc16_13TeV.${id}.*NTUP*r9364_r9315*/ --filter type=container | grep mc16 | awk -F"|" '{print $2}'`"; done
Release comparison studies
I am comparing unskimmed and data. rel21 submitted from: rel20 submitted from:
Skimmed vs. Unskimmed
I am submitting jobs for skimmed vs. unskimmed from s rel21 The EXOT5 prcessing is:
- test MC16 p3482 Unskimmed
mc16_13TeV.*.deriv.DAOD_EXOT5*p3482 - Bulk MC16 p3480 Skimmed
mc16_13TeV.*.deriv.DAOD_EXOT5*p3480 - Bulk data15
data15_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481 - Bulk data16
data16_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481 - Bulk data17
data17_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481
rel20.7 data: skim: unskim:
To submit xAH to the grid use:
nohup ./cmd_xAH.sh unskim_rel20.list apr18_rel20_unskim.v0 apr18_rel20_unskim.v0 2>&1 | tee unskim.log
To run xAH on data for rel20:
xAH_run.py --file $disk/samples/EXOT5/data15_13TeV.00284285.physics_Main.merge.DAOD_EXOT5.r7562_p2521_p2950/DAOD_EXOT5.10298488._000347.pool.root.1 --config=$ROOTCOREBIN/user_scripts/VBFInv/vbf20p7_data.py --mode "athena" direct
To run xAH on MC for rel20:
xAH_run.py --file $disk/samples/EXOT5/mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.DAOD_EXOT5.e3909_s2608_s2183_r7772_r7676_p2949/DAOD_EXOT5.11691357._000001.pool.root.1 --config=$ROOTCOREBIN/user_scripts/VBFInv/vbf20p7_mc.py --mode "athena" direct
17/04/18
To do:
- Produce truth plots with analyzer
- commit code
- check PFflow jets
Talk with Michaela:
- Check uncertainties in forward region with loose selection
- Check truth jets matched to PFlow or EMTopo jets
- Calculate MET with both jet collections https://indico.cern.ch/event/691744/contributions/2884336/attachments/1596016/2527893/uncertainties_preliminary_Moriond2018.pdf
Meeting about H>Invisible Combination: Run-2 workspace: /afs/cern.ch/user/w/weguo/public/llvv/LowMass/workspace/ ZHinv_2017_Paper Run-1 workspace: /afs/cern.ch/atlas/project/HSG6/HInv/v0/
- V+jets MC sliced in HT
- Highest slice at HT = 2.5 TeV (inclusive)
- V+jets split into EW and non-EW production
- EW more relevant at high mjj
- HT binning most helpful there
- Why LO? Theory: Precision SM backgrounds (Jonas) https://indico.cern.ch/event/669891/contributions/2813346/ attachments/1626134/2589821/JL_DMatLHC18.pdf
16/04/2018
To do for this week:
- make a plot quickly
done
- Run over truth
- check derivations EXOT5
done
- Run over derivation with ST code
- Produce rel20.7 vs. rel21 comparisons
- Evaluate MC statistics needs
- Update xAH
11/04/2018
To do:
- Upper limit impact
- Statistics status
Upper limit with HistFitter:
10/04/2018
Presented at Jet+MET on CMS results. The main feedback i got:
- V+jets Madgraph at LO + 4 legs is quite accurate, as a result MC stats is not an issue as they can generate many events while for us we get Sherpa NLO that is very slow and also comes with negative weights
- The Sherpa NLO should have a lower renormalization scale uncertainty but we are impacted by the statitics of the sample
- They choose to go for a mixed qqH and ggH signal almost at 50/50, this leads to a combination overlap up to 12% which leads to a 5% loss in expected exclusion sensitivity as a result they apply NO 3rd JET VETO.
Gaussian error propagation:
A +/- dA, B +/- dB, the ratio A/B +/- Sqrt(Power(dA/B,2)+Power(A*dB/(B*B),2))
09/04/2018
Spend the day reviewing the CMs result and making a nice summary of all the limits that exist: https://docs.google.com/spreadsheets/d/1UKwWUmNJoxXcy7MR3vq4JuMehypJsOG2Ea5-ADxvlx0/edit#gid=0
08/04/2018
Priority for this coming week is to prepare a summary of the VBF Full Run 2 analyis; challenges, what we did, what CMS did, and what we should be doing and addressing.
Also few items I need to catch up on:
https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.119.181803 https://gitlab.cern.ch/msaimper/ttDM_DESY_plotter
Local support meeting: https://indico.cern.ch/event/714155/
Update trigger menu for derivation: https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerPhysicsMenu#Physics_menu_for_2018 https://twiki.cern.ch/twiki/bin/view/Atlas/TriggerAPI
EB comments VBF: https://indico.cern.ch/event/716917/
AntiKt4TruthJets and AntiKt4TruthWZJets in derivation
07/04/2018
Added a path to exectuables in $HOME/bin/..., for now i have getPattern.sh. I plan to expand it to many scripts that I use everday. Also, i created this alias: grid. which sets up everything you need for the grid and if it doesn't have a proxy, it creates it.
06/04/2018
To do:
- Fix plot for Ben (minor)
- Look at truth level variables
- Add the orginal parton level info
- check that the lepton origin tells you where it is coming from
- Review the talks presented about VBF and prepare a reading material list
EB meeting with Hugo:
- Comparisons with run 1:
s/sqrt(b)gives you the sensitivity based on stat alone, whiles/sqrt(b+f*b)takes into account the systematic uncertainty where f is the percent - The conclusion is that going from run1 to scaled you don't loose, but if you apply run1 cuts to the run2 MC, you loose a lot, and with the new analysis cuts you recover the sensitivity
- Uncertainties between bins should not fluctuate too much
05/04/2018
Produced transfer factor uncertainty plots: (1+sigma_SR)/(1+sigma_CR) Huge improvement in the form of the reduction of the stat uncertainty due to the increase MC stat sample
03/04/2018
Back from vaccation.
Current goals: Higher level
- Understand the run2 ATLAS result
- Understand the run1 ATLAS
- Study the CMS run2 result
- Compile a summary of comments in circulation Technical
- Run on truth and produce histograms with
myVBFAnalyzer - Produce ntuples with
myNtupleMaker, run over withmyVBFAnalyzer
22/03/18
Working on package myNtupleMaker to produce ntuples
EB meeting:
- Main argument from Hugo is that the limit changed from 28% to 33% which is now problematic for him
Added the parton signal uncertainty, anti-id scale factor...
- we had an Exp(31%) and Obs(28%) in Run1 to Exp(33%) and Obs(31%) in Run2
The reason is that two analyses were combined in Run1 while not in Run2 -> do we want to design the most sensitve search in the full run2 paper based on these comments? i would say YES
- Why CMS is much better result?
For CMS, they are more agressive in dropping uncertainties that are otherwise important for us. Binning
- why is the JER is large (18% in the table)
it could be MC stats
Run the truth MiniNtuple maker with python $myNtupleMaker_DIR/data/OthAnalysis/TruthRun.py -c, add --nevents 100 for tests.
Some missing files in linking the mc TRUTH3 samples to my dust NEWLINKS: mc15_13TeV.364169.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV1000_E_CMS.deriv.DAOD_TRUTH3.e5340_e5984_p3401: missing 1 out of 20 files mc15_13TeV.364100.Sherpa_221_NNPDF30NNLO_Zmumu_MAXHTPTV0_70_CVetoBVeto.deriv.DAOD_TRUTH3.e5271_e5984_p3401: missing 2 out of 2 files. ... until 364113
21/03/18
Still on framework development to analyze ntuples
I need to fix these lines:
TH1F* cutflow = (TH1F*)currentFile.Get("cutflow");
TH1F* cutflowWeighted = (TH1F*)currentFile.Get("cutflow_weighted");
20/03/18
Get your code running to analyze VBF trees!
Code changes:
- ordering jets by pt needs revision: myVBFAnalyzer::sortIndices
nope it is fine! map is automatically ordered by the key
Code structure:
- Histograms names in the config file are used for the variables computed in the Looper.cxx
- Removed the skimming for now (produce skimmed output file)
command to run analyze $myVBF_DIR/data/myVBFAnalyzer/inputlist $myVBF_DIR/data/myVBFAnalyzer/globalSetup.txt $myVBF_DIR/data/myVBFAnalyzer/testhistograms.txt Inclusive
For the analysis, check the pt of the bosons since most of our events are at intermediate pTV around 150-250 GeV
Need to simplify the framework and run on my ntuple....
19/03/18
Reviewing Bing's framework:
- Advantages: quite flexible in building histograms, configuraiton file
- Disadbantage: need to write the analysis specific portion My old framework:
- probably quick to implement, but need to figure out if i can easily implement event.
Best is to do something in between, Bing's for the basic event processing then my setup for the analysis. Get to work...
Command to run the framework:
analyze inputlist $myVBF_DIR/data/myVBFAnalyzer/globalSetup.txt $myVBF_DIR/data/myVBFAnalyzer/dataHistograms.txt Inclusive
Updating the code: FourBJetAnalysis -> myVBFAnalyzer FourBAna -> myVBF run_4b -> analyze
I might be missing this sample from my list of files:``` mc15_13TeV.345596.PowhegPythia8EvtGen_NNPDF3_AZNLO_ggZH125_Zinc_HZZinv.evgen.EVNT.e6025 mc15_13TeV.308070.PowhegPythia8EvtGen_CT10_AZNLO_WmH125J_MINLO_jj_ZZinv.evgen.EVNT.e5831
`mc15_13TeV.308070.PowhegPythia8EvtGen_CT10_AZNLO_WmH125J_MINLO_jj_ZZinv.merge.AOD.e5831_s2726_r7772_r7676`
---
18/03/18
The EXOT5 prcessing is:
- test MC16 p3482 Unskimmed `mc16_13TeV.*.deriv.DAOD_EXOT5*p3482`
- Bulk MC16 p3480 Skimmed `mc16_13TeV.*.deriv.DAOD_EXOT5*p3480`
- Bulk data15 `data15_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481`
- Bulk data16 `data16_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481`
- Bulk data17 `data17_13TeV.*.physics_Main.deriv.DAOD_EXOT5*p3481`
---
15/03/18
I have to replicate these samples: ${disk}/vbf/rel21/060318/run/run_grid/retry1.list
with
`xAH_run.py --files retry1.list --inputRucio --inputList --config ${AnalysisCode_DIR}/data/VBFInv/truth.py --submitDir=submitDir_retry prun --optGridOutputSampleName=user.%nickname%.%in:name[2]%.%in:name[3]%.%in:name[6]%.%in:name[7]%_xAH_mar14_retry --optRemoveSubmitDir=1`
Normalization to EVNT samples:
` norm = L_data * cross section * eff / Sum of weights`
---
S / sqrt(B) -> 10S / sqrt(10B) = 3 S/sqrt(B)
S / sqrt(B) -> 5S / sqrt(5B) = 2.2 S/sqrt(B)
Naively, expect 0.25/2 -> 0.13
0.13 -> 0.17 is due to the W->Z xfer factor uncertainty -> Zll stats
---
14/03/18
We missed you around here! well last week was at RAL, see the hardware notebook.
To do:
- Define the MC samples to produce
- Determine the MC stats from truth
- Determine the stats for mc16a and mc16d for the sample extensions
MC:
W/Z extensions:
99999500 mc15_13TeV.364310.Sherpa_222_NNPDF30NNLO_Wenu_MAXHTPTV70_140.evgen.EVNT.e6209
99998000 mc15_13TeV.364311.Sherpa_222_NNPDF30NNLO_Wmunu_MAXHTPTV70_140.evgen.EVNT.e6209
99995500 mc15_13TeV.364312.Sherpa_222_NNPDF30NNLO_Wtaunu_MAXHTPTV70_140.evgen.EVNT.e6209
We already HAVE:
51891000 mc15_13TeV.364173.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV70_140_CVetoBVeto.evgen.EVNT.e5340
mc15_13TeV.364174.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV70_140_CFilterBVeto.evgen.EVNT.e5340
mc15_13TeV.364175.Sherpa_221_NNPDF30NNLO_Wenu_MAXHTPTV70_140_BFilter.evgen.EVNT.e5340
52369500 mc15_13TeV.364159.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV70_140_CVetoBVeto.evgen.EVNT.e5340
mc15_13TeV.364160.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV70_140_CFilterBVeto.evgen.EVNT.e5340
mc15_13TeV.364161.Sherpa_221_NNPDF30NNLO_Wmunu_MAXHTPTV70_140_BFilter.evgen.EVNT.e5340
52499500 mc15_13TeV.364187.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV70_140_CVetoBVeto.evgen.EVNT.e5340
mc15_13TeV.364188.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV70_140_CFilterBVeto.evgen.EVNT.e5340
mc15_13TeV.364189.Sherpa_221_NNPDF30NNLO_Wtaunu_MAXHTPTV70_140_BFilter.evgen.EVNT.e5340
---
05/03/18
- I have run Histfitter and put all my notes in ATLAS framework. I have problems to be resolved with Peter.
- Time to run the test with TRUTH3 and xAH code in truth mode
---
03/03/18
Let's get a framework running
---
Steps to running Histfitter:
python ./runWinter2015SameSign.py fit python ./runWinter2015SameSign.py merge python ./runWinter2015SameSign.py plot
Prepare/copyTree.C, to create histfitter input files (from ntuples), files i have from Peter are the output of this step
----
28/02/18
To do:
- Play with the python libraries for openCV
---
24/02/18
Working with openCV:
- Seems to be useful scripts in `samples/python2/asift.py` and `samples/python2/find_obj.py` under `/Users/othmanerifki/itk/jens/tutorial/opencv`
---
23/02/18
To do:
- check the message: "HelperFunctions.getPri...WARNING No primary vertex was found! Returning nullptr"
> done not important
- check that all triggers are in
> looks ok
then confirm derivtion
> sent a confirmation email
* Size of the new EXOT5:
events reduction size reduction event size [kb]\\
14.3% 2.4% 46.1
---
22/02/18
To do:
- Request truth derivation
- Fix xAH rel21 code and commit
- Setup SUSYTools code
What are the VBF analyses in ATLAS?
21/02/18
To do:
- Request truth derivation
- Request reco of missing samples
- Review Kathrine's material
- Run OpenCV
- Update systematics
Asked Peter for the full SS/3L setup:
- Code here: https://gitlab.cern.ch/disimone/SSHistHitter
- putting 50Gb of inputs needed in the grid now...
MC request:
- EVNT level are mc15, but the subsequent steps are from mc16
- If you find more than two datasets than you need to combine
- Quite a useful script to find the samples given a DSID and campaign `cernbox/myScripts/getMC16Containers.py`
- Now we add mc16a, mc16c, mc16e with the appropriate luminosity weights
- For full Run2 add mc16a, mc16c/d, and mc16e with the appropriate luminosity weights
Example:
1.mc16_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.EVNT.e3909_e5984
2.mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.evgen.EVNT.e3909
3.mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.evgen.EVNT.e4397
Talk with Dave:
- You should not combine mc samples with different etags
- Tags start as eXXX_sXXX_rXXX. You can care about the first e, first s, first r. if they are repeated, this is due to merging
- For example for mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.evgen.EVNT.e3909:
* contians 2 tids: 13162805 and 12944917
* it was merged as mc16a to give: mc16_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.EVNT.e3909_e598
>
Manipulations to get DAOD_TRUTH
for i in $(cat tmp); do if [[ $i = "mc16" ]]; then x=rucio list-dids $i | grep mc16 | awk -F"|" '{print $2}' | awk -F":" '{print $2}'; id=echo $x | awk -F"." '{print $2}'; out=rucio list-dids mc16_13TeV:mc16_13TeV*${id}*TRUTH*/ --filter type=container | grep mc16 | awk -F"|" '{print $2}' | awk -F":" '{print $2}'; echo "$id > $out"; fi; done
To get the campaign:
```bash
for f in `rucio list-content mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.evgen.EVNT.e3909/ | grep mc | awk '{print $2}'`; do rucio get-metadata $f | grep camp; done
20/02/18
To do:
- Run Histfitter
- Run OpenCV
19/02/18
Meeting with Ingrid:
- Work on the software aspect with Vancouver
- Order tools for the gantry: discuss with Sergio and Ingrid
Run:
git push --set-upstream origin VBFrel21 on /nfs/dust/atlas/user/othrif/vbf/rel21/190218/AnalysisCode
To do:
- Review EOS board session
done
- Review Kathrine's material
18/02/18
Week to do:
- Merge request for EXOT14, EXOT5
- Generate MC truth TRUTH derivation?
- Software framework
- Histfitter code
- MC truth checks
- Review codes from: Kate, Daniele, Francesco
- Decide on which approach to follow for module loading software
Luminosity plans for run 2: 2015: 3.2/fb 2016: 33.2/fb 2017: 43.8/fb 2018: 68/fb Total Expected Run 2: 148/fb
15/02/18
duplicate mc channels between Martha and Rui: 364103 364112 364122 364129 364129 364150
Specifications for the camera: working distance 0.75XL pressure sensor bright led main:
BNL_TMStave.lvprojGood place to start:https://github.com/sciollalab/BNL_ThermomechanicalStave/blob/master/Utils/Standalone/ContinousCameraStageV2.viPlans for VBF
https://indico.cern.ch/event/705499/contributions/2894850/attachments/1601412/2538996/Carlson_FullRun2_February15_2018.pdfExpect 150 1/fb-1 MC samples, writing some code at truth level Late fall or early 2019 on the V+jets Mjj slicing what are the actual plans for a good JES resolution for this year Contribute to jet uncertainties that matter to us Add new SR at low MET q/g tagger to identify tag jets flavor composition -> assumption of a given falvor composition, need to derive specific uncertainties to our analysis Central jet veto PFlow jets calibrated to 20GeV, calibrate to 15GeV The lower the 3rd jet pt threshold, the better it is drop access leptons in transition region Tau veto, but PFlow and track jets is better Truth LO sherpa samples In the past, fJVT will only get worse at higher pileup, studies say it is not true, perhaps extend with primary vertex assignment, be possible to study with MET and tag jet Tail from distributions is from jet removal, it should compensate for jets that were removed
Apply Deta for all trigger events factor of 3 reduction PFlow jets in SUSY11, leptons to veto, multijet triggers should not included 30/40 Hz, Can we reduce uncertaintes to forward jets PUPPY constituent subtration
rel21:
data16_13TeV.00311481.physics_Main.merge.AOD.r9264_p3083
mc16_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.AOD
mc16_13TeV.308276.PowhegPy8EG_NNPDF30_AZNLOCTEQ6L1_VBFH125_ZZ4nu_MET125.merge.AOD
rel20.7:
data16_13TeV.00311481.physics_Main.deriv.DAOD_EXOT14.r9264_p3083_p3213mc15_13TeV.301399.PowhegPythia8EvtGen_CT10_AZNLOCTEQ6L1_VBFH125_ZZ4nu.merge.DAOD_EXOT14.e3909_s2608_s2183_r7772_r7676_p3273
14/02/18
Progress:
Input file: data16_13TeV.00311481.physics_Main.merge.AOD.r9264_p3083/AOD.11038623._000002.pool.root.1
Size size/Event # Events
4.178 Gb 331.758 kb 12588
EXOT5 Nominal:
73.11 Mb 43.363 kb 1686
EXOT14 Nominal:
144.98 Mb 40.781 kb 3555
Questions: X Size of the nominal EXOT14 and EXOT5 What happens if i add all things I want to EXOT14 How can I reduce EXOT14 to 2%? X What will happen if i remove the mjj cut on EXOT5 Can I apply some filtering to EXOT5? What happens if i add PFlow jets to EXOT5? What happens if I add the di-lepton triggers? How about checking the SR, VR, and CRs?
Answers: working in /nfs/dust/atlas/user/othrif/vbf/derivation/140218/sandbox/
- Derivation EXOT14:
- passThrough Evt Red Size Red Siz/evt Evtents 100.0% 13.5% 44.5 1000
- JetPts 50,35: 27.2% 3.6% 43.3 272
- JetPts 40,40: 27.0% 3.6% 43.8 270
JetPtsDEta 23.4% 3.0% 41.3
Derivation EXOT5: add particle flow, di-lepton trigger
- passThrough 100.0% 13.1% 43.2 1000
- JetPts 40,40 11.7% 1.8% 49.2 117
- JetPts 40,40 and no Mjj cut 14.4% 2.2% 49.6 144
- JetPts 40,40 and max Mjj cut 14.1% 2.2% 49.9 141
- JetPts 40,40 and max Mjj cut and particle flow jets 14.1% 2.5% 57.7 141
- JetPts 40,40 and max Mjj cut and particle flow jets and dilepton 14.8% 2.6% 57.4 148
JetPts 40,40 and max Mjj cut and particle flow jets and dilepton and deta cuts 11.9% 2.2% 58.6. 119
Check rel20.7 vs. rel21
Strips module session Wednesday 9-10:30
13/02/18
Chats with Jens & Bernd:
Chats with Kathrine, Francesco, and Daniele:
Path to opencv:/usr/local/Cellar/opencv
Derivation work: When you are doing checks, running on data is enough: data16_13TeV.00311481.physics_Main.merge.AOD.r9264_p3083
- EXOT 14:
- Feb12_JetPts_v0 JetPts EXOT14 28.2% 4.2% 40.9
- Feb12_JetPtsDEta_v0 JetPtsDEta EXOT14 23.7% 3.3% 38.1
- Feb12_JetPtsDPhi_v0 JetPtsDPhi EXOT14 25.2% 3.6% 39.4
- Feb12_JetPtsDEtaDPhi_v0 JetPtsDEtaDPhi EXOT14 23.1% 3.2% 37.7
- Feb12_JetTrig_JetPts JetPts EXOT14 26.1% 3.9% 27.4
- EXOT5:
- Feb12_Nominal Nominal EXOT5 13.4% 2.1% 43.6
- Feb12_NoMjj NoMjj EXOT5 16.6% 2.6% 43.1
- Feb12_Mjj_max max EXOT5 23.1% 3.0% 24.0
12/02/18
- The AthDerivation latest with 21.2.15.0 is not working, revert back to
asetup 21.2.14.0,AthDerivation
Run new derivation tests with: data and high stats VBF
- EXOT14:
- Feb12_JetPts_v0
- Feb12_JetPtsDEta_v0
- Feb12_JetPtsDPhi_v0
- Feb12_JetPtsDEtaDPhi_v0
- Feb12_JetTrig_JetPts
- EXOT5:
- Feb12_Mjj_max
- Feb12_NoMjj
- Feb12_Nominal
Module Loading meeting:
- Labview system works, OpenCV based on c++ doesn't
- Pattern recognition
- Debate on either to go on C++, lab-view
- Work on Prototye bridge to load the module in the
- Camera auto-focusing: use Laplacian, standard deviations and fitting
- Urgent problem needs attention
- https://indico.cern.ch/event/694189/contributions/2885385/attachments/1598758/2533800/fiducial_proposals.pdf
- patter recognition, automated zoom lense, change or picture stack
- BNL: Laura Burgsten Brandeis,
11/02/18
Mysteries:
- running the derivaiton on this dataset works:
data16_13TeV.00311481.physics_Main.merge.AOD.r9264_p3083while this dataset doesn't:data16_13TeV.00307732.physics_Main.merge.AOD.f741_m1673Error message:PyJobTransforms.transform.execute 2018-02-11 11:06:31,228 WARNING Transform now exiting early with exit code 65 (Non-zero return code from AODtoDAOD (64); Logfile error in log.AODtoDAOD: "ToolSvc.DFJetCalib_AntiKt4EMTopo FATAL Could not retrieve xAOD::EventShape from evtStore.")
09/02/18
Derivation tests
- EXOT14:
- Test 1: cuts on di-jet variables with jet pt>40 GeV
- Feb9_passThrough_v0
- Feb9_JetPts_v0
- Feb9_JetPtsDEta_v0
- Feb9_JetPtsDPhi_v0
- Feb9_JetPtsDEtaDPhi_v0
- Test 2: cut on the two leading jets variables
- Feb9_Lead_JetPtsDEta_v0
- Feb9_Lead_JetPtsDPhi_v0
- Feb9_Lead_JetPtsDEtaDPhi_v0
- Test 3: add jet triggers
- Feb9_JetTrig_JetPts_v0
- EXOT5:
- Feb9_passThrough_EXOT5_v0
- Feb9_Nominal_EXOT5_v0
- Mjj modified LEFT
- EXOT2:
- Feb9_passThrough_EXOT2_v0
- Feb9_Nominal_EXOT2_v0
- Add triggers LEFT
To do:
- Derivation: EXOT14, EXOT5, EXOT2
- EXOT4: do test with two leading jets, and any jets above 40GeV
- Define 3 regions
08/02/18
- Updated
21.2-updateRel21EXOT14-2018-01-26to the latest version of code to be pushed, keep this for the official merge - Creat a new branch with tests i am performing
21.2-testsEXOT-2018-02-08
To do:
- Run derivation eta cuts in EXOT14:
- Define 3 regions:
- SR
- 2j VR: SR except |DeltaEta(jj)| > 2.5, Mjj < 1 TeV
- QCD VR: 1.8 < DPhi < 2.7, 3rd jet 25 < pt < 50, DEta > 3 leading jets, MET > 100, Mjj > 600 GeV
- Modify l114 of EXOT5 run test
Questions:
- do you really need to lower the jets pt?
we could survive but we need to change max Mjj for any jet combination
- derivations are included all the information that you need for performing the studies for your high eta jets
need to check by running xAH on
07/02/18
Progress:
- Submitted jobs for EXOT14 with DPhi added
realized a mistake with version v0, resubmitting with v1
- Changed pt cuts of EXOT5 to 50,35, and monojet cut to 50
Called
skim_JetPt50_35_Mjj150.v0forgot to add EXOT5 in the name - Added a list of SUSY11 triggers to EXOT14
- Fixed
2e*and2mu*in trigger thinning - Download grid output:
rucio download user.othrif:user.othrif.*v0_EXT0/ - I put all of the strings I need in
$disk/samples/DAOD/sample.listthen dofor i in $(cat sample.list); do rucio download $i; done - Created a list of files i produced for derivation tests
DAOD/base_samples.list
Manipulations worth saving:
- To extract files without log or EXOT0
ls | grep user.othrif. | sed -e 's/.v[0-9]_EXT0//g' | sed -e 's/.v[0-9].log//g' | uniq - Run the checkAODtoDAOD script with
for i in $(ls -d user.othrif.*); do echo "Processing $i"; python checkAODtoDAOD.py -g -f $i/log.AODtoDAOD; done 2>&1 | tee ouptut_check.log
To do:
- DPhi cut at derivation
- Run with jet triggers in
- Setup comparison of rel20.7 and rel21
- Run Histfitter code
06/02/18
To do:
- Run with DPhi cut at derivation level
- Setup comparison of rel20.7 and rel21
- Check the size of EXOT14
05/02/18
Progress:
- Fix derivation framework on the grid
done
- Fix the list of triggers needed
done
- Fix the AllVariables list
done
- Set meeting with Kate, Francesco, and Daniele
- Add PFlow jets and timing information
done
- Lunch some derivation jobs with various skimming cuts in data and MC
- data/signal with passthrough: passThrough.v0
- data/signal with jet pt cuts: skim_ptj1j2cuts.v0
- data/signal with jet pt cuts and DEta: skim_RequireJetsDEta.v0
- data/signal with EXOT5 passThrough:
- data.signal with EXOT5 skim nominal:
Week goals:
- Update EXOT14 derivation framework
- Rel20.7 vs. Rel21 comparisons
- Run SUSYTools framework
- ITK reading
To do:
- Fix derivation framework on the grid
- Produce small derivation for data files
- Run full production in the grid
- Produce data rel20.7/rel21
Questions:
- Can we add a single jet trigger to the derivation so that we don't rely on SUSY11?
- how to find relative sizes of xAODs?
02/02/18
- Plot theory uncertainties
- I am added to the int note of Run2
01/02/18
To do:
- run data, MC in rel20.7 and rel21
- Do derivation tests
- Talk at 3pm?
- Investigate trigger cutting more events
Problems: ALWAYS PUT THE SCOPE!! when using rucio!
Meeting VBF:
- It is not clear that the MC stat is enough for 100 fb-1
- Need to check the stats: 200-300 1/fb
- Implement SR/CRs and check statitical uncertainties with the new sliced samples
- What are the dsids of these events?
- New version of sherpa? Modeling undertainty? -
31/01/18
To do:
- Fix rel comparisons with passThrough flag
done
- Now setting up data and MC for rel20.7 and rel21 to do detailed comparisons
- Run into a problem running over data
Contacted xAH developers, Resolved, needed to force
m_forceInsituto false since Insitu calibrations are not available, but they will be - Produced rel21 data EXOT14 with passThrough flag from
data16_13TeV.00311481.physics_Main.merge.AOD.r9264_p3083/AOD.11038623._006398.pool.root.1, file located here:/nfs/dust/atlas/user/othrif/vbf/derivation/300118/sandbox/run_data - Producing rel20.7 data EXOT14 in the grid
- Generaring derivations of all MC files locally in rel21
- Now I have:
- rel20.7: Anaoutput for MC to hist-*, Grid job running for DATA hist-
13108059 - rel21: Derivations EXOT14 are being produced in passThrough for MC, Anaoutput for VBF and Data produced in passThrough
30/01/18
To do:
- Get simple code to test derivation running
I am trying running passthrough on, this was the culprit
Reco_tf.py ... --passThrough 1You can always check in ami what exact flags are set with a p-tag https://ami.in2p3.fr/app?subapp=tagsShow&userdata=p3273
29/01/18
To do:
- Read chapter 9
done
- Submit a merge request for derivation
done
- do derivation tests
- Include triggers
- Fix allvariables in content list
- Add jet timing info and PFlow jets
26/01/18
First things first:
figure out why met is different in rel20.7 vs. rel21
you can run the Histfitter code in https://gitlab.cern.ch/VBFInv/StatsTools
- listened to talk about http://pytorch.org/
25/01/18
- Presented at VBF meeting
- resolve derivation problems: MET plot, fix trigger, derivation in grid, deltaPhi and DeltaEta cuts
- derivation:
- goal is 30TB for all data and MC
- Try a cut of Dphi of 2.7
- passthrough, skimmed, unskimmed
- how far from EXOT5 can we go
- work on SUSYTools version of the analysis: Mono-jet, MultiB frameworks
- run data/MC rel20.7 vs. rel21 comparisons
- Athena style SUSYTools
24/01/18
- Meeting with Local Support:
- Work on Sheffield on Sleeved technique for pipe welding: Jimmy interested in welding
- Welding between what and what?
Nice talk about the construction of the petal
Produce rel20.7 VBF analysis files
- Produce EXOT14 rel21 derivations
- Produce rel21 VBF analysis files
- Plot comparisons of VBF signal, Z, and W in inclusive, SR, CR
- Prepare presentation
23/01/18
- Run comparison rel20.7 vs. rel21
- Get the Ntuple reader running from Bing
- Run on full ttbar, Z > nu nu, VBF
- Got rel20.7 and rel21 running to produce microtuple
- Now develop code to analyze them
Chat with Jens:
- Jens visit March 7-8th
22/01/18
- Goals of the week:
- Continue reading ITK
- Talk to Jens, schedule a visit
- Run derivation
Do rel20.7 vs rel21 comparisons
Work with Thomas to resolved the cernbox issue. No solution yet but here is what we have done:
condor dust cd scratch/condor_example/ emacs myjob.submit # change to `requirements = OpSysAndVer == "CentOS7"` condor_submit -i myjob.submitHowever,
cvmfswas not in the path. We couldn't getgfalFSnorvoms-proxyThomas will see and get back to me. Instructions (not complete) here: https://confluence.desy.de/pages/viewpage.action?pageId=81976453
19/01/18
- Work on derivation, getting running step by step
- how to check that you are picking up the correct dervivation when modifying EXOTxx.py
- very slow, problems with derivation... sent an email to Ben
18/01/18
- Work on derivation, already running into problems in first attempt
- Add "AntiKt4EMPFlowJets" to reducedJetList in EXOT14, but only do one change at the time
- Having problems with running the derivation
- going through the derivation framework tutorial to know what i am doing
17/01/18
- Update the configuration files with the latest recommendations
Done
- Work on the derivation
- Run Bing's framework to analysize ntuples
- sometimes the ssh from lxplus to hep1 requires password, other times it doesn't, strange?!?
16/01/18
- Still problem with xAH and MuonEfficiencyCorrector, opened an issue with xAH community
turned out to be a bug. Now all fixed
- Got the xAH running
- I need to work on derivation
15/01/18
- Continue running xAH framework
- Problem with PlotVBFInvEvent hanging
Generating an unrealistic list of histograms, set m_limit False.
- No trigger recommendations yet for rel21
- There is a problem with MuonEfficiencyCorrector, emailed expert. -
12/01/18
- Get the config file running with basic recommendations
11/01/18
- In process of running the xAH framework
Give a bit more descriptif titles than "done for
" - Dealing with pileup reweighting problem
the MC campaign and which pileup files you are using matters!! you know this...
- Now it is time to create your final objects using the latest recommendations, then run the analysis code on them -
10/01/18
- Bad start with git problems since i am using cernbox and pushing/pulling from my laptop and lxplus -> BAD. Now, only push/pull from my laptop and use the lxplus cernbox as a soft link ONLY.
- For dealing with shared data, you have few options:
- Option1: Add
atlas_install_data( share/*.py )andatlas_install_data( data/* )to your CMakeLists.txt then access it via in your python script viapath_config_files=ROOT.PathResolverFindCalibDirectory("VBFInv/") - Option2: In your stearing macro you can just add the string, then do the heavy lifiting in your c++ code by adding
#include "PathResolver/PathResolver.h"andstd::string fullPathToFile = PathResolverFindCalibFile("some/file.root")as explained in https://twiki.cern.ch/twiki/bin/viewauth/AtlasComputing/PathResolver - Code runs wnere I replace
$ROOTCOREBIN/data/VBFInv/GRL_v88.xmlwithpath_data_files+"/GRL_v99.xml - It seems
m_runOverSkimAODbreaks the code for some reason c.setalg("JetCalibrator", JetCalibratorDict)is giving trouble: ''' Attempt to retrieve nonexistent aux data item::JVFCorr'''
- Dealing with pileup reweighting problem
09/01/18
- Added all the libraries from Rootcore. code compiling fine.
- Reviewing the code:
- we have algorithms that get added via xAH_config and configured using XXXDict.
- the baseline code that defines all the Dict is in vbfinv_baseline20p7.py
- Then these dictionaries are added to the algorithms via
c.setalg(algname, algDict)in vbf20p7.py - - The location of the build is
$myVBFrel21_DIRand the architecture folder inside the build folder is$AnalysisBase_PLATFORM-
08/01/18
- Received answers from Attila and Max. Apparently one can use FTP with Sublime to access remote repositories: -https://wbond.net/sublime_packages/sftp
- Setting up VBF rel20.7 code with rel21
- Run into problem in the latest commit by Ben, reverted to a297bd0017f72e0afa855afc8a56c29c55f764eb, and emailed Ben
Solved Ben forgot something.
- Keeping a rel20.7 working and setting up a rel21 version in parallel
- Commit rel21 to git to start developement of CMakeLists.txt for VBFInv package: https://:@gitlab.cern.ch:8443/othrif/myVBFrel21.git
- Problem with .gitlab-ci.yml "fatal: unable to access xAODAnaHelper". Still couldn't resolve it. Contacted Attila.
Giordon gave the answer to add
GIT_SSL_NO_VERIFY: "true" - Problem with running the rel20.7 version of the code.
Due to wether you are running on MC or data, Change L11 to MC, and set L13 to false in scripts/vbf20p7.py
- Some advanced tools using Docker where one can run the full analysis locally:
- https://hub.docker.com/r/ucatlas/xah/
- https://hub.docker.com/r/ucatlas/analysisbase/tags/
05/01/18
- Collected more questions on the tutorial and completed it
04/01/18
Questions on tutorial: https://atlassoftwaredocs.web.cern.ch/ABtutorial/basic_xaod_modify/
- Have AnalysisBase know by IDE like sublime
- How to change
setMsgLevel (MSG::INFO);from config file - Deep copy
recordgives error:Trying to overwrite object with key "GoodJets" - "jets_shallowCopy" should be "shallowCopy"
- change between 0 and nullptr in the declarations
- Sometimes everything is running with "AnalysisJob_run.py" and all the sudden i get:
-bash: /afs/cern.ch/user/o/othrif/workarea/myRel21tutorial/build/x86_64-slc6-gcc62-opt/bin/AnalysisJob_run.py: /usr/bin/env: bad interpreter: Permission deniedbut if I dopython x86_64-slc6-gcc62-opt/bin/AnalysisJob_run.pyit works fine. I tried logging out and logging back in, removing everything and starting from scratch but still run into this problem. I resolved it by runningchmod a+x x86_64-slc6-gcc62-opt/bin/AnalysisJob_run.pynot sure why it worked before without the need of this step. Now i run:make && chmod a+x x86_64-slc6-gcc62-opt/bin/AnalysisJob_run.py && rm -rf ../run/* && AnalysisJob_run.py -o ../run/t - How to make scripts like "AnalysisJob_run.py" exectubales appearing in bin/.
- I get the error when wanting to add a tree
error: invalid use of incomplete type 'class EL::Worker' TFile *outputFile = wk()->getOutputFile (m_outputName);It was just missing '#include
'. - Is it normal that a package compiles fine locally, but in Gitlab docker, it fails. This happened for :
/builds/othrif/myRel21Template/AnalysisPackage/AnalysisPackage/AnalysisAlgorithm.h:19:10: fatal error: 'MuonMomentumCorrections/IMuonCalibrationAndSmearingTool.h' file not found
03/01/18
- I continued working through the analysis tutorial.
- Did other bureaucracy things.
- Checked the position in CNRS. It opens the first week of December and is due the first week of January.
02/01/18
- I have created a website to have a web based access to my logs: https://gitlab.com/othrif/othrif.gitlab.io
- I created a new repo: https://gitlab.cern.ch/othrif/myRel21Template to work on the tutorial https://atlassoftwaredocs.web.cern.ch/ABtutorial/basic_messaging/
- I have setup cernbox and sublime so that anychanges i make with sublime can be compiled in lxplus without the need of git push or scp.