The purpose of this cheat sheet is to log all the tricks and tools I use that are rather general and useful for computing. Such as Condor, bash short scripts, and so on.

Table of Contents generated with DocToc

Condor commands:
Kerberos:
Git
```
      - [Git tips](#git-tips)
```
Mounting CVMFS on your mac:
Sublime:
```
      - [Sublime tips:](#sublime-tips)
```

      - [Run gitbook:](#run-gitbook)
      - [Gitbook tips:](#gitbook-tips)

NAF on your MacOS

root version

lsetup "root 6.06.04-x86_64-slc6-gcc49-opt"

Bash tips:

If you want to find matches in the same line and get the word that comes after the space:

echo "blabla THIS ... a bunch of random stuff ... blabla THIS" | grep -oP "(?<=blabla )[^ ]+"

OR

echo "blabla THIS ... a bunch of random stuff ... blabla THIS" | awk '{for(i=1;i<=NF;i++) if ($i=="blabla") print $(i+1)}'

Let's you want to add numbers:

echo "blabla 111 ... a bunch of random stuff ... blabla 222" | awk '{for(i=1;i<=NF;i++) if ($i=="blabla") print $(i+1), total += $(i+1)} END{ print total}'

To copy data from the "machine-where-precious-data-is" to the machine you are working on:

ssh user@machine-where-precious-data-is "tar czpf - /some/important/data" | tar xzpf - -C /new/root/directory

Reversly:

tar cpf - /some/important/data | ssh user@destination-machine "tar xpf - -C /some/directory/"

Kill all remote sessions of a user:

pkill -KILL -u othrif

Remove ^M

dos2unix <filename>

Concatenate PDFs

gs -q -sPAPERSIZE=letter -dNOPAUSE -dBATCH -sDEVICE=pdfwrite -sOutputFile=out.pdf in1.pdf in2.pdf in3.pdf

emacs macro

start def: C-x (, end def: C-x ), execute 37:C-u 37 C-x e, execute infinite: C-u 0 C-x e

afs permissions

afind /afs/cern.ch/user/o/othrif/public -t d -e "fs setacl -dir {} -acl system:anyuser rl"

list folder contents (including hidden)

du -shc -- {.[!.],..?,}*

Remove temporary files

find ~ -type f \( -name -o -name '*~' \) -delete

Replace strings in shell

sed -i -- 's/foo/bar/g' *

Remove empty lines from file:

sed '/^$/d' file.list > tmp

Add size of files referenced by symbolic links:

stat -Lc %s  user.othrif.v09.*/* | awk '{sum+=$1} END {print sum}'

Condor commands:

https://confluence.desy.de/pages/viewpage.action?pageId=67639562

condor_submit myjob.submit
condor_q id

condor_q # Show your active or waiting jobs
condor_q -all # Show all active or waiting jobs
condor_status # Show all active workernodes with slot info

Kerberos:

In a mac, add the following lines in /etc/ssh/ssh_config and ~/.ssh/config

Host *
GSSAPIAuthentication yes
GSSAPIDelegateCredentials yes
copy the edu.mit.Kerberos file to  /Library/Preferences/edu.mit.Kerberos

See Nils Krumnack email from 03/02/2016

useful commands:

kinit othrif@CERN.CH
kinit othrif@DESY.DE
klist -A
# if you have both tickets and you want to switch
kswitch -p othrif@CERN.CH
# OR
kswitch -p othrif@DESY.DE

Git

Git tips

Some basic git commands:

git clone ssh://git@gitlab.cern.ch:7999/othrif/VBFInvAnalysis.git
git status
git add .
git commit -m "change"
git push -u origin master
git pull origin master
git branch -l
git checkout [-b] <branch> # switch or create with -b
git merge <branch>
git ls-files --deleted -z | xargs -0 git rm # in case you removed files
git log # to find all the commits
git checkout a4f5957a0cdcf6d6f3a551f4a0822c0327ce64bc # SHA-1 of the commit
git rev-parse HEAD

If you want to create a new branch and check out an old commit

git checkout -b <new branch> <SHA-1>
git add .
git commit -m "added a new branch from old commit"
git push --set-upstream origin <new branch>

If you do something wrong and want to revert to the previous version:

 git reset --hard <SHA-1>
 git push origin HEAD:master

If you change the repo from different locations, to merge the two:

git remote update
git commit -m "fixing conflict"
git merge origin/master
git push

Create a new branch from existing branch:

git checkout -b new_branch old_branch

Merge your completed work on branch with master:

git checkout master
git pull origin master
git merge test
git push origin master

For gitlab to work with kerboros and mac, you may need:

git config --global http.emptyAuth true

change remote repostiory from http to ssh

git remote set-url origin <ssh address of repo>

check if the change happened with:

git remote -v

Gitlab with Sublime: to push a change do: ctrl+shift+quick add, ctrl+shift+quick commit, ctrl+shift+push, ^ ` to show/hide console

If you have problems with git ssh keys then do:

atlas
lsetup git
ssh-add -t24h

check if it is working

ssh -v git@gitlab.cern.ch -p 7999

Adding submodules:

git submodule add https://:@gitlab.cern.ch:8443/atlas-phys-susy-wg/StopPolarizationReweighting.git
cd StopPolarizationReweighting
git checkout 83fec592
cd ..
git add StopPolarizationReweighting .gitmodules
git commit -am "Adding StopPolarizationReweighting submodule"

To get the latest submodule from master, cd to submodule

git pull origin master

tips from David

Case when you want to undo what you did BEFORE COMMITTING
```
git reset HEAD --
# or
git reset HEAD filename
```
Case when you want to undo what you did AFTER COMMITTING but vefore PUSHING
```
git reset HEAD~
```
to get back to the stage as if you did a clean checkout
```
git reset HEAD~ --hard
```
If you already PUSHED and wanted to undo the last commit but preserve all the history:
```
git revert <HASH>
```

Quick tips:

git ci: for commit
git st: for status
git lg: to get the nice log

how to branch:

git checkout -b mytestbranch
# modify mytestbranch and commit to the branch
git checkout master
# modify master and commit to the branch > now mytestbranch and master diverged
git merge mytestbranch

how to tag:

git tag -a vXX-220518 -m "test tag version"
git push origin vXX-220518

If you tagged a version of the code then needed to change it, after adding committing, do:
```
git tag -f -a <tagname>
git push -f --tags
```
checkout specific tag
```
git tag
git checkout tags/XYZ
```

Delete tags

git push --delete origin TAGNAME
git tag --delete TAGNAME

go to commit you want then tag it:

git checkout <HASH>
git tag <myfirsttag>
git checkout master

if you are in detached state where the HEAD is not pointing to anything, you do:
```
git checkout -b <mynewbranch>
```
if you push and you realize that your local repo is different from the remote where something else has been pushed
```
git fetch
git merge origin/master
#SIMLAR to
git pull
```
if you want to checkout a remote branch to have it locally and its name is oldCodeIn
```
git checkout oldCodeIn
```

Force a branch to a different commit or a branch

git branch -f master origin/oldCodeIn

Branch and Merging

git checkout -b myCode-2018-XX-YY
# Modify
git add .
git commit -m "Branch to merge"
git diff origin/master
git push --set-upstream origin myCode-2018-XX-YY

Then you are given a link to copy and open in git webpage to perform the merging.

Mounting CVMFS on your mac

https://cernvm.cern.ch/portal/filesystem/quickstart https://twiki.cern.ch/twiki/bin/view/AtlasComputing/Cvmfs21 Files available in the main folder: cvmfs_check.sh to be placed in /Users//private cvmfs.filesystems.plist under /Library/LaunchDaemons

Sublime:

Sublime tips:

to add comment: /** <tab>
EasyClangComplete recognizes CMake
to indent: opt+shift+f
Jump to matching bracket: ctrl+m
Sublime text packages are located in /Users/othmanerifki/Library/Application Support/Sublime Text 3/Packages
For sftp, location is Library/Application\ Support/Sublime\ Text\ 3/Packages/User/sftp_servers/

Gitbook:

Run gitbook:

Install GitBook npm install gitbook-cli -g
Fetch GitBook's latest stable version gitbook fetch latest
Preview your project: gitbook serve
Add content
Generate the website: gitbook build (optional)
Push your changes to the master branch: git push

Gitbook tips:

[Link website][ci] then define it at the end [ci]: https://about.gitlab.com/gitlab-ci/
bold with **bold**
```yaml bla bla bla``` for configuration files

DocToc: Website

Install DocToc npm install -g doctoc
In terminal, doctoc file1.md file2.md
You can even add a table of content compatible with other sites

NAF on your MacOS

Do sshfs othrif@naf-atlas11:/nfs/dust/atlas/user/othrif/ /Users/othmanerifki/naf
sshfs -o reconnect,compression=yes,transform_symlinks,ServerAliveInterval=45,ServerAliveCountMax=2,ssh_command='autossh -M 0' othrif@naf-atlas11:/nfs/dust/atlas/user/othrif /tmp/naf

ROOT tips

TTree::Draw

-TTree::Draw overwrites hists by default, even though the symbol is “>>”:

tree->Draw(“variable1 >> hist”)
tree->Draw(“variable2 >> hist”) #this overwrites hist

-TTree::Draw fills 2D histograms backwards:

tree->Draw(“variable1:variable2 >> hist”) #actually, variable1 goes to the y-axis and variable 2 goes to the x-axis!

Apply selection:

tree->Draw(“variableY:variableX>>hist”, “(variableZ>10 && variableZ<100) * eventweight”, “colz”);

Setting aliases: If you call t->SetAlias("MyVar", "SomeComplicatedExpression") you can then use "MyVar" to represent that expression in later TTree::Draw calls.

Virtual environment

crete a virtual environment:
```
mkvirtualenv <virtual name>
```
access an existing environment
```
workon <virtual name>
```
exit the environment
```
deactivate
```

Pyroot

To keep plot after you execute macro:
- raw_input("press enter to continue...") just press a key to exit
- gApplication.Run() but you have to do File>Clone ROOT to exit
Draw histograms from TTree:
- scan all tree satisfying expression condition: tree.Draw("jet_pt >> hist", condition)
- continuously append to a tree tree.Draw("jet_pt >> +hist", condition)

Homebrew

Virtual environment

https://hackercodex.com/guide/python-development-environment-on-mac-osx/ http://docs.python-guide.org/en/latest/dev/virtualenvs/

# Install virtualenvwrapper
brew install python
python -m pip install virtualenvwrapper
source $(brew --prefix)/bin/virtualenvwrapper.sh

# Set up a temporary virtual environment
mktmpenv

# Install the package of interest as well as homebrew-pypi-poet
pip install some_package homebrew-pypi-poet


# Destroy the temporary virtualenv you just created
deactivate

cd my_project_folder
virtualenv my_project
virtualenv -p /usr/bin/python2.7 my_project
source my_project/bin/activate
pip install requests
deactivate
pip freeze > requirements.txt
pip install -r requirements.txt

Installing ROOT in MacOS

Download root and create a build directory as in /Applications/root Modify the following paths to make sure pyroot picks up the correct python libraries

export pexe="/usr/local/bin/python"
export pinc="/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/Headers"
export plib="/usr/local/Cellar/python@2/2.7.14_3/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib"
cmake ../root_src_190418 -DPYTHON_EXECUTABLE=${pexe} -DPYTHON_INCLUDE_DIR=${pinc} -DPYTHON_LIBRARY=${plib} -Droofit=ON -Dtmva=ON
make
source /Applications/root/build/bin/thisroot.sh

Root version in $ROOTSYS/include/RVersion.h Numpy version import numpy numpy.version.version

results matching ""

No results matching ""