Read, parse and provide CMS High-Level Trigger and Luminosity information such as trigger menus, trigger paths, filter names, etc.
The tasks defined in this project are based on the luigi analysis workflow (law) package. For more information, see
Available tasks:
GetDatasetLFNsGetLumiDataGetMenusFromDatasetGetMenusInDataGetPathsFromDatasetGetPathsFromMenuGetPathsFromRunsGetFilterNamesFromMenuGetFilterNamesFromRunGatherMCFiltersGatherDataFilters
The setup and code in this project use pip3 and python3.
git clone https://github.com/riga/cms-hlt-parser
cd cms-hlt-parser
# a proxy is required for communicating with DAS and the HLT database
voms-proxy-init -voms cms
# always run the setup (which installs some software _once_)
source setup.shBefore diving into the various tasks listed below, you might want to start with some simple commands to get a feeling of what they do and how the results look like.
# command
> law run hltp.GetLumiData \
--hlt-path 'HLT_IsoMu2?_v*' \
--print-summary
# output
runs : 478
lumi : 59735.969 /pb
paths: 3
- HLT_IsoMu27_v14
- HLT_IsoMu27_v15
- HLT_IsoMu27_v16# command
> law run hltp.GetMenusFromDataset \
--dataset /TTToSemiLeptonic_TuneCP5_13TeV-powheg-pythia8/RunIIAutumn18MiniAOD-102X_upgrade2018_realistic_v15-v1/MINIAODSIM \
--print-summary
# output
menus: 1
- /frozen/2018/2e34/v3.2/HLT/V1# command
> law run hltp.GetPathsFromRuns \
--run-numbers 315640,315641 \
--print-summary
# output
+------------------------------------------+---------------+-----------------------+
| HLT menu | Runs | Matching HLT path(s) |
+==========================================+===============+=======================+
| /cdaq/physics/Run2018/2e34/v1.2.2/HLT/V4 | 315640,315641 | HLT_AK4CaloJet100_v10 |
| | | HLT_AK4CaloJet120_v9 |
| | | HLT_AK4CaloJet30_v11 |
| | | HLT_AK4CaloJet40_v10 |
| | | HLT_AK4CaloJet50_v10 |
| | | HLT_AK4CaloJet80_v10 |
| | | HLT_AK4PFJet100_v17 |
| | | HLT_AK4PFJet120_v16 |
# ... and many more
# consider adding "--hlt-paths <csv_separated_patterns>" to filter themTo execute a task, run law run hltp.<task_name> [parameters].
Please find a list of all available tasks, parameters and examples below.
Also, some task parameter have defaults that lookup values in the law.cfg config file.
The output file(s) of a task can be obtained by adding --print-status 0 or --print-output 0 to the respective law run command.
They are mostly JSON files containing the described output information.
Also, almost all tasks print a summary after they successfully run.
In case a task is already completed (i.e., its output exists), it is not run again and no summary is printed.
To print only the summary, add --print-summary to the law run command.
> law run hltp.GetDatasetLFNs [parameters]
Queries the DAS service to get the logical file names (LFNs) for a dataset (e.g. /TTToSemiLeptonic_TuneCP5_13TeV-powheg-pythia8/.../MINIAODSIM).
The output is the list of LFNs.
Parameters:
--dataset: A dataset as shown in DAS. Patterns not supported.
Dependencies:
none
> law run hltp.GetLumiData [parameters]
Uses brilcalc lumi to get luminosity data for a trigger path (name or pattern), given luminosity and normtag files.
The output is a dictionary run -> {lumi, path}.
Parameters:
--hlt-path: A trigger path. Patterns allowed.--lumi-file: A luminosity file. Defaults to thehltp_config.lumi_fileconfig.--normtag-file: A normtag file. Defaults to thehltp_config.normtag_fileconfig.
Dependencies:
none
> law run hltp.GetMenusFromDataset [parameters]
Returns the menus found in the n-th file of a dataset using the hltInfo command.
The output is a list of trigger menus.
Parameters:
--dataset: A dataset as shown in DAS. Patterns not supported.--file-index: The number of the file to query. Defaults to 0.
Dependencies:
flowchart LR
GetMenusFromDataset --> GetDatasetLFNs
> law run hltp.GetMenusInData [parameters]
Uses brilcalc trg to obtain all trigger menus used for data-taking and maps them to the runs the menus they were used in.
The output is a dictionary menu_name -> {menu_id, runs}.
Parameters:
--show-runs: When used, all run numbers are printed in the summary table instead of only their count. Defaults to false.
Dependencies:
none
> law run hltp.GetPathsFromDataset [parameters]
Returns the triggers paths found in the n-th file of a dataset.
Internally, this is done by opening the file with fwlite and reading the trigger names using the trigger bits.
The output is a list of trigger paths.
Parameters:
--dataset: A dataset as shown in DAS. Patterns not supported.--file-index: The number of the file to query. Defaults to 0.
Dependencies:
flowchart LR
GetPathsFromDataset --> GetDatasetLFNs
> law run hltp.GetPathsFromMenu [parameters]
Returns the triggers paths for a trigger menu hltConfFromDB.
The output is a list of trigger paths.
Parameters:
--hlt-menu: The trigger menu to query. Patterns not supported.
Dependencies:
none
> law run hltp.GetPathsFromRuns [parameters]
Returns the trigger paths for trigger menus which were active in certain runs, optionally filtered by a list of trigger path patterns.
The output is a dictionary menu -> {runs, paths}. The columns of the summary table are "HLT menu", "Runs" and "Matching HLT path(s)".
Parameters:
--run-numbers: A list of run numbers to query. Can be inclusive ranges in the format"start-end". When empty, the run numbers from the luminosity file defined by--lumi-fileare used.--lumi-file: A luminosity file. Defaults to thehltp_config.lumi_fileconfig.--hlt-paths: Optional trigger paths to filter the paths obtained by querying a trigger menu (associated to a run). Patterns allowed.--table-format: The tabulate table format. Defaults to"grid".
Example output:
Dependencies:
flowchart LR
GetPathsFromRuns --> GetMenusInData
GetPathsFromRuns -- multiple --> GetPathsFromMenu
> law run hltp.GetFilterNamesFromMenu [parameters]
Gets the names of all EDFilter modules of a trigger path as defined in a trigger menu.
Internally, hltConfigFromDB is used to obtain the configuration file of the path, the configuration is loaded, and the filter modules are identified programmatically.
The output is a list of dictionaries {name, parameters}.
Parameters:
--hlt-path: The trigger path to query. Patterns not supported.--hlt-menu: The trigger menu to query. Patterns not supported.
Dependencies:
none
> law run hltp.GetFilterNamesFromRun [parameters]
Gets the names of all EDFilter modules of a trigger path used in a specific run.
Internally, GetMenusInData is used to make the connection between trigger menu and run number, and GetFilterNamesFromMenu is invoked internally as a dynamic dependency.
In fact, the output is identical to GetFilterNamesFromMenu, i.e., it is a list of dictionaries {name, parameters}.
Parameters:
--hlt-path: The trigger path to query. Patterns not supported.--run-number: The run number to query.
Dependencies:
flowchart LR
GetFilterNamesFromRun --> GetMenusInData
GetFilterNamesFromRun --> GetFilterNamesFromMenu
> law run hltp.GatherMCFilters [parameters]
Gathers information about the EDFilter names of several trigger paths used for MC production.
The menus are obtained by querying the respective first files of some datasets.
The output is a list of dictionaries {menu, datasets, paths}, where paths itself is a list of dictionaries {name, filters}.
The summary consists of two tables.
The first table shows which "HLT path(s)" belong to which "HLT menu" as found in which "Dataset(s)".
The second table contains the actual filter names.
The columns are "HLT path(s)", "HLT menu(s)" and "Filter names".
Parameters:
--datasets: The MC datasets to query. Patterns not supported. Defaults to the entries in thehltp_mc_datasetsconfig.--hlt-paths: The trigger paths to query. Patterns allowed. Defaults to the entries in thehltp_pathsconfig.--verbose-datasets: When used, the full dataset names are printed in the first summary table. Defaults to false.--table-format: The tabulate table format. Defaults to"grid".
Example output:
Dependencies:
flowchart LR
GatherMCFilters -- multiple --> GetMenusFromDataset
GetMenusFromDataset --> GetDatasetLFNs
GatherMCFilters -- multiple --> GetPathsFromMenu
GatherMCFilters -- multiple --> GetFilterNamesFromMenu
> law run hltp.GatherDataFilters [parameters]
Gathers information about the EDFilter names of several trigger paths in several trigger menus employed for data-taking.
Valid run numbers are taken from a luminosity file.
The output is a list of dictionaries {menu, runs, paths}, where paths itself is a list of dictionaries {name, filters}.
The summary table has the columns "HLT path(s)", "Runs", and "Filter names", and optionally "HLT menu(s)".
Parameters:
--hlt-menus: The trigger menus to query. Patterns allowed. Defaults to the entries in thehltp_data_menusconfig.--hlt-paths: The trigger paths to query. Patterns allowed. Defaults to the entries in thehltp_pathsconfig.--lumi-file: A luminosity file. Defaults to thehltp_config.lumi_fileconfig.--show-menus: When used, the optional"HLT menu(s)"column is shown in the second summary table. Defaults to false.--verbose-runs: When used, the run numbers in the second summary table are not contracted to ranges. Defaults to false.--table-format: The tabulate table format. Defaults to"grid".
Example output:
Dependencies:
flowchart LR
GatherDataFilters --> GetMenusInData
GatherDataFilters -- multiple --> GetPathsFromMenu
GatherDataFilters -- multiple --> GetFilterNamesFromMenu