EDM4hep I.

EDM4hep II.

#-------------  CalorimeterHit
edm4hep::CalorimeterHit:
  Description: "Calorimeter hit"
  Author : "F.Gaede, DESY"
  Members:
    - uint64_t cellID            //detector specific (geometrical) cell id.
    - float energy               //energy of the hit in [GeV].
    - float energyError          //error of the hit energy in [GeV].
    - float time                 //time of the hit in [ns].
    - edm4hep::Vector3f position //position of the hit in world coordinates in [mm].
    - int32_t type               //type of hit. Mapping of integer types to names via collection parameters "CalorimeterHitTypeNames" and "CalorimeterHitTypeValues".

Datasets

Plethora of processes are pre-generated and available from EOS

• Two main production campaigns in use:
• Spring 2021
• Winter 2023
• Processes are identified by its name, e.g.: p8_ee_WW_ecm240
• The production Database browsable at:
  fcc-physics-events.web.cern.ch
• Example:
  Delphes events, IDEA, FCCee, winter 2023
• EOS directory:
  /eos/experiment/fcc/...
• Generation handled by EventProducer
  • Heads up: Will change soon (Dirac, iLCDirac)

EOS Space

ROOT RDataFrame

• Describes processing of data as actions on table columns
  • Defines of new columns
  • Filter rules
  • Result definitions (histogram, graph)
• The actions are lazily evaluated
• Multi threading is available out of the box
• Optimized for bulk processing
• Allows integration of existing C++ libraries

Functional Approach

• The operations on the dataframe happen with small stateless functions:

• or with structs, which have internal state:

Analysis as a graph

Analysis can be imagined as a graph composed out of building blocks

To generate graph of your analysis:

              
fccanalyses run analysis_script.py --graph
              
            

Analysis as a graph

Analysis as a graph

Analysis Architecture I.

One can write and run the analysis in several ways

• Managed mode: fccanalysis run my_ana.py
  • The RDataFrame frame is managed by the framework
  • User provides Python analysis script with compulsory attributes
  • Libraries are loaded automatically
  • Dataset metadata are loaded from remote location — CVMFS/HTTP server
  • Batch submission on HTCondor
  • Customization: Possible at the level of analyzer functions
  • Intend for: Quick analysis, no advanced analyzer functions

Writing an analyzer function

• Analyzer function is a C++ function or struct
• Typically and analyzer is a struct which operates on an EDM4hep object
• Optional dependencies for analyzers can be: FastJet, DD4hep, ACTS and ONNX
• ROOT RDataFrame needs to be aware of the analyzer function
  • Provided as a string
  • Loaded and JITed by the ROOT.gInterpreter
  • Compiled in the library

Workflow

• The complete analysis in managed mode is divided into three steps (example):
  • analysis_stage1.py, ... — pre-selection stages, analysis dependent, usually runs on batch
  • analysis_final.py — final selection, produces final variables
  • analysis_plots.py — produces plots from histograms/TTrees
• or into two with the help of Histmaker (example):
  • The pre-selection stages and final stage are combined together
  • Plotting step
• Disclaimer: Plotting facilities are rudimentary, improvements are welcome :)

Improvements

Making FCCAnalyses more robust framework

• Global reorganization of the internal structure --- fccanalysis + sub-commands
• Synchronized logging functionality across the whole framework (Python + RDataFrame + ROOT)
• Created man pages (terminal + web)
• General safety and robustness
• Reviving FCCAnalyses package in Key4hep stack
• Testing the whole analysis chain

fccanalysis sub-commands

              
[jsmiesko@death-machine FCCAnalyses (master =)]$ fccanalysis --help

...

sub-commands:
  sub-command           one of the available sub-commands
    init                generate a RDataFrame based FCC analysis
    build               build and install local analysis
    test                test whole or a part of the analysis framework
    pin                 pin fccanalyses to the current version of Key4hep stack
    run                 run a RDataFrame based FCC analysis
    final               run a RDataFrame based FCC analysis final configuration
    plots               run a RDataFrame based FCC analysis plot configuration
              
            

Logging functionality

              
[jsmiesko@death-machine FCCAnalyses (master =)]$ fccanalysis --help
usage: fccanalysis [-h] [-v | -vv | -vvv] sub-command ...

FCCAnalyses v0.8.0

options:
  -h, --help            show this help message and exit
  -v, --verbose         make output verbose
  -vv, --more-verbose   make output more verbose
  -vvv, --most-verbose  make output even more verbose
              
            

              
#include "RLogger.hxx"
R__LOG_INFO(ROOT.Detail.RDF.RDFLogChannel(), "Info message")
              
            

Manual pages

Testing FCCAnalyses package

              
source "${FCCTESTS_STACK}"

RNDMSTR="$(sed 's/[-]//g' < /proc/sys/kernel/random/uuid | head -c 12)"
WORKDIR="${FCCTESTS_TMPDIR}/fccanalyses-stack-full-analysis-${RNDMSTR}"

mkdir -p "${WORKDIR}" || exit 1
cd "${WORKDIR}" || exit 1

fccanalysis run ${FCCANALYSES}/../share/examples/examples/FCCee/higgs/mH-recoil/mumu/analysis_stage1.py || exit 1
fccanalysis run ${FCCANALYSES}/../share/examples/examples/FCCee/higgs/mH-recoil/mumu/analysis_stage2.py || exit 1
fccanalysis final ${FCCANALYSES}/../share/examples/examples/FCCee/higgs/mH-recoil/mumu/analysis_final.py || exit 1
fccanalysis plots ${FCCANALYSES}/../share/examples/examples/FCCee/higgs/mH-recoil/mumu/analysis_plots.py
              
            

• Multiple FCCAnalyses tests running every morning
  • Plan to integrate them into Key4hep validation

Work in Progress

Unlocking full potential of ROOT RDataFrame and EDM4hep inside FCCAnalyses framework

• EDM4hep RDataSource
• Sample metadata (Dirac)
• Import FCCAnalyses / analysis compartmentalization
• Standard library of the analyzers
• Slow or fast decay of the EDM4hep in the analysis
• Test / Validation facilities
• Event visualization

EDM4hep RDataSource

Preserving EDM4hep Associations in RDataFrame

• Podio/EDM4hep has several layers
• Highest layer provides associations
• Having associations greatly improves writing/understanding of the analyzers
• Majority of the analyzers needs adjustment

-----

For more details about EDM4hep, see Thomas' talk

Sample metadata

Sample output path example
/eos/experiment/fcc/prod/fcc/ee/winter2023/91.19gev/Zbb/idea/delphes/00012345/

-----

See also Lorenzo's talk

import FCCAnalyses

              
import FCCAnalyses
import ROOT

ROOT.gROOT.SetBatch(True)

def main():
    '''
    Example analysis entry point
    '''

    sample = FCCAnalyses.Sample('p8_ee_WW_ecm240')

    FCCAnalyses.register_analyzers('examples/FCCee/import/AddAnalyzers.h')

    dframe = fccana.get_dataframe(sample)
    dframe2 = dframe.Define("particles", "gen_particles()")
    dframe3 = dframe2.Define("particles_pt", "MCParticle::get_pt(particles)")
    hist = dframe3.Histo1D("particles_pt")
    hist.Print()

    canvas = ROOT.TCanvas("canvas", "", 450, 450)
    hist.Draw()
    canvas.Print('test.pdf')

if __name__ == '__main__':
    main()
              
            

FCCAnalyses library

• Vertexing
• ACTS vertex finder
• Event variables
• Calorimeter hit/cluster variables
• Reconstructed/MC particle operations
• Flavour tagging
• Jet clustering/constituents

Event visualization

Pythia 8 | ee → ZH @ 240 GeV

To visualize your MC Particle tree, do:

• source /cvmfs/sw.hsf.org/key4hep/setup.sh
• edm4hep2json -l Particle -n 10 /eos/experiment/fcc/ee/generation/DelphesEvents/winter2023/IDEA/p8_ee_WW_ecm240/events_059793334.root -o p8_ee_WW_ecm240.json
• visit: https://key4hep.github.io/dmx/ and upload your .json file

Event visualization

Pythia 8 | ee → ZH @ 240 GeV