Juraj Smieško (CERN)

FCC Week 2025

Vienna, AT

21 May 2025

Content

• Analysis technologies
  • FCCAnalyses: status and plans
  • Other analysis approaches
• Centralized dataset productions
  • Event Producer
  • Status of DIRAC based productions
• Visualization

Analysis technologies

FCCAnalyses Overview

Analysis framework inside Key4hep ecosystem build
on top of the ROOT RDataFrame

• Provides standard library of functions
  • Many C++ HEP frameworks available
• Automatic retrieval of metadata for the centrally produced datasets
  • Including the location and selection of the input files
• Runs the ROOT RDataFrame
  • Local or remote execution
• Helps with histograms/plots
  • Export of results to other tools
• Registry for the analyses
  • Dedicated place for all case-studies

Versions of FCCAnalyses

Or how to get and run FCCAnalyses

• Latest version of FCCAnalyses can be obtained from:
  • GitHub:
    git clone git@github.com:HEP-FCC/FCCAnalyses.git
  • Key4hep nightlies stack:
    source /cvmfs/sw-nightlies.hsf.org/key4hep/setup.sh
• Latest released version of FCCAnalyses is v0.10.0 and can be obtained from:
  • GitHub:
    git clone --branch v0.10.0 git@github.com:HEP-FCC/FCCAnalyses.git
  • Key4hep release stack:
    source /cvmfs/sw.hsf.org/key4hep/setup.sh
• Specialized version for winter2023 samples can be obtained only from:
  • GitHub:
    git clone --branch pre-edm4hep1 git@github.com:HEP-FCC/FCCAnalyses.git
• Recommendation:
  FCCAnalyses can be run without compiling — fccanalysis command is part of the Key4hep stack

Standard library of analyzers

Set of self-contained functions/functors operating on ROOT dataframe

Towards better defined analysis script

Analysis encapsulated into a class

Analysis CLI arguments

Analysis script can use parameters provided from the command-line.

Submit sub-command

Decoupling submission machinery from analysis execution

FCCAnalyses Plans

• Focus on distributed computing
  • Allow running on GRID, Slurm or other platforms
• Make FCCAnalyses able to run on non LXPlus machines
• Better ML support
  Depending not only on the ROOT capabilities
  • A master student will start investigations soon
• Expand plotting capabilities
  • Many products in the Python HEP space
  • Integrate existing tools and/or make FCCAnalyses more interoperable
• Improve performance of EDM4hep relationship handling
  • podio::DataSource performance affected by podio/EDM4hep data alignment
• Streamline NTuple production mechanism

Other analysis approaches

Python and Julia based analysis approaches

• podio Python bindings
• Coffea — awkward-arrays based columnar analysis framework
  • Schema for the spring2021 campaign added recently
  • Effort continues in coffea-fcc-analyses (P. Yadav)
• Julia: high-level, general-purpose dynamic programming language
  • Reading of the EDM4hep handled by the EDM4hep.jl package
  • How to do a HEP analysis summarized by P. Mato in
    Analysis with the Julia language (8th FCC Physics workshop)
• C++: Raw podio/EDM4hep, RDataFrame with analyzers from FCCAnalyses

Centralized dataset productions

Centrally produced datasets

Bookkeeping of centrally produced datasets

• Datasets include generator level files, parametrized simulation and Fullsim samples for FCC-ee and FCC-hh
• Stored on EOS at:
  /eos/experiment/fcc/<accelerator-type>/generation/
/eos/experiment/fcc/prod/fcc/<accelerator-type>/
  • Ongoing transition from EventProducer to iLCDirac (FCC configurations)
• Detailed information about the samples published on FCC Physics Events website
  • Storage backend now uses the SQL database instead of plain JSON files
  • Make dataset database searchable — 2025 Summer Student Project
• FCCAnalyses framework automatically picks up the sample information from YAML and JSON files
  The interface under overhaul, to allow the information to be consumed also by other analysis solutions
• Old FCC-hh Fullsim and Delphes samples moved to tape
  • Sample metadata kept on the website for historical record

DIRAC / iLCDirac

Interware to exploit distributed heterogeneous resources

• Created by LHCb, these days used by Belle2, ILC, CTAO, CEPC, …
• High level interface to interact with Grids, Clouds, HPCs and Batch systems
• Data / Metadata management
  • File transfers / replication, metadata augmented file catalog
• Web interface to control aspects of the system
  Transformations, jobs, accounting, system administration
• Resources organized into virtual organizations (VOs)
• iLCDirac: set of extensions / applications developed for the linear collider studies

DIRAC / iLCDirac @ FCC

Resources available in the FCC Virtual Organization

• FCC resources administered by FCC VO
  • Could be used by individual users and for centralized dataset preparation
  • Support e-group: fcc-vo-support@cern.ch
• Storage Elements attached to FCC VO:
  • GLASGOW-DISK: Operational
  • CNAF-DISK: Operational
  • BARI-DISK: In progress
• Started to exercise the system to produce a test campaign
  • Full campaign will be launched after EDM4hep 1.0 is finalized
  • Configuration kept in FCCDIRAC repository
• Software needs to be wrapped into a DIRAC application
  • Many available thanks to iLCDirac
  • More added to support users/productions of FCC
    • More MC generators, interfaces for Key4hep based applications
• Will need to come up with the organizational structure for the production of the datasets

How to use iLCDirac summarized by A. Sailer in: Monte Carlo Productions for Full Simulation Studies
(The third ECFA workshop on e+e- Higgs, Electroweak and Top Factories)

Visualization

Visualization Overview

Tools living in or cooperating with the Key4hep ecosystem

• Part of an established framework
  • Geant4 Qt6 visualization
  • ROOT: JSROOT, TEve, TGeo
  • DD4hep: geoDisplay, ddsim
• For global overview
  • CED: OpenGL based, originates in iLCSoft
  • Phoenix: an experiment independent web-based event display
• Specialized use cases
  • calodisplay: purpose build visualization tool
  • eede: EDM4hep event data explorer
  • PandoraMonitoring: ROOT based visualization environment for Pandora
  • Computational graph of the analysis, example:
    • fccanalysis run -r ana_script.py

Visualization status

Need for more specialized features tailored for MC, tracking, calorimetry, …

• Web based geometry visualization
  • Usually web tools offer most hassle free experience
  • Good for the overview, lack of specialization
    • Expansion of current capabilities to cover common specialized tasks: MC, tracking, calorimetry, ...
  • Improve geometry conversion and manipulation
    • Missing user configurability
• Datamodel visualization
  • Expansion of supported collections
  • Embedability can extend number of use cases
• Analysis visualization
  • Only basic tree visualization
  • Missing configurability
  • Lack of interactivity

Engage with / Revive the efforts of HSF Visualization WG

Conclusions

Backup

All the Links

Key4hep

Coherent set of packages, tools, and standards for different collider Concepts

• Common effort from FCC, CLIC/ILC, EIC, CEPC, Muon Collider, …
  • Preserves and adds onto existing functionality from iLCSoft, FCCSW, CEPCSW, …
  • Builds on top of the experience from LHC experiments and results of targeted R&D (AIDA, …)
  • Many institutes involved: CERN, DESY, IHEP, INFN, IJCLab, …
• Each project rebases its stack on top of Key4hep
• Having common building blocks enables synergies across collider communities
• Main ingredients:
  
  • Event data model: EDM4hep, based on PODIO, AIDA project
  • Event processing framework: Gaudi, used in LHCb, ATLAS, …
  • Detector description: DD4hep, AIDA project
  • System to build, test and deploy: Spack, suggested by HSF + CVMFS

EDM4hep

Common language for processing and persistifying data

PODIO Datasource

Preserving EDM4hep relationships in RDataFrame

• Collection objects in PODIO/EDM4hep can be accessed on several layers
• Highest layer provides one-to-many and many-to-many relationships
• Easy access to the related objects greatly improves writing and understanding of the analyzer
• Price for this convenience is performance
  Alternative: NTuple creation in Gaudi algorithm

Enabling PODIO Datasource in the analysis:

• Use analyzers which take as input EDM4hep collections:
  edm4hep::ReconstructedParticleCollection, edm4hep::RecoMCParticleLinkCollection, ...
• Instruct FCCAnalyses to use podio::DataSource:
  self.use_data_source = True or fccanalysis run --use-data-source ana_script.py

EOS Analysis Space

Various intermediate files of common interest can be stored centrally

Analysis registry

Central registry for the FCC case studies

• Two repositories:
  • FCCeePhysicsPerformance lists FCC-ee case studies
  • FCChhPhysicsPerformance contains FCC-hh physics performance documentation
• Experimentally one can create analysis package for analysis specific code
• Rudimentary and in need of overhaul
  • Based on the post FSR needs

Tests and benchmarks

Ensuring correctness and performance

• Tests of the FCCAnalyses framework are done daily
  • Test suite is published in FCCTests
  • Goal is to ensure FCCAnalyses be available in the Key4hep nightlies stack
  • Every test run inside independent subprocesses
  • Tests need access to LXPlus like environment
    • Missing HTCondor testing
• Benchmarks of FCCAnalyses is done after every merge of a PR
  • Benchmarks run on small set of events
  • There is no guaranteed machine to run on
  • Benchmarks of Higgs mass recoil example on LXPlus:
    20k–45k evt/s