There are several important schema changes to improve TDAQ configuration description:
All such changes have been discussed during 2005 TDAQ Workshop in Mainz (click link for more information).
In the past it was necessary to create one configuration database object per one process running online. The total number of such processes including HLT processing tasks may reach in the final system 10,000. However most of such applications have very similar data values and only differ by values of few attributes (usually by name). The primary idea is to describe similar applications by single object and put little knowledge into DAL algorithms to calculate unique attribute values. In this case we will gain storage space and performance (less space, faster read, easier readable and editable).
The proposal document is available from EDMS: https://edms.cern.ch/document/684859. There are several extensions created after document publish date, which are listed below.
The Partition::get_all_applications() algorithm is implemented for C++ and Java DAL and it should be used instead of old Partition::get_applications() one, that knows nothing about template applications.
The C++ algorithm's prototype can be found in generated dal/Partition.h file:
void get_all_applications(std::vector<daq::core::AppConfig>& out, Configuration& db,
std::set<std::string> * app_types = 0,
std::set<const Segment *> * use_segments = 0,
std::set<const Computer *> * use_hosts = 0) constwhere:
- the out parameter is the result of the algorithm's work (see dal/app-config.h file for more information about AppConfig class to get application parameters such as: application id, data collection node id, pointers to application, segment and host configuration database objects);
- the optional app_types parameter defines set of application class names (also takes into account their subclasses), which objects have to taken into account (use all applications, if the parameter is 0);
- the optional use_segments parameter defines set of segments, which applications have to be taken into account (use all segments, if the parameter = 0);
- the optional use_hosts parameter defines set of hosts where the applications have to run on (use all hosts, if the parameter = 0).
Above optional parameters can be combined.
To test get_all_applications() algorithm work there is dedicated binary dal_dump_app_config:
Usage: dal_dump_app_config [-d database-name] -p partition [-t [types ...]] [-c [ids ...]] [-s [ids ...]] Options/Arguments: --data | -d database-name name of the database (ignore TDAQ_DB variable) --partition-name | -p partition name of the partition object --application-types | -t types filter out all applications except given classes (and their subclasses) --hosts | -c ids filter out all applications except those which run on given hosts --segments | -s ids filter out all applications except those which belong to given segments
Example below shows how to get all template and run control applications on host lxplus001 for a test partition:
bash$ dal_dump_app_config -d oksconfig:daq/partitions/lxplus_template_tests.data.xml \ -p lxplus-test -c lxplus001 -t TemplateApplication RunControlApplication =================================================================================================================================== | num | Application Object | Host | Segment | MP:id | Application unique id | =================================================================================================================================== | 1 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32767 | test-L2@L2PUTemplateApp:01-19:lxplus001:0 | | 2 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32766 | test-L2@L2PUTemplateApp:01-19:lxplus001:1 | | 3 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32765 | test-L2@L2PUTemplateApp:01-19:lxplus001:2 | | 4 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32764 | test-L2@L2PUTemplateApp:01-19:lxplus001:3 | | 5 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32763 | test-L2@L2PUTemplateApp:01-19:lxplus001:4 | | 6 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32762 | test-L2@L2PUTemplateApp:01-19:lxplus001:5 | | 7 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32761 | test-L2@L2PUTemplateApp:01-19:lxplus001:6 | | 8 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32760 | test-L2@L2PUTemplateApp:01-19:lxplus001:7 | | 9 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32759 | test-L2@L2PUTemplateApp:01-19:lxplus001:8 | | 10 | test-L2@L2PUTemplateApp | lxplus001@Computer | 01-19@HLTSegment | 32758 | test-L2@L2PUTemplateApp:01-19:lxplus001:9 | | 11 | RC@RunControlApplication | lxplus001@Computer | setup@OnlineSegment | 0 | RC@RunControlApplication | | 12 | 19@RunControlApplication | lxplus001@Computer | 01-19@HLTSegment | 0 | 19@RunControlApplication | | 13 | efd@RunControlApplication | lxplus001@Computer | efd_sf@EF_SubFarm | 0 | efd@RunControlApplication | ===================================================================================================================================
The dal package implements algorithms for the Message Passing Node ID calculations and replaces obsolete Application package. If you are using Application package, do the following:
The DAL's algorithm supports different ranges for node 16-bits IDs depending on application type instead of fixed 1K range supported by obsolete Application package as shown below:
Application Type Number of instances Node ID Ranges PT 32K [32K .. 64K[ L2PU 8K [24K .. 32K[ SFI 4K [20K .. 24K[ ROD 2K [18K .. 20K[ ROB 2K [16K .. 18K[ EFD 1K [15K .. 16K[ SFO 1K [14K .. 15K[ DFM 1K [13K .. 14K[ L2SV 1K [12K .. 13K[ ROS 1K [11K .. 12K[ ROIB 1K [10K .. 11K[ L2RH 1K [09K .. 10K[ unknown / not used 9K [00K .. 09K[
The node IDs of template applications go down from maximum range values; the IDs of the non-template applications of the same type are growing from minimal values to maximum. For correct node message passing node ID calculation a database class is considered to be a template class describing PT, L2PU or SFI, if it's name contains "Template" literal with one of the corresponding literals "PT", "L2PU" or "SFI", e.g. classes with names L2PUTemplateApplication or TemplateL2PUApp are considered as classes describing template L2PU applications. The attribute TemplateApplication::Type described by the proposal document is replaced by above convention.
There are several dependencies of the ATLAS online software from external packages having non-CMT structure. To describe paths to shared libraries and binaries mapped on CMT platforms and platform-dependent environment variables it was proposed (see EDMS document https://edms.cern.ch/document/732735) to introduce several new configuration classes:
The Variable class has been extended. There is new 0..N relationship TagValues pointing to the TagMapping object. In the context of an application the DAL algorithms calculating value of variable checks platform for application and if explicitly defined via TagValues relationship, the algorithm takes variable value from the Value attribute of the TagMapping object.
See also graphical view of above classes in the dal/data/schema/views/SoftwareRepository.view.ps file.
The SW_ExternalPackage objects, related TagMapping and Variable objects are automatically generated by the dal_create_sw_repository utility running in the DAQRelease. For external packages it uses the following data file syntax:
macro sw.external.package.X:installation.path 'inst-path' # X - any external package, inst_path - root directory of it's installation macro sw.external.package.X:cmt.tag cmt-tag # cmt tag for given platform, e.g. i686-slc3-gcc323-dbg macro sw.external.package.X:lib.mapping shared-lib-mapping # path to libraries from package root, e.g. slc3_ia32_gcc323/lib macro sw.external.package.X:bin.mapping binary-mapping # path to binaries from package root, e.g. slc3_ia32_gcc323/bin macro sw.external.package.X:uses Y # Y - sw package, which is used by package X macro sw.external.package.X:uses Z # Z - another sw package, which is used by package X macro sw.external.package.X:needs.environment 'VAR1' # package X needs process environment variable VAR1 macro sw.external.package.X:needs.environment 'VAR2' # package X needs another process environment variable VAR2 macro sw.environment.variable.VAR3:value 'FOO' # value of environment variable VAR3, e.g. '${X}/slc3_ia32_gcc323/root'
Above data file is generated for each cmt platform during DAQRelease build and is passed to the dal_create_sw_repository utility.
Below there are several generated interlinked config objects, that allow to build correct run-time environment for online application using COOL package.
The COOL package description:
Object COOL@SW_ExternalPackage Name CMT external package COOL InstallationPath ${COOL_HOME} Uses XercesC@SW_ExternalPackage
SEAL@SW_ExternalPackage
CORAL@SW_ExternalPackage
POOL@SW_ExternalPackageBinaries i686-slc3-gcc323-opt-for-slc3_ia32_gcc323/bin@TagMapping
i686-slc3-gcc323-dbg-for-slc3_ia32_gcc323/bin@TagMapping
i686-slc3-gcc344-opt-for-slc3_ia32_gcc344/bin@TagMapping
i686-slc3-gcc344-dbg-for-slc3_ia32_gcc344/bin@TagMappingSharedLibraries i686-slc3-gcc323-opt-for-slc3_ia32_gcc323/lib@TagMapping
i686-slc3-gcc323-dbg-for-slc3_ia32_gcc323/lib@TagMapping
i686-slc3-gcc344-opt-for-slc3_ia32_gcc344/lib@TagMapping
i686-slc3-gcc344-dbg-for-slc3_ia32_gcc344/lib@TagMapping
SEAL is one of the packages used by COOL:
Object SEAL@SW_ExternalPackage Name CMT external package SEAL InstallationPath ${SEAL_HOME} Uses XercesC@SW_ExternalPackage ProcessEnvironment SEAL_KEEP_MODULES@Variable
SEAL_PLUGINS@Variable
SEAL_CONFIGURATION_FILE@VariableBinaries ... as in COOL ... SharedLibraries ... as in COOL ...
The SEAL_PLUGINS is one of the variables used by SEAL. It is multi-value environment variable variable, which value depends on platform:
Object "SEAL_PLUGINS@Variable" Name SEAL_PLUGINS Value /afs/cern.ch/sw/lcg/app/releases/COOL/COOL_1_3_1/slc3_ia32_gcc323/lib/modules:
/afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_1_4_2/slc3_ia32_gcc323/lib/modules:
/afs/cern.ch/sw/lcg/app/releases/POOL/POOL_2_4_1/slc3_ia32_gcc323/lib/modules:
/afs/cern.ch/sw/lcg/app/releases/SEAL/SEAL_1_8_1/slc3_ia32_gcc323/lib/modulesTagValues i686-slc3-gcc344-opt-to-SEAL_PLUGINS@TagMapping # points on specific value for gcc344-opt
i686-slc3-gcc344-dbg-to-SEAL_PLUGINS@TagMapping # points on specific value for gcc344-dbg
For a binary, that belongs to the TDAQ sw repository (that in turn uses COOL) the run-time environment will have all required variables and paths set, e.g.:
bash$ dal_dump_apps -d oksconfig:daq/partitions/be_test.data.xml -p be_test -a oks2cool -s Dump 1 non-template applications: ### application 'oks2cool@ResourceApplication' ### - possible program file names: * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/installed/i686-slc3-gcc323-opt/bin/oks2cool - environment variables: * ROOTSYS="/afs/cern.ch/sw/lcg/external/root/5.10.00c/slc3_ia32_gcc323/root" * SEAL_CONFIGURATION_FILE="/afs/cern.ch/sw/lcg/app/releases/COOL/COOL_1_3_1/src/RelationalCool/tests/seal.opts" * SEAL_KEEP_MODULES="true" * SEAL_PLUGINS="/afs/cern.ch/sw/lcg/app/releases/COOL/COOL_1_3_1/slc3_ia32_gcc323/lib/modules: /afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_1_4_2/slc3_ia32_gcc323/lib/modules: /afs/cern.ch/sw/lcg/app/releases/POOL/POOL_2_4_1/slc3_ia32_gcc323/lib/modules: /afs/cern.ch/sw/lcg/app/releases/SEAL/SEAL_1_8_1/slc3_ia32_gcc323/lib/modules" * TDAQ_APPLICATION_OBJECT_ID="oks2cool@ResourceApplication" # ... skip the rest ... - binary search paths: # i.e. what goes into PATH variable * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/installed/share/bin * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/installed/i686-slc3-gcc323-opt/bin * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/external/share/bin * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/external/i686-slc3-gcc323-opt/bin * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/installed/share/bin * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/installed/i686-slc3-gcc323-opt/bin * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/external/share/bin * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/external/i686-slc3-gcc323-opt/bin * /afs/cern.ch/sw/lcg/external/Boost/1.33.1/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/app/releases/COOL/COOL_1_3_1/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/XercesC/2.7.0/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/app/releases/SEAL/SEAL_1_8_1/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_1_4_2/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/app/releases/POOL/POOL_2_4_1/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/root/5.10.00c/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/oracle/10.2.0.1/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/mysql/5.0.18/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/uuid/1.38/slc3_ia32_gcc323/bin/ * /afs/cern.ch/sw/lcg/external/pcre/4.4/slc3_ia32_gcc323/bin/ - paths to shared libraries: # i.e. what goes into LD_LIBRARY_PATH variable * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/installed/i686-slc3-gcc323-opt/lib * /afs/cern.ch/atlas/project/tdaq/cmt/pre-01-06-00/external/i686-slc3-gcc323-opt/lib * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/installed/i686-slc3-gcc323-opt/lib * /afs/cern.ch/atlas/project/tdaq/DataFlow/releases/tdaq-common-01-04-00/external/i686-slc3-gcc323-opt/lib * /afs/cern.ch/sw/lcg/external/Boost/1.33.1/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/app/releases/COOL/COOL_1_3_1/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/XercesC/2.7.0/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/app/releases/SEAL/SEAL_1_8_1/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/app/releases/CORAL/CORAL_1_4_2/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/app/releases/POOL/POOL_2_4_1/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/root/5.10.00c/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/oracle/10.2.0.1/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/mysql/5.0.18/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/uuid/1.38/slc3_ia32_gcc323/lib/ * /afs/cern.ch/sw/lcg/external/pcre/4.4/slc3_ia32_gcc323/lib/
Note, the PATH and LD_LIBRARY_PATH are fully constructed from used sw packages and partition's repository root. They cannot be defined by any other mean (e.g. creating in database explicit LD_LIBRARY_PATH variable).