Clara framework is multi-dimensional elastic system, and is capable of auto-scaling vertically as well as horizontally. For example Clara will multi-thread services within an application to utilize all available cores of a node (vertical scaling), and the same time will also expand to available nodes in the network (accessible LAN or WAN) to consume cores on network distributed nodes (horizontal scaling).

However, it is understood that this multi-dimensional scaling cannot go linearly forever, due to the data/event provisioning system latency. So, to guarantee linear scalability we must process in parallel data/events from different data sources or data provisioning systems.

Vertical scaling

Vertical scaling is achieved by processing multiple events in parallel running threads (so called multi-threading). The only thing required from the user to vertically scale his/her engine code is to program it thread enabled. All the rest of the details, such as creating, maintaining threads and thread pools, event distribution and synchronization, etc. are hidden from users and is provided by the Clara framework.

To enable the vertical scaling user sets the following parameters, for local running:

clara> set threads 8

and for farm deployment:

clara> set farm.cpu 8

Controlled horizontal scaling

The Clara CLI is designed to auto configure horizontal scaling, ensuring linear scalability of entire data processing.

As usually, files.list contains multiple data files to be processed. If the CLI parameter: farm.scaling is set to 0, then the entire data set will be processed in a single Clara DPE (single node), multi-threaded (vertically scaled) over all available cores of that single node.

However, this processing will not be horizontally scaled, meaning that the data files in the files.list will be processed sequentially. Yet, in case user sets farm.scaling = n, that will tell Clara to process group of n files from the files.list data-set in parallel, on different farm nodes. This will speed up processing of the entire data-set N-data_files/n times.


For example a user prepares a data processing with the following Clara CLI settings

$ $CLARA_HOME/bin/clara-shell

   ██████╗██╗      █████╗ ██████╗  █████╗
  ██╔════╝██║     ██╔══██╗██╔══██╗██╔══██╗ 4.3.0
  ██║     ██║     ███████║██████╔╝███████║
  ██║     ██║     ██╔══██║██╔══██╗██╔══██║
  ╚██████╗███████╗██║  ██║██║  ██║██║  ██║
   ╚═════╝╚══════╝╚═╝  ╚═╝╚═╝  ╚═╝╚═╝  ╚═╝

 Run 'help' to show available commands.

clara> show config
servicesFile:        "/group/da/vhg/testbed/clara/myClara/plugins/clas12/config/services.yaml"
fileList:            "/group/da/vhg/testbed/clara/myClara/plugins/clas12/config/files.list"
inputDir:            "/group/da/vhg/testbed/clara/myClara/data/input"
outputDir:           "/group/da/vhg/testbed/clara/myClara/data/output"
threads:             32
reportEvents:        NO VALUE
skipEvents:          NO VALUE
maxEvents:           NO VALUE
logDir:              "/group/da/vhg/testbed/clara/myClara/log"
feHost:              NO VALUE
fePort:              NO VALUE
session:             "gurjyan"
description:         "clara"
useFE:               true
javaMemory:          NO VALUE
javaOptions:         NO VALUE
farm.cpu:            32
farm.memory:         30
farm.disk:           3
farm.time:           1440
farm.os:             "centos7"
farm.stage:          NO VALUE
farm.track:          "debug"
farm.system:         "jlab"
farm.scaling:        0

The files.list contains 11 files.

clara> show files

Now let us require processing groups of 2 files in parallel in different farm nodes.

clara> set farm.scaling 2

This will tell Clara to split the files.list data-set in six subsets, where the first five will have two files and the last one will have a single file.

The internal metadata description for this sub data-sets will be stored in a temporary directory inside $CLARA_HOME/plugins/clas12/config, under the name .<session> (in the presented example it would be $CLARA_HOME/plugins/clas12/config/.gurjyan).

$ cd $CLARA_HOME/plugins/clas12/config/.gurjyan
$ ls
clara_0  clara_1  clara_2  clara_3  clara_4  clara_5

Now if we start farm processing in the CLI, Clara will submit 5 farm jobs.

clara> run farm
Parsing script ... (it may take while)
Parsing script ... (it may take while)
Parsing script ... (it may take while)
Parsing script ... (it may take while)
Parsing script ... (it may take while)
Parsing script ... (it may take while)

clara> show farmStatus
42398564  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12
42398567  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12
42398570  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12
42398573  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12
42398577  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12
42398579  gurjyan     A    priority    --         ...urjyan-clara   Aug 22 16:21  --         --         clas12