How Batch Scheduling and Processing Works

FlexNet Manager Suite 2023 R1 (On-Premises)

Batch scheduling and processing, summarized in the diagram above, runs as follows:

1. The batch scheduler service receives a message requesting a particular task. Messages may come from (among other sources):
   • BatchProcessTask.exe or BatchProcessTaskConsole.exe (including command-line input). In the standard implementation, both these executables operate in their own distinct application domains, allowing for live updates to a high-availability functioning system (for details, see Updating a Running System: Shadow Copy).
   • The web interface for FlexNet Manager Suite (for example, when an operator has uploaded spreadsheet or other data).
   • The ManageSoftRL endpoint (the web service that receives uploads FlexNet inventory from inventory beacons).
   • The inventory-beacons endpoint (used by inventory beacons to collect policy and updated rules, as well as to upload third-party inventory).
   • The Activation Wizard (where you enter details of your license to use FlexNet Manager Suite).
2. The message is validated with the following tests:
   • The message type is one of the supported batch processor tasks. If not, the failure is logged. (All supported tasks are listed in Batch Scheduler Command Line.)
   • The scoping is correctly supplied, as listed in Batch Scheduler Command Line (for example, if a group identifier is required, one is supplied and a tenant ID is not supplied; or for tasks requiring a tenant ID, that the ID is correctly supplied, and no group is present). If the scoping is not correctly specified, the message is failed, and the failure is logged. For a standard (single tenant) on-premises implementation, no tenant ID is required for any task.
   • The message is not a duplicate of one already in the task queue. A message is a duplicate if it has the same task type, group name, tenant UID, and parameter set (both names and values). Duplicates are discarded, and logged with the fact that they are duplicates.
3. Validated messages are saved to the database. This allows for recovery after a disastrous failure, such as the server suddenly going down.
4. Saved, valid messages are added to the in-memory pending messages queue.
5. The scheduler service regularly assesses the set of currently pending messages (from the oldest to the most recent) against any other tasks that are currently executing. The constraints include:
   • Limits where no more than one of a particular task may be running at a time
   • Mutual exclusions, where task B may not start while task A is running.
   For the list of constraints, see Batch Processing Constraints.
6. When the scheduler decides a task is safe to execute, it sends a message to the batch processor.
7. The batch processor collects the task from the head of the queue, and executes it, returning a message to update the task status as required.
   The possible status values that may be updated by either the batch scheduler or the batch processor (and are visible in the log files) are:

   Status	Updated by	Description
   Duplicate	Batch scheduler	New incoming task is a duplicate of an existing task that has a Submitted status. Duplicate entries have the same Task type Group name Tenant UID Parameter set.
   Submitted	Batch scheduler	Task submitted and is waiting (possibly on other tasks) for dispatch to the processor service.
   Queued	Batch scheduler	Task dispatched to the tasks queue by the scheduler service, but not yet picked up by the processor service.
   Processing	Batch processor	Task accepted by the batch processor service, and processing is in progress.
   Success	Batch processor	Processor service reported the successful completion of the task (this may be reporting the return code from a child process).
   Error	Batch processor	Error while processing task.

8. While processing, the batch processor continues to send 'heartbeat' messages to the batch scheduler (the heartbeat continues whether the batch processor is busy or idle).
   Tip: If the heartbeat messages fail for a specified period (default: 5 hours), all tasks recorded as Processing are marked as failed, and given the status Error. This may happen, for example, if the batch processor service was killed and failed to reappear.
9. When the processing of the task completes, the status is updated to either Success or Error. Results are logged in %ProgramData%\Flexera Software\Compliance\Logging\BatchProcessScheduler\BatchProcessScheduler.log on the batch server (or the server hosting this functionality), with the log entry including text like the following:

   received status update for task 'Success' from processor serverName

   The same log message includes a task identifier in the following format:

   Task[159add91-a5e4-4755-9ff4-d514baae2e11]:

   To identify the type of task, you can:

   • Search for the first occurrence of this ID in the batch scheduler log, such as:

     Message DataWarehouseUpdateRights[159add91-a5e4-4755-9ff4-d514baae2e11] 
     sent for processing

   • Search for the first occurrence of this ID in the BatchProcessor.log (in the same folder), which also lists the full command that the batch processor executed:

     DataWarehouseUpdateRights[159add91-a5e4-4755-9ff4-d514baae2e11] 
     Starting process with command line '"C:\Program Files (x86)\Flexera 
     Software\FlexNet Manager Platform\DotNet\bin\ComplianceReader.exe" 
     -us false -importtype Exporters -e BusinessIntelligenceRights'

A by-product of this process is that it is not possible to schedule a particular task at a precise time. Scheduled times really mean "not earlier than" and "as close as possible to", and the actual execution start time depends on there being no constraints from other tasks to delay execution.

Using the batch scheduler to organize tasks ensures that:

• Processes for importing the product libraries (such as the Application Recognition Library), which exert locks on various database tables, do not cause failures in inventory and other kinds of imports which may access the same tables
• Imports automatically avoid each other, instead of failing if they are accidentally run at the same time
• Business and third-party inventory imports uploaded by an inventory beacon are automatically processed as they arrive at the central application server. In multi-tenant environments, the batch scheduler can maximize the parallelism between imports for different tenants.
• Tasks originating from different sources (listed at the start of this topic) can be executed on demand when possible, without causing random failures through accidentally requesting incompatible processes at overlapping times.