Scheduling API Jobs in FSM Data Ingestion Utility
After adding at least one organization in the Organizations page as per Managing Organizations in FSM Data Ingestion Utility, go to the API Jobs page (API Jobs > API Jobs) and select the Jobs Scheduled tab.
The Jobs Scheduled tab includes a table that lists the scheduled API jobs to import data from your organization’s multiple SaaS applications to FSM Data Ingestion Utility. Use the Filter pulldown menu to filter scheduled jobs by SaaS application.
This section describes the following FSM Data Ingestion Utility API Jobs menu tasks.
• | Adding a New API Job |
• | Metadata Mapping |
• | Importing an API Job |
• | Exporting an API Job |
• | Viewing API Job Logs |
Complete the following steps to add a new API job on the API Jobs page.
Important:You must add at least one API job to populate the table on the API Jobs page.
To add a new API job:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Jobs Scheduled tab. |
3. | Click the Add New API Job button, which opens the Add API Jobs dialog. The following table describes the Add API Jobs functions. |
Function |
Description |
Refresh |
Refreshes the list of available SaaS applications for the associated Organization ID. |
Pause |
Temporarily stops the execution of the scheduled job. |
Clear |
Clears all the fields. |
Enable Paging |
For details, see Enabling Paging Information. |
Test Connection |
For details, see Testing the Connection. |
Next |
Allows you to go from the Params tab to the Headers tab and then to the Body tab. |
Cancel |
Cancels the API job configuration. |
4. | Complete the Add API Job fields. For details, see the following table. |
Field |
Description |
Job Name |
Enter the appropriate name in the Job Name field. |
Application |
Select the appropriate SaaS application name from the associated Organization ID’s prepopulated dropdown menu. |
Task |
Select the appropriate Flexera SaaS Management integration task to associate with the SaaS application usage data to be transferred to FSM Data Ingestion Utility. For Task descriptions, see the “Tasks to Load Data” section in SaaS Data Payload. |
Schedule |
Select the appropriate frequency (Daily, Weekly, Monthly) and time (hour, minutes, AM/PM) to run the scheduled API job. For Weekly scheduled jobs, select the appropriate day of the week. For Monthly scheduled jobs, select the appropriate day of the month. The time is displayed per your local time. |
Method |
Select either GET or POST. |
API URL |
Enter the appropriate API URL to ingest the SaaS data. |
5. | The Add API Job page has two phases: Request Info and Metadata Mapping. Complete the following Request Info tasks before Metadata Mapping. |
• | Configuring the Params Tab |
• | Configuring the Headers Tab |
• | Configuring the Body Tab |
• | Enabling the Bearer Token |
• | Enabling Paging Information |
• | Testing the Connection |
The parameter configured within the Params tab is passed in the API URL as part of the query string. For details, see the following screenshot.
The following table describes how to configure the fields in the Params tab.
Field |
Description |
System Key |
The fromDate and toDate functions filter the data between the dates. These values cannot be deleted or updated since these keys are system keys. You cannot add other keys, but you can add a Mapped Field. |
Mapped Field |
This key is for response mapping with the fromDate and toDate functions. These values are completely optional. You can also map your own parameters to be passed in the API URL query string or header parameters. |
Value |
The format of the fromDate and toDate key values need to be entered in MM/DD/YYYY format. Other values can be in any other format like text or number as per required to be passed. |
Action |
These key values cannot be deleted because these keys are system keys. |
Headers can be configured to provide authorization, authentication, content type, origin, and more. These parameters are passed as a header in the client request. For details, see the following screenshot.
The following table describes how to configure fields in the Headers tab.
Field |
Description |
Key |
Enter the header parameter in this field. Example: Enter Authorization for authenticating the API. |
Value |
Values in this field can be related to a key. Example: Enter the authorization token for a bearer token or a basic token. |
Action |
Clicking the Delete link in the Action field does not delete the values in the Value field. |
The Body tab is configured for the POST API. In the Body tab, go to the text box and enter the POST API in the JSON format where the property and its value can be assigned. Example: { “createdDate”: “2022-12-01”, “name”:”josef” }
As an option, you can enable a bearer token for API jobs. This feature works by selecting the Default or the Advanced installation option in Installing FSM Data Ingestion Utility. Select Advanced to run FSM Data Ingestion Utility as a Windows Service and to schedule API tasks. The following links detail the process for enabling the bearer token.
• | Enabling Advanced AUTH Information for a New API Job |
• | Enabling Advanced AUTH Information for an Existing API Job |
• | Supported OAuth Methods in FSM Data Ingestion Utility |
• | Common Attributes Information in JSON Templates for All Supported OAuth Methods |
• | Client Secret Based Authentication |
• | Client User Based Authentication |
• | Refresh Token Based Authentication |
Enabling Advanced AUTH Information for a New API Job
The following steps are required to enable advanced AUTH information for a new API job. For details, see the following screenshot.
To enable advanced AUTH information for a new API job:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Request Info phase. |
3. | Go to the Advanced Auth Info tab. |
4. | Select the Advanced Auth checkbox to enable the field for configuring JSON data as per the appropriate OAuth method. |
Enabling Advanced AUTH Information for an Existing API Job
The following steps are required to enable advanced AUTH information for an existing API job. For details, see the following screenshot.
To enable advanced AUTH information for an existing API job:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Request Info phase. |
3. | Go to the Headers tab. |
4. | In the Action column, click Delete to delete the existing Authorization header information. |
5. | Go to the Advanced Auth Info tab and select the Advanced Auth checkbox to enable the field for configuring JSON data as per the appropriate OAuth method. |
Supported OAuth Methods in FSM Data Ingestion Utility
FSM Data Ingestion Utility currently supports the following OAuth methods. Click the appropriate link for more details.
• | Client Secret Based Authentication |
• | Client User Based Authentication |
• | Refresh Token Based Authentication |
Common Attributes Information in JSON Templates for All Supported OAuth Methods
The following common attributes information in JSON templates are applicable for all supported OAuth methods in FSM Data Ingestion Utility.
• | "token_expiry_in_minutes" is an optional field. If a minutes value is not entered, the default time to generate the token will be 5 minutes. This configurable value is stored in the Settings table. If a minutes value is entered, the entered value will take priority over the default value of 5 minutes in the Settings table. |
• | "form-data" displays by default when you do not enter a value for "headers": { "Content-Type": } |
• | "token_url" : "Token URL value", is a required field to generate the access token. The token URL value is specific to the application. |
Client Secret Based Authentication
The following sample template is for the JSON "body" section using client secret based authentication.
Note:The following JSON fields are required for client secret based authentication:
• | "auth_type" |
• | "token_url" |
• | "body" section fields "grant_type", "client_id", and "client_secret" |
{
"auth_type" : "ClientSecretAuthentication",
"token_url" : "Token URL value",
"token_expiry_in_minutes" : "Specify the time in minutes to after which new access token needs to be generated",
"headers": {
"Content-Type": "form-data"
},
"body": {
"scope": "scope value",
"grant_type" : "client_credentials",
"client_id" : "Client ID value",
"client_secret": "Client Secret Value"
}
}
Client Secret Based Authentication Error Message
If any of the mandatory fields are missing in the JSON "body" section, an error message mentioning the missing parameters will display after clicking the Test Connection button in the Advanced Auth Info tab.
Below is a sample error message.
Please provide valid Advanced Auth information.
client_id is required under body
client_secret is required under body
Client User Based Authentication
The following sample template is for the JSON "body" section using client user based authentication.
Note:The following JSON fields are required for client user based authentication:
• | "auth_type" |
• | "token_url" |
• | "body" section fields "grant_type", "client_user", and "client_password" |
{
"auth_type" : "ClientUserAuthentication",
"token_url" : "Token URL value",
"token_expiry_in_minutes": "Specify the time in minutes to after which new access token needs to be generated",
"headers": {
"Content-Type": "form-data"
},
"body": {
"grant_type" : "client_credentials",
"scope": "scope value",
"client_user": "Client User value",
"client_password": "Client Password value"
}
}
Client User Based Authentication Error Message
If any of the mandatory fields are missing in the JSON "body" section, an error message mentioning the missing parameters will display after clicking the Test Connection button in the Advanced Auth Info tab.
Below is a sample error message.
Please provide valid Advanced Auth information.
client_user is required under body
client_password is required under body
grant_type is required under body
Refresh Token Based Authentication
The following sample template is for the JSON "body" section using refresh token based authentication.
Note:The following JSON fields are required for refresh token based authentication:
• | "token_url" |
• | "auth_type" |
• | "body" section fields "grant_type" and "refresh_token" |
{
"token_url": "Token URL value",
"auth_type": "RefreshTokenAuthentication",
"token_expiry_in_minutes": "10",
"headers": {
"Content-Type": "application/x-www-form-urlencoded"
},
"body": {
"grant_type": "refresh_token",
"client_id": "",
"client_secret": "",
"refresh_token": "Refresh Token value"
}
}
Refresh Token Based Authentication Error Message
If any of the mandatory fields are missing in the JSON "body" section, an error message mentioning the missing parameters will display after clicking the Test Connection button in the Advanced Auth Info tab.
Below is a sample error message.
Please provide valid Advanced Auth information.
refresh_token is required under body
Paging information is the separate sections from params to configure the full or delta load of the task selected. It has predefined parameter keys to select and those are mapped to the URL, body, or response keys.
Best Practice:If a single API call response size is more than 10MB, then it should be configured with pagination.
If required, complete the following steps to enable paging information.
To enable paging information:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Request Info phase. |
3. | Go to the Params tab. |
4. | Select the Enable Paging checkbox to enable the Paging Info tab. For details, see the following screenshot. |
5. | Go to the Paging Info tab. |
6. | Refer to the following table to configure the fields in the Paging Info tab. |
Field |
Description |
||||||||||||||||||||||||||||||
Key |
The system key is used to process pagination with different options in API configuration. From the dropdown menu, select the appropriate predefined key:
|
||||||||||||||||||||||||||||||
Mapped Key |
This key is to be mapped as a part of the actual URL query string, request body for the POST API, or the result on executing the API. |
||||||||||||||||||||||||||||||
Value |
This field is optional. If you enter the value, it considers this value prior to the result of the API. |
||||||||||||||||||||||||||||||
Parameter Type |
From the dropdown menu, select the appropriate predefined parameter:
|
||||||||||||||||||||||||||||||
Action |
Clicking the Delete link in the Action field deletes a row in the Paging Info tab. |
After completing all the required fields in the Params, Headers, and Body tabs of the Add API Job page, you can test the API connection. Clicking the Test Connection button executes an API and checks whether or not there is an API response. For details, see the following screenshot.
The following table describes the possible API Test Connection messages.
Test Connection Message |
Description |
200 - OK - Test Connection Succeeded |
Successful Test connection |
401 - Unauthorized - Test Connection failed |
Authorization is incorrect/not provided |
503 - Service Unavailable - Test Connection failed |
The API is not ready to process requests. |
500 - Internal Server Error - Test connection failed |
There is an error in getting a response for the configured API. There is something wrong with the original API to execute. |
1000 - Internal Error - Test connection failed |
Unable to execute API with the provided configuration |
Important:On the Add API Job page, you need to test your API connection in the Request Info phase before completing this section.
The second phase of the Add API Job page is Metadata Mapping. Click the Metadata Mapping icon at the top of the Add API Job page to display the Metadata Mapping table. This table allows you to map fields between the selected task fields in the FSM Data Ingestion Utility and the field headers in the input API. For details, see the following screenshot.
The following table describes the Metadata Mapping fields.
Field |
Description |
||||||
Application |
This is the selected SaaS application name for the associated API job. |
||||||
Mapped Fields |
The input API header fields to be mapped are displayed within this field. |
||||||
Action |
|
Complete the following steps to map fields between the selected task fields in the FSM Data Ingestion Utility and the field headers in the input API.
To map between FSM Data Ingestion Utility and the field headers in the input API:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Metadata Mapping phase to display the Metadata Mapping table. |
3. | In the Actions column, click Map. The Select Mapping Field dialog opens. The field mapping between the FSM Data Ingestion Utility task fields and the input API fields is configured. |
4. | Select the required Test Connection Response for the task field. For details, see the following screenshot. |
5. | Click Save. The API field is mapped and is listed in the Mapped Fields column. |
Note:If you click Save without mapping the mandatory fields, an error message displays with the names of the missing field names. Example: The following fields are mandatory and cannot be empty: ssoAppName, ssoAppDisplayName, appInstanceID.
The following table lists the mandatory fields for the integration tasks in the FSM Data Ingestion Utility.
Task |
Mandatory Field |
Application Roster |
Unique ID |
|
|
First Name |
|
Last Name |
|
Application Usage |
Unique ID |
Occurred At |
|
Application Discovery |
SSO App Name |
SSO App Display Name |
|
App Instance ID |
|
SSO Application Roster |
Unique ID |
|
|
First Name |
|
Last Name |
|
App Instance ID |
|
SSO Application Usage |
Unique ID |
Occurred At |
|
App Instance ID |
|
Product Consumption |
ID |
Occurred At |
|
Usage Group |
After you successfully completed Metadata Mapping, you can now import your organization’s SaaS data using the SaaS vendor’s API. The SaaS data needs to be in a .txt file.
API jobs configured using the Advanced Auth Info tab in Enabling the Bearer Token can be imported for copy auth information for a new API job.
Complete the following steps to import an API job in the FSM Data Ingestion Utility.
To import an API Job:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Jobs Scheduled tab. |
3. | Click the Import button, which opens the Import API Configuration dialog. |
4. | Click Select File Path to open Windows Explorer. |
5. | Select the appropriate .txt file and click Open. |
In the Import API Configuration dialog, the Input File Path field is populated with the file path of the selected .txt file. The API data displays in the text box underneath.
The following import process applies to API jobs configured using the Advanced Auth Info tab in Enabling the Bearer Token. After the API job is imported, it is decrypted and displays the data using the proper JSON format in the Import API Configuration text box. For details, see the following screenshot.
6. | Click Import. The selected .txt file is imported into the FSM Data Ingestion Utility and displays an added row in the Jobs Scheduled tab of the API Jobs page. The Schedule column will not be populated. For details, see the following screenshot. |
7. | For scheduling the newly imported API job, click Edit In the Actions column and go to Adding a New API Job. |
Note: If you try to run the imported API job without a schedule, an error message displays: Please update mandatory fields in the job.
8. | If required, complete the authorization for the newly imported API job. For details, see Configuring the Headers Tab. |
The FSM Data Ingestion Utility provides a convenient workflow for exporting an API job under the following circumstances.
Creating a SaaS Integration for a Customer
If you are a Flexera Partner or a Flexera Services Consultant, you can develop a SaaS integration for a customer within the FSM Data Ingestion Utility. After adding the API job, you can export the .txt file to a customer. Then the customer imports the .txt file as described in Importing an API Job.
Submitting a File for Troubleshooting
If you are having issues configuring an API job within the FSM Data Ingestion Utility, you can export the .txt file and submit it to Flexera Support for further analysis. Once the issue is resolved, you can import the revised .txt file as described in Importing an API Job.
Exporting Copy Auth Information for a New API Job
API jobs configured using the Advanced Auth Info tab in Enabling the Bearer Token can be exported for copy auth information for a new API job.
Complete the following steps to export an API Job from the FSM Data Ingestion Utility. For details, refer to the following screenshots.
To export an API job:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Jobs Scheduled tab. |
3. | Go to the appropriate Export table row. |
4. | Click the checkbox in the Export column. You can export multiple API jobs at one time. |
5. | Click the Export button. The Browse for Folder dialog opens. |
6. | Select the appropriate file path for storing the exported API Jobs. Exported API Jobs are saved as .txt files. Advanced Auth Info in Enabling the Bearer Token are saved as .txt files in encrypted format as shown in the following screenshot outlined in red. |
7. | Click OK. If the API Jobs were successfully exported, the following message appears: Export Successful. If no jobs were selected in the Export column of the API Jobs table, the following error message displays: Please select the jobs that you want to export. |
The FSM Data Ingestion Utility provides API Job Logs of all the executed scheduled API jobs for your organization’s SaaS applications.
The path of the API job logs is C:\ProgramData\FlexeraIngestionUtility\Flexera_Log_Directory
Note:The path of the API job logs is not configurable.
To view the API job logs:
1. | Go to the API Jobs page (API Jobs > API Jobs). |
2. | Select the Jobs Logs tab. |
3. | Use the Filter pulldown menu to filter job logs by SaaS application. |
4. | Go to the appropriate Job ID table row. |
5. | In the View column, click the View link to open the API job log. For details, see the following table and screenshots. |
The following table describes the columns in the Job Logs tab.
Table Column |
Description |
Job ID |
Each API job you create has a unique identifier, which is automatically generated by FSM Data Ingestion Utility. |
Job Name |
This is the Job Name you entered while Adding a New API Job. |
Application |
This is the SaaS Application name that was selected while Adding a New API Job. |
Start Time |
This column displays the date and time when the scheduled API job started. The scheduled job frequency was selected while Adding a New API Job. |
Run Time |
This column displays the duration of the scheduled APIjob run. |
Status |
This column displays the API job status. |
View |
Click the View link to open the Log Viewer, which displays the Job Log information for the scheduled job run. See the following sample screenshot. Note:If you selected the Advanced installation option in Installing FSM Data Ingestion Utility to run FSM Data Ingestion Utility as a Windows Service, the Job Log information displays the XML attribute ExecutedbyService=“Yes” next to the TaskName. For details, see the red box in the following screenshot. |