Smart Upload lets you upload all the data from the source system to Enboarder and let Enboarder decide the action on the subsequent increment load to determine the action, whether it's to launch a workflow, update a workflow or cancel a workflow.
The biggest challenge customers have with CSVs is getting the data out of the source system as it needs customer IT team involvement for getting the CSV feed in the Enboarder format for processing. With Smart Upload, Enboarder allows customers to upload all the employee data as a snapshot upload which only requires minimal IT involvement. They just need to send the incremental feed on a regular interval and Enboarder determines the new workflow to be launched, existing workflows to be modified, or any workflow to cancel using the snapshot that was loaded.
Snapshot Upload Modes
There are three modes that a snapshot file can be loaded in the system:
NEW (Create data registry): A new snapshot file is uploaded. Any existing snapshot data will be purged from the system and a new snapshot will be created. Data will be created in the snapshot table and no workflows will be launched. It will be usually a one-time upload done during the start of the implementation.
UPDATE (Add to data registry): Data in the file will be added to the existing data in the snapshot table. No workflows will be launched in this mode.
INCREMENT (Update data & workflows based on snapshot): Data from this feed will be evaluated against the snapshot.
- Data does not exist in the snapshot > it is treated as a new hire and a new workflow is launched. Data is added to the snapshot after launching the workflow.
- Data exists in the snapshot > it uses action as an update and tries to update the existing workflow. If there is no workflow launched, it will result in an error.
- Termination Date on data > it will stamp the termination date on workflow and workflow will be canceled on the termination date.
NOTE: Snapshot uses an external id field in the data to determine if data exists in the snapshot.
Mapping the snapshot is similar to the existing CSV mapping. The difference between standard mapping and smart mapping is:
- Smart mapping does not have an action to map. The system determines the action itself by looking at the snapshot.
- Smart mapping has an additional column 'Termination Date' available for mapping.
While the snapshot file can have data from a big data set, it is possible to load a small subset of that data as a snapshot. Imagine a company has snapshot data for all departments, however, they want to go live with only one department first. Then it is possible to load the data of only that department as a snapshot.
For doing this, there is a provision to provide filters during the smart upload. There are three options for filters:
- None - no filter will be applied and all data will be loaded as a snapshot.
- Simple filter (select specific rows) - you can define a column and a value. The column is the Header name in the CSV while the value that needs to be included in the snapshot (can be a comma separate list).
- Advance filter - provide filter expression as a string with a complex condition.
The following are the ways to configure the advanced filter:
All column names and values must be enclosed with double quotes i.e. ".
Operator can be =, !=, IN and NOT IN.
Conjunction can be AND or OR.
("Country" = "Australia" AND "Department" != "Sales")
("Country" = "Australia" AND ("Department" != "Sales" OR "Designation" = "VP"))
("Country" = "Australia" AND ("Department" != "Sales" OR "Designation" = "VP")) OR ("Country" = "New Zealand" AND "Division" IN "South","East")
With the latest release, smart upload filters now support the date filters where any date field in CSV can be compared against the current date. For example, below is the filter expression that can be used to load data that has been updated in the last 2 days:
"Last update date" >= "LAST_N_DAYS:2"
The last update date is the CSV field containing the update date on the record and LAST_N_DAYS:2 is the expression used. In this expression 2 can be changed as required.
There is also support added for NEXT_N_DAYS:3 for future computation dates.
Start Date Filter
There is an option to filter records and process only filtered records that have a start date after a pre-configured date.
The use case here is, a customer exports all data from their existing system but they want to launch workflows that were hired after a specified date. So in daily feed, they will have complete export of their data but Enboarder will process only those records that have a hire date after a specific date.
NOTE: The filter will be applied to the CSV column that is mapped to Enboarder's start date field.
Filtered Records in CSV Report (Upload Logs)
A new configuration has been added that will govern whether a filtered out record should be written to a log file or not.
If checked, it will not write the skipped record in the log file. Otherwise, it will be logged as action as skipped.
By default, skipped rows are written into the log file.
Termination Date and Flag
While doing an incremental upload, it is possible to cancel a workflow on a specific date. It will help in the cases where an employee is leaving a company on a specific date and we need to terminate his/her workflow on that specific date.
For this, there is a checkbox 'cancel workflows on termination date', that will determine, along with the termination date to cancel a workflow. If this flag is checked and the termination date is mapped as provided for a workflow, that workflow will be canceled on the termination date by the Enboarder workflow engine.
Snapshot Auto Upload (via SFTP)
All the snapshot activities can be either done manually or automatically via SFTP. It is possible to upload a snapshot file in a new/update or increment mode via SFTP. For SFTP, these modes are determined from the file naming convention.
When defining smart upload options, there is an option to define file name prefix for all these modes. Also, there is a checkbox 'cancel workflows on termination date' to allow cancellation of workflow on termination date.