CLI User Replicator
The User Replicator is intended for the downloading of users, cost centers, and/or user roles from pre-defined supported files and for storing its information in the YSoft SafeQ Management Service internal database. YSoft SafeQ provides a REST web service as a middleware, so authentication is required.
By default, the User Replicator will not edit user entities (user, user role, cost center) created from different source (e.g. LDAP replication). To allow such editing, please see section Allow to combine user entity sources.
At a Glance
See the YSoft shell information page for information on how to run shell.
CLI User Replicator
The command for the User Replicator is "user replicate"
User Replicator do not edit another user entity (user, user role, cost centre) by default.
User Replicator has the following parameters:
- --username Name of user connected to the rest service (administrator username)
- --password User password
- --tenant Tenant domain identification (if not set, the default value is 'tenant_1')
- --host URL of YSoft SafeQ Management Service (If not set, the default value is https://localhost . The port number may be omitted as well, default is 80.)
Example of command line arguments
user replicate --username username --password password --tenant tenant_1 --host https:
//localhost
With the parameters above, the .bat file for a fast run can be created:
@echo off
set IMPORTDIR=%SAFEQ_HOME%\utilities\Import tool\
cd %IMPORTDIR%
call "%IMPORTDIR%import_tool.bat" user replicate --username csv --password csvimport --tenant tenant_1 --host https://localhost
WARNING: Avoid using characters such as %, ^, or & in passwords when invoking import_tool.bat from an external script. These characters can cause command corruption, leading to an Unauthorized error. This is a known limitation - while such passwords may work correctly when entered directly in a shell, they can break when passed through batch scripts. The following special characters are known to work reliably in batch contexts: ! @ # $ * ( ) _ + = - \ { [ ] }
The password in the .bat file cannot be encrypted, improved security can be achieved by using combination of PowerShell and DPAPI to encrypt the password and execute the import:
<#
.DESCRIPTION
Script executes CLI user replicator
The script enhances security by storing password on HDD in encrypted way.
The encryption is tied to Windows user, utilizing Windows Data Protection API - DPAPI.
The account running the script must have WRITE permissions for the script file to be able to encrypt the password.
Script should be executed on regular basis via Task Scheduler
Action: powershell.exe
Arguments: -ExecutionPolicy Bypass -File "<pathtofile>\import.ps1"
Run whether user is logged on or not, with highest privileges if needed.
You may need to unblock ps1 file before executing it:
Run PowerShell as Administrator
Run: Unblock-File -Path <pathtofile>\import.ps1
#>
#-----------------------------------------------------------[Parameters]-----------------------------------------------------------
$logDirectory = "$env:SAFEQ_HOME\..\Customization\import\log"
$importDir = "$env:SAFEQ_HOME\utilities\Import tool"
$username = 'admin'
$tenant = 'tenant_1'
$srvhost = 'https://localhost'
# $sqpassword initially set as plain text, upon first execution it will be auto-replaced with encrypted value
# $sqpassword plain text must be enclosed in single quotations '', not in ""
$password = 'mysecretpassword'
#-----------------------------------------------------------[Execution]------------------------------------------------------------
$today = Get-Date -Format "yyyy-MM-dd"
$logFile = Join-Path $logDirectory "import_$today.log"
# Ensure log folder exists
if (!(Test-Path $logDirectory)) {
New-Item -ItemType Directory -Path $logDirectory | Out-Null
}
# --- Rotate old logs (keep 7 most recent) ---
$allLogs = Get-ChildItem -Path $logDirectory -Filter "import_*.log" | Sort-Object LastWriteTime -Descending
$logsToDelete = $allLogs | Select-Object -Skip 7
foreach ($log in $logsToDelete) {
Remove-Item $log.FullName -Force
}
Add-Content $logFile "`n$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") SCRIPT START $($MyInvocation.MyCommand.Name)"
# VALIDATE PASSWORD CHARACTERS (other characters may not be passed properly via import_tool.bat)
$allowedPattern = '^[a-zA-Z0-9!@#$*()_+\=\-\\{}\[\]]*$'
if ($password -notmatch $allowedPattern) {
# Find all invalid characters
$invalidChars = ($password.ToCharArray() | Where-Object { $_ -notmatch '[a-zA-Z0-9!@#$*()_+\=\-\\{}\[\]]' }) -join ', '
$errmsg = "Password contains invalid character(s): $invalidChars. Allowed: a-z, A-Z, 0-9, and !@#$*()_+=-\{[]}"
Write-Error $errmsg
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") $errmsg"
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") SCRIPT TERMINATED"
exit 1
}
# PASSWORD ENCRYPTION VIA DPAPI
$requiresEncryption = $false
$securePassword = $null
$decryptAttempted = $false
try {
$securePassword = $password | ConvertTo-SecureString -ErrorAction Stop
} catch {
$decryptAttempted = $true
if ($password.Length -gt 50) {
$errmsg = "Password decryption failed, and value appears encrypted (length > 50). It was likely encrypted using a different Windows account or on another machine. Cannot continue."
Write-Error $errmsg
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") $errmsg"
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") SCRIPT TERMINATED"
exit 1
} else {
$requiresEncryption = $true
}
}
if ($requiresEncryption) {
Write-Host "Plaintext password detected. Encrypting..."
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Plaintext password detected. Encrypting..."
# Encrypt the password
$securePassword = ConvertTo-SecureString $password -AsPlainText -Force
$encryptedString = $securePassword | ConvertFrom-SecureString
$replacementLine = '$password = "' + $encryptedString + '"'
# Replace this line in the script itself
$scriptPath = $MyInvocation.MyCommand.Path
try {
(Get-Content $scriptPath) -replace '^\s*\$password\s*=.*', $replacementLine | Set-Content $scriptPath
Write-Host "Password encrypted and script updated."
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Password encrypted and script updated."
} catch {
$errmsg = "Error while replacing password with encrypted value. Check that account running the script has WRITE permissions to the script file. Script will continue but password will remain unencrypted."
Write-Host $errmsg
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") $errmsg"
}
}
# PASSWORD DECRYPTION FOR USE IN MEMORY
try {
$plainPassword = [System.Runtime.InteropServices.Marshal]::PtrToStringAuto(
[System.Runtime.InteropServices.Marshal]::SecureStringToBSTR($securePassword)
)
} catch {
Write-Error "Failed to decrypt the password. Exiting."
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Failed to decrypt the password. Exiting."
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") SCRIPT TERMINATED"
exit 1
}
# RUN THE IMPORT TOOL
$cmdhiddenpass = "$importDir\import_tool.bat user replicate --username $username --password <hidden> --tenant $tenant --host $srvhost"
Write-Host "Executing: $cmdhiddenpass"
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") Executing: $cmdhiddenpass"
$cmd = "`"$importDir\import_tool.bat`" user replicate --username $username --password $plainPassword --tenant $tenant --host $srvhost"
try {
Add-Content "$env:SAFEQ_HOME\utilities\Import tool\ysoft-shell.log" "`n"
Write-Host "For debug info see: $env:SAFEQ_HOME\utilities\Import tool\ysoft-shell.log"
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") For debug info see: $env:SAFEQ_HOME\utilities\Import tool\ysoft-shell.log"
} catch {
$errmsg = "Error while testing WRITE access to $env:SAFEQ_HOME\utilities\Import tool\ysoft-shell.log . Check that account running the script has WRITE permissions to it. Script will continue but debug information will not be available."
Write-Host $errmsg
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") $errmsg"
}
Start-Process "cmd.exe" -ArgumentList "/c $cmd" -WorkingDirectory $importDir -NoNewWindow -Wait
Add-Content $logFile "$(Get-Date -Format "yyyy-MM-dd HH:mm:ss") SCRIPT END"
The following steps are needed to enable CSV file replication:
Configure the replication source directory by editing the userReplicationDir configuration property through System Settings. Usually, it is an absolute path on a local computer.
Example of root folder for replication files
- Create CSV files according to the structure described in the next chapter. Use UTF-8 for CSV file encoding.
- Configure the sending of emails when replication fails—see Replicator e-mail configuration page.
- Start replicator by running command line (for its parameters, see above).
- For regular synchronization with the CSV file, schedule the import via system scheduled tasks.
- Information about replication results will be saved into the log file (ysoft-shell.log) stored next to the import_tool.bat file.
Supported file formats: CSV (separated by ';'), Microsoft Excel (XLS and XLSX).
Allow to combine user entity sources
Set configuration property allowCombiningUserEntitySources to allow to combine user entity (user, user role, cost center) sources during replication. This configuration option applies only to creating and updating of user entities.
File Structure
- Before replication, the respective files must be stored in the directory according to the configuration (see above).
- If you store a subdirectory in the aforementioned directory, the files from all the subdirectories are parsed recursively.
- These files must be stored in the format shown below. The files are processed in alphabetical order based on their names. (For replication, only some of them may be used.)
- Only one type of input information per file is allowed, e.g., you cannot import users and update card numbers in the same file.
- Files must be imported in logical order, e.g., users must be created before a card is assigned to them.
- The User Replicator is not able to create/modify the money accounts in YSoft SafeQ Payment System.
Supported input types
- User information
- Card numbers
- Cost centers
- Roles
- Mapping roles to users
- Update of users
- Deletion of users
- Set user's password
Replication of complete user list
Replicates the complete list of all users created from an external source.
<user ID>;<login,alias,...>;<first name>;<last name>;[email];<centre number>;[home directory]
The replication has the following features:
- If the userID equals zero, then the user is updated based on the login (first record). Otherwise, the user with ID matching the userID is updated.
- If the replicator finds an existing user, the replicator compares existing records. In the case of changes, it updates relevant data (and adds and/or removes data).
- If the replicator does not find a user, a new one is created. The user ID for the new user is added by database sequence, not by the userID specified in the data.
- Records not present in the list (and created by the replicator before) will be marked deleted.
PIN or card number update
Adds additional information to an existing list of users (internal and/or replicated from another source).
PINCARD;<login>;<PIN/card number>
or deprecated variant (where user login cannot contain only numbers):
<login>;<PIN/card number>
- New PINs or card numbers will be added to corresponding records.
- PINs or card numbers previously imported with CLI User Replicator for a user will be replaced.
- Empty fields in place of a PIN/card number imply the removal of all associated records.
- When importing a PIN already encrypted via MD5, add the tag md5@@ before the whole PIN
- e.g., jondoe;md5@@PINe10adc3949ba59abbe56e057f20f883e
- Records (for a given login) not present in the list (and created by a replicator before) will be marked deleted.
Cost center list replication
The list of all cost centers to be replicated into the internal database.
DEPT;<centre number>;<centre name>
- Centers not included in the list (and created by the replicator before) will be removed (marked deleted).
- Other cost centers will be added and/or updated.
Role list replication
The list of all roles to be replicated into the internal database.
ROLE;<role name>;<role description>
- Roles not included in the list (and created by the replicator before) will be removed (marked deleted).
- Other roles will be added and/or updated.
Assignment of user to role
The list of roles the user has been assigned to.
RMAP;<login>;<role name>
- Only roles in the list will be kept for a user (plus the role Everyone); the others (created by replicator before) will be removed.
Set password for user
This replication type can be used to initialize a password for replicated (or any other) users.
A password is set only if the user does not have one. This should prevent the situation when the next replication overwrites the current user's password that the user chose instead of the default one (for example, via the Dashboard widget, if the password change action is enabled via their user's rights).
PASSWORD;<login>;<password>
- Password cannot be empty and must be entered as plain text
Update of users
While the replication of users require a list of all users (not-listed users are deleted), the update of users updates only particular users.
USERUPDATE;[userID];[login,alias,...];[name];[surname];[email];[centre number];[home directory]
- If the userID is not empty, the user is searched by ID. Otherwise, the user is searched by login.
- Parameters can be empty. Only filled-in values are updated for the user.
- Users can be updated only if they were created by the replicator before.
Deletion of users
While the replication of users deletes all not-listed users, deletion of users deletes only particular users.
USERDELETE;[userID];[login,alias,...]
- If the userID is not empty, the user is deleted based on ID. Otherwise, the user is deleted based on login.
- Users can be deleted only if they were created by the replicator before.
There are two types of item in the row format prescription:
- An item in "< >" is required
- An item in "[ ]" are optional
Sample row:
0;johndoe;John;Doe;john.doe@example.com;0;