# RealMigrator Documentation
# Introduction
With RealMigrator it is possible to migrate anything and everything to Microsoft 365. RealMigrator helps to solve any roadblocks when migrating to Windows 10, by offering detailed asset inventory, local as well as home share data migration to OneDrive including files and folders, and in addition local PST detection and migration to Office 365 Mailboxes.
RealMigrator is a single file application running in user mode. Based on a flexible set of configurations, RealMigrator will start to analyze the local machine to prepare an ongoing and controlled upload. When finished, the client can easily get a fresh setup of Windows 10 without losing user data.
It is possible to deploy RealMigrator with preferred deployment strategy, whether that be Login Script, GPO, SCCM, or even by mail, and get instant information about all assets and data to migrate. An individual configuration for groups or locations starts afterwards to synchronize all user documents and settings, IE favorites, PST-files and anything else to its future destination, like OneDrive, Exchange Online and Azure.
During Migration, there is no user disturbance. An optional tray icon signals the work in progress and informs about the state of a device on request, but everything happens fully automated in the background with taking care of bandwidth-throttling and metered connection. The sync watches for document changes and works continuously.
The following documentation gives a detailed overview about RealMigrator.
# Beginning
At the beginning, a user will start with the **Server Configuration**. This menu is the first menu which users will see when they start RealMigrator. This Server Configuration is necessary for general RealMigrator configurations and settings, like user roles and administrator roles, bandwidth settings or UI settings.
The Chapter **Sidebar** is an overview of the different menus which available in RealMigrator and a short explanation of their functions and tasks.
Last, **Binaries** is a storage and list of all scripts and binaries which are in use in RealMigrator. Additionally, this chapter will show how scripts can import to RealMigrator.
# Server Configuration
Server Configuration is the start screen of RealMigrator:
Server Configuration gives an overview about **Client Version**, **Super Administrators**, **Administrators**, **View-Only Administrators** and **Users**.
By default all data will be migrated to the RealMigrator cloud storage farm. But when a company decides to migrate to its own server or storage the input field **External Storage** is visible (below input field View-Only Users).
**Global Client Configuration** contains default settings for migration. Via own scripts it is possible to individualize the migrations.
## Roles in RealMigrator
The following table gives an overview about the roles and their permissions.
| Role | Explanation |
| ----------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Super Administrator | All RealMigrator rights and permissions. |
| Administrator |
Limited permissions. Only permissions to create and edit Groups and users and clients. No permissions to edit Modules and Configurations
|
| View-Only Administrator | Only permission to browse through RealMigrator. |
| User | A user has only the permission to list and edit client objects which have the same UPN. Use `*` to match all users. |
## Global Client Configuration
Global Client Configuration is necessary for general migration configurations and general migration settings. A super administrator (an administrator as well) has the possibility to configure a **Log Level**, a **Module loop delay** and a **Reset code**.
| Setting | Explanation |
| ----------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Log Level | Three different log levels are available: **Error**, **Info** and **Debug** |
| Module loop delay | Sleeptime after looping through modules. The default value are 1 800 000 ms (30 minutes). |
| Reset code | A client is always checking the own configurations after a defined period of time. This means that the client compares the selected client settings with the selected server settings. In addition the client connect to the server based on Module loop delay. By this time value the client is starting a Module loop and the client is checking the Reset code. If there is a change the client will takes over the new configuration. If there is no change, the client will still use the existing configuration. |
### Bandwidth Settings
RealMigrator offers seven settings for bandwidth. Each of them have defaults. Of course a super administrator (an administrator as well) has the possibility to set own settings.
| Settings | Explanation |
| ------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ |
| Maximum bandwidth | Default bandwidth are 2000 kBit/s |
| Slow network barrier |
Will be detected as 'slow' network if network speed is below this value. Default are 800 kBit/s. 0 means disabled
|
| Unusable network barrier |
Will be detected as 'unusable' network if network speed is below this value. Default are 200 kBit/s. 0 means disabled
|
| Minimum sleep time | Time to sleep per request. Default are 1000 ms. **0** means disabled |
| Slow network usage | Maximum percent of usage on a slow network. Default are 25 % (0.25) |
| Unusable wait time | Time to sleep in an unusable network. Default are 900 seconds (15 minutes) |
| Use metered connection | If to use network connections which are marked as metered |
### UI Settings
During a migration, RealMigrator shows clients different screens and instructions. With **UI Settings** a super administrator can configure the instructions, explanations texts and dialog screens. Rudimentary interface settings are also manageable.
Beside the defaults (German and English) it is possible to add further languages. Available are Arabic, Chinese, French, Hindi, Japanese, Portuguese and Spanish.
**NOTE**: No translations or phrases are available in these seven languages. English texts are given and an own translation must be implemented
### Script
RealMigrator offers a default script for **Global Client Configuration**. But it is possible to edit the existing script or to writing a new script.
These following functions have to be part of the script:
| Script | Functions |
| ----------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `getClientId()` | Returns a string that uniquely identifies the client |
| `getGroupId()` |
Returns the id string of a group where the client belongs to. This might be some kind of location information or any other grouping criteria.
|
| `getDisplayName()` | Returns a display name that is used by the admin web |
| `getMainConfig(config)` |
Returns a dictionary with further configuration parameters. - Input parameter: server configuration
|
## Further Settings
There are even more settings at the top of the page.
### Import Scripts, Client and Save
| Setting | Explanation |
| -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Import Scripts |
Import a new configuration. Select the carbon.zip folder to update your configurations. For further details about script import see \[Binaries]\({{< ref "beginning/binaries/\_index.md" >}})
|
| Client |
Import Client: Import a client from a PC or machine to RealMigrator Delete Client: Delete an existing client in RealMigrator Download Client: Download a client from RealMigrator
|
| Save | With a click on **Save** all changes in **Server Configuration** will be saved |
### Copy Storage Token and Admin Consent
| Setting | Explanation |
| ------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Copy Storage Token | This token contains the whole migration configuration and the migration data. With a storage token you have access to your storage (via SAS-key) |
| Admin Consent | Sign-in to your tenant (with a global administrator). Via **Admin Consent** you configure the RealMigrator permissions for the migration. Furthermore Azure Active Directory checks if RealMigrator gets granted access to data. |
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values you have to click **Save** at the top of the page.
# Sidebar
RealMigrator offers a number of options to configure a migration in different ways. These options give abilities to configure and adjust **Binaries**, **Server Configuration**, **Modules**, **Groups**, **Clients**, **Reporting** and **Projects**.
The following table gives a short overview about these options and their functions:
| **Option** | **Explanation** |
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Binaries | List of binaries which are part of RealMigrator |
| Server Configuration (start screen) |
Information about client version and an overview about current administrators and users. Also bandwidth settings and UI settings (UI during a migration)
|
| Modules |
Five different modules: Inventory: Collects machine data through WMI, registry and other means File: Searches for files and copies them to a defined destination PST: Searches for Outlook PST files and copies them to defined storage Custom: Possibility to create own scripts Server: Information about PST import server and Teams notifications.
|
| Groups | Creation of new groups and details about existing groups, like migration progress |
| Clients | A list of assigned clients for a migration |
| Reporting | Reports about (upcoming) processes and system errors |
| Projects | Overview about current migration projects |
| Help | Links to the RealMigrator documentation and the API documentation |
# Binaries
This menu contains a list of binaries which are part of **RealMigrator**, like .json files or .chai scripts.
## Import Binaries
Beside the default existing binaries, it is possible to import own binaries and scripts. There are two different options to import binaries into **RealMigrator**:
### Option 1
1. Click the **Drag & drop field**
2. Choose a file
3. Finally, click **Open**
### Option 2
1. Search a script or binary on the own computer
2. Drag and drop this script or binary into the **Drag & Drop field**
## Filter by Path
**Filter by Path** is a filter function based on the file name. To search for a file, enter the name of the file. The filter shows the results automatically in the list.
## Binary Details
To see further details or to delete a binary, click on a name of a binary.
This example above shows a script for the binary **backup/module.chai**.
A detail overview contains information about last modification, content-type, content-length, an ETag and a MD5.\
Additionally, the script is shown as well. It is possible to edit this script manually with a click into the script field. A click on the **Save** button will save the edits.
With a click on the **Open** button, the script will open in a new tab. It is not possible to edit a script in this tab.
### Delete a script
If a script is no longer necessary or incorrect, it is possible to delete it. Click **Delete** in the upper-right corner to delete a script.
# Client Executables
## Clients
In general, clients are services which send requests to programs, computers or servers. Such requests set up connections between a request sender and a request recipient. Once started, such a connection will run itself. RealMigrator offers three different clients. The following chapter will present them.
Click the link to download RealMigrator client.exe files:
The download will start automatically. A Zip folder (**carbon.zip**) appears:
Extract all files:
# client.exe
* client.exe installs itself automatically
* Auto update function
* No configuration necessary
When a client.exe file will download from RealMigrator the name of the file will contain additionally the name of a project, like: **client-\[projectname].exe**
# chaiCmd.exe
* Command .exe file
* Suitable for testing configurations and settings
* Suitable for reporting
### A exemplary chaiCmd.exe usage
Create reporting with **server\_query.chai** (**carbon** => subfolder **testscripts**)
**chaiCmd.exe ..\testscripts\server\_query.chai**
Usage: server\_query.chai \[PROJECT] \[TOKENFILE] \[COMMAND] \[ARGS...]
Command can be one of the following:
| Command | Result |
| ---------- | ------------------------------------------------------------------- |
| groups | list groups |
| software | list installed software (Additional args: \[GROUPID-optional]) |
| progress | list client progress (Additional args: \[GROUPID-optional]) |
| errors | list client errors (Additional args: \[GROUPID-optional]) |
| files | get files report (Additional args: \[CLIENTID]) |
| clientjson | clientjson download client json data (Additional args: \[CLIENTID]) |
| log | download log file (Additional args: \[CLIENTID]) |
| module | download module data (Additional args: \[CLIENTID] \[MODULE]) |
### Report generation
> **chaiCmd.exe ..\testscripts\server\_query.chai \[projectname] token.txt>test.csv**
It creates a progress reporting. Optional use a GroupID to specify a report.
> **chaiCmd.exe ..\gkScriptModule\testscripts\server\_query.chai \[projectname] token.txt groups**
It creates a GroupID - name list
# rmService.exe
* It is a background service
* No automatic installation
### Service Installation
To install **rmService.exe** store it on the computer. Run the .exe file with a command-line interpreter.\
Enter the following command:
```
install []
```
A **UPN** (User Principal Name) is optional, but a **project name** is essential for the installation command.
### Upload Agent
To install the service as an upload agent, you need to provide a **project name** and a **token**. You can generate a token on the server configuration page. Both parameters are required.
```
install-uploadagent
```
# Modules
Configure and edit RealMigrator modules before starting a migration. Modules have different settings and different tasks.
With **Inventory,** it is possible to configure general settings which data RealMigrator will collect from a computer or a machine.
**File Module** and **PST Module** are about migration settings of files and PST files. Different settings are available to configure these modules and a migration in different ways.
Use **Custom Module** to customize a migration. This module can have several functions or roles. For example, a backup function.
With **Server Module,** it is possible to configure a PST Import.
All these modules also have a script function to customize a module in an own way.
**Folder list** is an additional chapter to File Module and PST Module.
**Own Blob Storage** contains an example of how to create an own Blob Storage for File module, PST module and/or Custom module.
# Module Configuration
With Module Configuration, a Super Administrator has the possibility to determine which module will be **On** and which one will be **Off**. As default, **On** is preselected for all **Modules**.
To open Module Configuration, click **Modules**
The following overview will appear:
## Custom Module
This module can have different features or configurations, depending on the following module settings:
| Setting | Explanation |
| ---------------- | ------------------------------------------------------------------------------------------------------------------------------- |
| Off | Custom Module is deactivated |
| Custom | Custom Module is working with chaiScripts. A customer can use chaiScripts to customizes this module in a own way. |
| File | Custom Module has the role of a further File Module |
| PST | Custom Module has the role of a further PST Module |
| Server Migration | It collects data from a file share server and migrates them to a defined storage (like OneDrive, SharePoint or a Blob Storage). |
# Inventory
With Inventory, RealMigrator collects machine data through **Windows Management Instrumentation** (WMI). Depending on settings and configurations, RealMigrator collects data about disk drives, processors, drivers, monitors, printers, software settings and more. This data will migrate to Windows 10 to configure a computer with the previous settings from Windows 7.
## Inventory Settings
| Setting | Explanation |
| ----------------------- | -------------------------------------------------------------------------------------------------------- |
| Run WMI Queries | RealMigrator will run the selected WMI Queries |
| Read Installer registry | RealMigrator will read the installer registry on a client computer |
| Enum drives | RealMigrator collects data about existing drives |
| Enum mail accounts | RealMigrator collects data about existing mail accounts |
| Check network bandwidth | RealMigrator will check and record the network bandwidth of a client computer or a whole company network |
| Execute script breakout | |
| Log level | Available log levels are **Error**, **Info** and **Debug**. |
| Power mode | If this mode is activated, power down is prevented during migration. |
## WMI Queries
Some **WMI Queries** are preset as standard. As it was mentioned before, RealMigrator has the ability to read system and computer settings.
| Setting | Explanation |
| ------- | ---------------------------------------- |
| Key | Name of the query |
| Path | Query will running from the entered path |
| Query | Tasks of the query |
### Query Order
Use the **up and down buttons** to change the order of the queries
## Create and Delete queries
At the bottom of the page there are buttons to create new queries or delete existing ones
* To add further queries, click the button **Query**
* To delete the last created query, click the button **Last Query**
* To delete all of them, click the button **All**
## Script
RealMigrator offers a default script for Inventory. But it is possible to edit the existing script or to write a new script.
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values, you have to click **Save** at the top of the page.
# File
The File Module contains options to configure the searches of files which have to migrate. RealMigrator searches these files and copies them to a defined destination.
## Upload Store
Upload stores always support basic file operations (upload file, create directory). They can support delete operations on files and directories to support proper synchronization.\
In addition, they can support blockmode uploads, which enable differential uploads based on hash maps.
| Setting | Explanation |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Type |
File: Writes to a file back-end, either local or on a file server OneDrive: Stores to the user's OneDrive via a RealMigrator server Graph API Blob Storage: Stores a blob storage via RealMigrator server Blob API
|
| Root path | Default root path |
| Diff file size | Default size are 4 MB. When a file exceeding this size it will be uploaded in blocks. It is possible to define a own size |
| Diff block size | When a file has to uploaded in blocks the default size is 1 MB |
| Path blacklist | Paths which are listed in a blacklist are excluded from all catalogs and will not be part of migrations (see \[Well-known folder] ({{< ref "modules/wellknownfolder/\_index.md" >}}) for folder (paths) list) |
## Catalog
A catalog defines which files RealMigrator has to migrate and where to find them.
| Setting | Explanation |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name | Name of the catalog |
| Path | Concrete path and place where RealMigrator will find the files (see \[Well-known folder] ({{< ref "modules/wellknownfolder/\_index.md" >}}) for folder (paths) list) |
| Remote path | Defined destination for **Path** |
| Filter | RealMigrator migrates the listed file formats |
| Blacklist | When activated RealMigrator will migrate all file formats except the formats from **Filter** |
| Synchronization Type |
One-Move Sync: Copies remote storage and removes them if they are deleted in the source path Copy: Copies changes but does not remove deleted files Move: Removes local file after they were copied to the remote storage
|
### Catalog Order
Use the **up and down buttons** to change the order of catalogs
### Create and Delete catalogs
At the bottom of the page, there are buttons to create new catalogs or delete existing ones
* To add further catalogs click the button **Catalog**
* To delete the last created catalog click the button **Last Catalog**
* To delete all of them, click the button **All**
## Log level and Reset code
| Setting | Explanation |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Log level | Available log levels are **Error**, **Info** and **Debug**. |
| Reset Code | A client is always checking the own configurations after a defined period of time. This means that the client compares the selected client settings with the selected server settings. In addition the client connect to the server based on \[Module loop delay]\({{< ref "beginning/serverconfig/\_index.md#global-client-configuration" >}}), a defined period of time. By this time value the client is starting a Module loop and the client is checking the Reset code. If there is a Reset code change the client will takes over the new configuration. If there is no change, the client will still use the existing configuration. |
| Power mode | If this mode is activated, power down is prevented during migration. |
## Script
RealMigrator offers a default script for File. But it is possible to edit the existing script or to write a new script.
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values, you have to click **Save** at the top of the page.
# PST
The PST Module contains options to configure the searches of PST files which have to migrate. RealMigrator looks for these PST files and copies them to a defined destination.
## Upload Store
Upload Stores always support basic PST file operations (upload file, create directory). They can support delete operations on PST files and directories to support proper synchronization.\
In addition, Upload Stores can support blockmode uploads, which enable differential uploads based on hash maps.
| Setting | Explanation |
| --------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Type |
File: Writes to a file back-end, either local or on a file server OneDrive: Stores to the user's OneDrive via a RealMigrator server Graph API Blob Storage: Stores a blob storage via RealMigrator server Blob API
Default size are 4 MB. When a file exceeding this size it will be uploaded in blocks. (It is possible to define a own size)
|
| Diff block size | When a file has to uploaded in blocks the default size is 1 MB |
| Path blacklist | Paths which are listed in a blacklist are excluded from all catalogs and will not be part of migrations (see \[Well-known folder] ({{< ref "modules/wellknownfolder/\_index.md" >}}) for folder (paths) list) |
## Catalog
A catalog defines which files RealMigrator has to migrate and where to find them
| Setting | Explanation |
| ----------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Name | Name of the catalog |
| Path | Concrete path and place where RealMigrator will find the files (see \[Well-known folder] ({{< ref "modules/wellknownfolder/\_index.md" >}}) for folder (paths) list) |
| Remote path | Defined destination for **Path** |
### Filter Function
The input field **Filter function** only appears when a catalog is set on ***custom***
In comparison to the filter from the File Module, the **Filter function** is more extensive and more exact. The usual filter just searching for file formats. With the **Filter function,** you can search for file formats and email attachments, files with specific sizes and more.
Because the **Filter function** has to be implemented into a script (see **Script** below) it can be customized to the own needs and wants.
After a function is defined in a script, the name of this function has to enter the input field **Filter function**. When this is done, the Filter function is ready and RealMigrator will use it during a migration.
### Catalog Order
Use the **up and down buttons** to change the order of the queries:
### Create and Delete catalogs
At the bottom of the page, there are buttons to create new catalogs or delete existing ones:
* To add further catalogs click the button **Catalog**
* To delete the last created catalog click the button **Last Catalog**
* To delete all of them, click the button **All**
## Log level and Reset code
| Setting | Explanation |
| ---------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Log level | Available log levels are **Error**, **Info** and **Debug**. |
| Reset Code | A client is always checking the own configurations after a defined period of time. This means that the client compares the selected client settings with the selected server settings. In addition the client connect to the server based on \[Module loop delay]\({{< ref "beginning/serverconfig/\_index.md#global-client-configuration" >}}), a defined period of time. By this time value the client is starting a Module loop and the client is checking the Reset code. If there is a Reset code change the client will takes over the new configuration. If there is no change, the client will still use the existing configuration. |
| Power mode | If this mode is activated, power down is prevented during migration. |
## Script
RealMigrator offers a default script for PST. But it is possible to edit the existing script or to write a new script.
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values, you have to click **Save** at the top of the page.
# Custom
Depending on the [Module Configuration](/modules/module-configuration) selection, the Custom Module contains different options and settings. When File or PST are selected, Custom Module will do the same as File or PST (For more information see [File documentation](/modules/file) and [PST documentation](/modules/pst).
The default configuration for Custom Module is **Server Migration**.
## Server Migration Settings
| Setting | Explanation |
| -------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Service only | If enabled, the server migration module will be executed only on clients running the RealMigrator service executable |
| Log level | Available log levels are **Error**, **Info** and **Debug** |
| Reset code | A client is always checking the own configurations after a defined period of time. This means that the client compares the selected client settings with the selected server settings. In addition the client connect to the server based on \[Module loop delay]\({{< ref "beginning/serverconfig/\_index.md#global-client-configuration" >}}), a defined period of time. By this time value the client is starting a Module loop and the client is checking the Reset code. If there is a Reset code change the client will takes over the new configuration. If there is no change, the client will still use the existing configuration. |
| Path blacklist | Paths which are listed in a blacklist are excluded from all catalogs and will not be part of migrations (see \[Well-known folder] ({{< ref "modules/wellknownfolder/\_index.md" >}}) for folder (paths) list) |
### Discover Sources
* If **Local drives** is set to **enabled**, all local drives are discovered.
* If **Local shares** is set to **enabled**, all local shares are discovered.
**Folder scan depth** = Defines how many subfolders levels should be discovered.
### Path list
| Setting | Explanation |
| ----------------- | ------------------------------------------------------- |
| Path | Migration path |
| Folder scan depth | Defines how many subfolders levels should be discovered |
## Migration Catalogs
### Source
* Paths define a source for a migration
* You can add or delete paths
### Destination
* Defines a destination for a migration: **OneDrive**, **Sharepoint** or **Blob Store**. If you select on of these destinations you have to configure different settings:
OneDrive:
* **User UPN**: Enter a the UPN of the user. You can leave this field empty and use the "GetUPNforPath()" script breakout for each direct subfolder (home share mode)
* **Root path**: Optional path in the target OneDrive "/realmigrator"
* **File selector**: Pipe-delimited list of files that should be in- or excluded from being copied
* **Backlist**: If enabled, all paths which are listed in **File selector** will be excluded from all migration rules
Sharepoint:
* **Sharepoint URL**: Enter the Sharepoint URL for the destination of the migration
* **Root path**: Optional path in the target document library "/realmigrator"
* **File selector**: Pipe-delimited list of files that should be in- or excluded from being copied
* **Backlist**: If enabled, all paths which are listed in **File selector** will be excluded from all migration rules
Blob Store:
* **Root path**: Optional path in the target blob store, e. g. "/realmigrator"
* **Diff file size**: Files exceeding this size will be uploaded blockwise. Default: 4 MB
* **Diff block size**: Block size for blockwise upload. Default: 1 MB
* **File selector**: Pipe-delimited list of files that should be in- or excluded from being copied
* **Backlist**: If enabled, all paths which are listed in **File selector** will be excluded from all migration rules
### Catalog Order
Use the **up and down buttons** to change the order of paths:
### Create and Delete catalogs
At the bottom of the page there are buttons to create new catalogs or delete existing ones:
* To add further catalogs click the button **Catalog**
* To delete the last created catalog click the button **Last Catalog**
* To delete all of them, click the button **All**
## Script
To create and configure own **Migration Rules** use **Script**
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values, you have to click **Save** at the top of the page.
# Server
The Server Module contains configurations for a **PST Import** server. Furthermore, this module contains settings for **Teams notifications**.
## Server Configuration
**Log level** = available are **Error**, **Info** and **Debug**.
Set **User Onboarding** to *enabled* to display the following settings:
| Settings | Explanation |
| ---------------- | -------------------------------------------------------------------------------- |
| Username | Please provide a user with **User administrator** role assigned and MFA disabled |
| Password | Password related to **Username** |
| Group membership | Comma-separated list fo Azure-AD groups. User will be put into each group |
Set **PST Import** to *enabled* to display the following settings:
| Settings | Explanation |
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Username | Please provide a user with **Mailbox Import Export** role assigned and MFA disabled |
| Password | Password of the mailbox importer |
| Target Path | Specifies the Mailbox folder that PST files are be imported to. Use **/** to import into the mailbox root, leave blank to generate the path from PST's source path |
| Archive | Enable this option to import the PSTs to the user's archive mailbox |
| Multi-Geo capabilities | Enable this option if your Exchange Online tenant is configured for multi-geo support |
| Remove completed requests | Enable this option if you want to remove completed import requests |
## Teams notification
When Teams notification is enabled, RealMigrator will send notifications and messages to a defined Microsoft Team. Therefore, it is necessary to enter a **Teams URL**.
Select or deselect the following notification reasons:
* **New client**
* **Client migration finished**
* **PST import finished**
## Script
To create and configure own **Migration Rules,** use **Script**
## Reset Button
If you click the **Reset** button in the **Config** tab or in the **Script** tab, all your entries will be reset to the RealMigrator default values.
To accept the default values, you have to click **Save** at the top of the page.
# Folder Lists
The following tables contain paths which are part of Upload Stores and Catalogs in the [File module](/modules/file) and in the [PST module](/modules/pst). Depending on Custom module settings (see [Module Configuration](/modules/module-configuration)) the paths are a part of [Custom module](/modules/custom) as well.
## General Folder List
This list contains paths for general folders, like Documents, Music or Desktop.
| Name | Path | ID |
| ------------------------ | -------------------------------------------------------------------- | ------------------------------ |
| Documents | %USERPROFILE%\Documents | FOLDERID\_Documents |
| Links | %USERPROFILE%\Links | FOLDERID\_Links |
| Music | %USERPROFILE%\Music | FOLDERID\_Music |
| Contacts | %USERPROFILE%\Contacts | FOLDERID\_Contacts |
| Desktop | %USERPROFILE%\Desktop | FOLDERID\_Desktop |
| Camera Roll | %USERPROFILE%\Pictures\Camera Roll | FOLDERID\_CameraRoll |
| Pictures | %USERPROFILE%\Pictures | FOLDERID\_Pictures |
| Account Pictures | %APPDATA%\Microsoft\Windows\AccountPictures | FOLDERID\_AccountPictures |
| Videos | %USERPROFILE%\Videos | FOLDERID\_Videos |
| Downloads | %USERPROFILE%\Downloads | FOLDERID\_Downloads |
| Favorites | %USERPROFILE%\Favorites | FOLDERID\_Favorites |
| Cookies | %APPDATA%\Microsoft\Windows\Cookies | FOLDERID\_Cookies |
| Temporary Internet Files | %LOCALAPPDATA%\Microsoft\Windows\Temporary Internet Files | FOLDERID\_InternetCache |
| History | %LOCALAPPDATA%\Microsoft\Windows\History | FOLDERID\_History |
| Libraries | %APPDATA%\Microsoft\Windows\Libraries | FOLDERID\_Libraries |
| Network Shortcuts | %APPDATA%\Microsoft\Windows\Network Shortcuts | FOLDERID\_NetHood |
| Printer Shortcuts | %APPDATA%\Microsoft\Windows\Printer Shortcuts | FOLDERID\_PrintHood |
| Programs | %APPDATA%\Microsoft\Windows\Start Menu\Programs | FOLDERID\_Programs |
| Quick Launch | %APPDATA%\Microsoft\Internet Explorer\Quick Launch | FOLDERID\_QuickLaunch |
| Recent Items | %APPDATA%\Microsoft\Windows\Recent | FOLDERID\_Recent |
| Ringtones | %LOCALAPPDATA%\Microsoft\Windows\Ringtones | FOLDERID\_Ringtones |
| Searches | %USERPROFILE%\Searches | FOLDERID\_SavedSearches |
| SendTo | %APPDATA%\Microsoft\Windows\SendTo | FOLDERID\_SendTo |
| Templates | %APPDATA%\Microsoft\Windows\Templates | OLDERID\_Templates |
| Start Menu | %APPDATA%\Microsoft\Windows\Start Menu | FOLDERID\_StartMenu |
| Startup | %APPDATA%\Microsoft\Windows\Start Menu\Programs\StartUp | FOLDERID\_Startup |
| Administrative Tools | %APPDATA%\Microsoft\Windows\Start Menu\Programs\Administrative Tools | FOLDERID\_AdminTools |
| Application Shortcuts | %LOCALAPPDATA%\Microsoft\Windows\Application Shortcuts | FOLDERID\_ApplicationShortcuts |
| Temporary Burn Folder | %LOCALAPPDATA%\Microsoft\Windows\Burn\Burn | FOLDERID\_CDBurning |
## Common Folder List
This list contains paths which related to all user profiles, like templates or administrative tools.
| Name | Path | ID |
| --------------------------- | ---------------------------------------------------------------------------- | -------------------------- |
| Common Administrative Tools | %ALLUSERSPROFILE%\Microsoft\Windows\Start Menu\Programs\Administrative Tools | FOLDERID\_CommonAdminTools |
| Common OEM Links | %ALLUSERSPROFILE%\OEM Links | FOLDERID\_CommonOEMLinks |
| Common Programs | %ALLUSERSPROFILE%\Microsoft\Windows\Start Menu\Programs | FOLDERID\_CommonPrograms |
| Common Start Menu | %ALLUSERSPROFILE%\Microsoft\Windows\Start Menu | FOLDERID\_CommonStartMenu |
| Common Startup | %ALLUSERSPROFILE%\Microsoft\Windows\Start Menu\Programs\StartUp | FOLDERID\_CommonStartup |
| Common Templates | %ALLUSERSPROFILE%\Microsoft\Windows\Templates | FOLDERID\_CommonTemplates |
## OneDrive Folder List
The following list contains paths for OneDrive migrations.
| Name | Path | ID |
| ------------------ | -------------------------------- | ----------------------------- |
| OneDrive | %USERPROFILE%\OneDrive | FOLDERID\_SkyDrive |
| OneDrive Documents | %USERPROFILE%\OneDrive\Documents | FOLDERID\_SkyDriveDocuments}, |
| OneDrive Pictures | %USERPROFILE%\OneDrive\Pictures | FOLDERID\_SkyDrivePictures |
# Own Blob Storage
To switch a project to an own blob storage, do the following:
| Task | Image |
| ----------------------------------------------------- | -------------------------------- |
| 1. Log in to [Azure Portal](https://portal.azure.com) | |
| 2. Go to **Storage accounts** |  |
| 3. Choose one of your blob storages | |
| 3. Then, navigate to **Access keys** |  |
Copy one of the two **Connection Strings**. A string looks like the following example:
> DefaultEndpointsProtocol=https;AccountName=myBlobAccountName;AccountKey=9aCzs76n0E7y5BpEi2gVsSv433BZa22leDOZXXeXIUa==;EndpointSuffix=core.windows.net
To create a blob storage container, obtain a container URL from Sample Container URL:
Concatenate a connection string + **;BlobEndpoint=** + container URL to blob configuration string
#### Sample
> DefaultEndpointsProtocol=https;AccountName=myBlobAccountName;AccountKey=9aCzs76n0E7y5BpEi2GvsSv433BZa22leDOZXXeXXIUa==;EndpointSuffix=core.windows.net\
> ;BlobEndpoint=
Navigate to myproject.realmigrator.com/config[^1] and enter a blob configuration string as External Storage. Then redeploy code and configuration files.
[^1]: Use an actual project name instead of 'myproject'
# Groups
With a group, it is possible to unify migrations for more than one client. The use of a group makes migration more comfortable.
The chapter **Groups** has different functions. A (super) administrator can select a date of migration. It is possible to select the modules which must be part of a migration and which not. Of course, adding groups is part of this menu too.
# Groups General Overview
This menu shows:
* A summary of all registered groups - **Groups (Total xx)**
* Add new groups function
* A filter function - **Filter by Client** (name search)
## Groups Settings
With a click on a group, the following menu appears:
This menu contains information about a group client:
* **Display Name** (Group Name)
* **Assigned Objects**
* Date selection
* Available modules
* Task of selected module (**Skip**, **Discover** and **Migrate** - explanation see table below)
| Command | Description | file module | PST module | inventory module |
| ------------ | ----------------------------------------------------------------------------------- | :---------: | :--------: | :--------------: |
| **skip** | module is not executed | + | + | + |
| **discover** | module discovers items the client and sends a report, but does not migrate any data | + | + | + |
| **migrate** | migrates data continuously | + | + | - |
# Groups Configurations
## Add a new group
1. Click **+Add** at the top of the page
2. Assign a group name
3. Click **Save**
## Set Module Configuration
1. Click **Start selection** at the top of the page
2. Click on one or more groups to select them
3. Click **Set Module Configuration**
4. Use the drop-down menus to configure the modules (**Skip**, **Discover** or **Migrate** - explanation see table below)
5. Click **Ok**
| Command | Description | file module | PST module | inventory module |
| ------------ | ----------------------------------------------------------------------------------- | :---------: | :--------: | :--------------: |
| **skip** | module is not executed | + | + | + |
| **discover** | module discovers items the client and sends a report, but does not migrate any data | + | + | + |
| **migrate** | migrates data continuously | + | + | - |
## Multiple group deletion
1. Click **Start selection** at the top fo the page
2. Click on one or more groups to select them
3. Click **Delete** to delete the chosen groups
## Users Overview
This overview shows all registered clients in the group. These listed clients are equal to the listed clients in the input field **Assigned Objects**, but the user overview offers more information about a client.
The following table will explain about this information (attributes)
| Attribute | Explanation |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Client | Client name based on a UPN |
| Last Message |
Message about the last status of the client. For example: Finished migration: Migration is done Sleeping: Migration is paused running file module: Running file migration
|
| Migration Status |
Finished migration (check mark) = Migration is done without errors Finished migration with error (exclamation mark) = Finished migration but with an error Sleeping migration or running migration and more (progress icon) = migration is not finished
|
| Progress | A progress bar |
| Timestamp | Date and time of **Last Message** |
| Object ID | Identification number of the current migration |
| Group ID | Identification number of group where the client is registered |
A click on a client name will give additional information. For this topic see the documentation chapter \[Clients Menu]\({{< ref "clientmenu/client-information/\_index.md" >}}).
## Assign to group
With **Assign to group,** it is possible to assign a client to a further group:
1. Click **Start selection** at the top of the menu
2. Choose one or more clients
3. Click **Assign to group** at the top of the menu
4. Choose a group
5. Click **Assign**
## Delete a group
To delete a group, click **Delete** in the upper-right corner of the screen.
# Clients Menu
The menu **Client** is an important menu in RealMigrator. In this menu, you can see all registered clients and their migration status. In addition, you can configure and edit your clients in this menu.
# Overview
The Clients menu in RealMigrator contains all clients which a part of the project.
This overview offers different functions:
* A summary of all registered clients - **Clients (Total xx)**
* A filter function - **Filter by Client** (name search)
* Seven different attributes
## Attributes
| Attribute | Explanation |
| ---------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| Client | Client name based on a UPN |
| Last Message |
Message about the last status of the client. For example: Finished migration: Migration is done Sleeping: Migration is paused running '\[example]' module: Migration for this module is running.
|
| Migration Status |
Finished migration (check mark) = Migration is done without errors Finished migration with error (exclamation mark) = Finished migration but with an error Sleeping migration or running migration and more (progress icon) = migration is not finished
|
| Progress | A progress bar |
| Timestamp | Date and time of **Last Message** |
| Object ID | Identification number of the current migration |
| Group ID | Identification number of group where the client is registered |
## Attributes order
As default, the list follows an alphabetic order (A to Z) refers to **Client**. With a click on the attribute **Client,** the order changed to a Z to A order.\
With clicks on other attributes, the list will deliver other orders.
## Assign to group
1. Click **Start selection** at the top of the menu
2. Choose one or more clients
3. Click **Assign to group** at the top of the menu
4. Choose a group
5. Click **Assign**
## Delete a client
1. Click **Start selection** at the top of the menu
2. Choose one or more clients
3. Click **Delete** in the upper-right corner of the menu
# Client Information
With a click on a client, the following menu appears:
This menu contains information about a single client:
* Date and time of **Last Message**
* UPN and E-Mail address of the client
* ObjectID and GroupID
* Custom Note about the client
Furthermore, this menu contains details about the migration progress of all modules
# Module Information
## Progress in general
In this example **File** and **PST** are skipped and **Inventory** is finalized
## Inventory Tab
The Inventory tab shows details about **bandwidth**, **drives** and **installerCache**
## File Tab
The File tab shows a list of migrated files and documents. Click the three red dots to see more details.
### **Synchronization states**
Name
Value
Description
st_check
0
Check for files to upload if the item is a directory, ask for a filter if it is a file. After checking, there are no files to upload.
st_use
1
Has files to upload if the item is a directory, upload if it is a file.
st_exclude
2
Do not check if the item is a directory, do not upload if it is a file.
st_delete
3
Delete remote.
st_sync
4
File was synchronized.
st_agent
5
Directory will be processed by upload agent.
## PST Tab
The PST tab shows a list of migrated PST files and PST documents. Click also the three red dots to see more details.
### **Synchronization states**
Name
Value
Description
st_check
0
Check for files to upload if the item is a directory, ask for a filter if it is a file. After checking, there are no files to upload.
st_use
1
Has files to upload if the item is a directory, upload if it is a file.
st_exclude
2
Do not check if the item is a directory, do not upload if it is a file.
st_delete
3
Delete remote.
st_sync
4
File was synchronized.
st_agent
5
Directory will be processed by upload agent.
## Custom
Depending on your configurations of the **Custom Module**, the Custom tab will show different results. When Custom is configured as PST or File it will look like them (see above). In the following example Custom is configured as **Server Migration**.
To add a catalog to a migration, click **Add catalog** on the following screen:
The submenu **Add catalog entry** appears. To add a catalog for a local filesystem, enter a name, add filters (optional) and/or enable or disable **Blacklist**. If Blacklist is enabled, RealMigrator will migrate all file formats except the formats that are listed above **Filter**.
After you enter a name and select filters, you have to select a source. **Source** and **Local filesystem** are selected as defaults. Choose your drives and files that you want to migrate.
As an alternative to **Local filesystem** you can choose a **Network share** as a source. Enter a **Path**, a **Username** and a **Password**. The username is used for authentication and the password authenticates the user.
After you chose your source, you have to select a destination for your migration. Click **Destination** to switch to destination settings. You have three destination options: **Blob storage**, **OneDrive** and **Sharepoint**.
As an option, you can enter a path on target blob storage. If you choose **OneDrive** you have an extra option where you can enter a UPN. And if you choose **Sharepoint,** you have an extra option where you can enter an Url.
# Reporting
To have an overview about running migrations, finished migrations or migrations errors, RealMigrator offers a reporting tool. This tool is **Reporting** - a menu with different filters to create various reports.
For further evaluations and analyses, reports are downloadable (as CSV and TSV) and customizable (via script).
# Reporting
In the Reporting menu there are three different filters: **Data Source**, **Report** and **Group**. Additionally, all of these filters have different attributes. Available selections depending on the attributes of other filters.
> For example: When a user choose PST for **Data Source**, only PST state and PST details are available for **Report**. If an user choose Progress for **Data Source** then there are four different settings are available in **Report**: Client State, Module State Per Client, Group Assignments and Group Commands.
The following table gives a detail overview of all three filters:
| Data Source | Report | Group |
| ----------- | ------------------------------------------------------------------------------------- | ----------------------------------------------------------- |
| Progress |
Client State Module State Per Client Group Assignments Group Commands
|
Unassigned Clients All Clients Existing groups
|
| Inventory |
Client State Module State Per Client Group Assignments Group Commands
|
Unassigned Clients All Clients Existing groups
|
| File |
File state File state anonymized
|
Unassigned Clients All Clients Existing groups
|
| PST |
PST state PST details
|
Unassigned Clients All Clients Existing groups
|
| Datasource | Report |
Unassigned Clients All Clients Existing groups
|
Choose filters and click **Run Query** to create a report.
The following example shows reports with the filters attributes **Progress**, **Client State** and **All Clients**:
## Filter Rows
Click **Filter Rows** to open the filters (upper-right corner above the list)
The filters appear:
Enter a single letter, a pair of a letter or a word and the filter will show a result which is equal to the entered letters.
For example, **in**:
The filter shows **in progress** and **finished** because **in** is part of both words.
Click **Filter Rows** again to close the filters.
## Download report
The example above contains information about all clients and their status of migration. Additionally, information about the groups where the client is assigned. To download a report, click **Download report** at the top of the page. Choose **As CSV file** or **As TSV file**:
## Advanced Report
To configure the report list in any own way, click **Advanced Report** at the top of the page:
# Customize Reports
Reports can be customized (added, removed or changed) by uploading a custom `reports.js` to [Binaries menu](/beginning/binaries).
## Reference
The JavaScript file must consist of an object, where each key is a data source for the report. The value for each data source is an array consisting of reports. Each array item is an object that needs to have the following properties:
| Property | Required | Description |
| ----------- | -------- | ------------------------------------------- |
| `key` | yes | The unique key for the report |
| `run` | yes | The run function to fetch the data |
| `text` | no | The text display in the selector |
| `extractor` | no | The function to extract the returned data |
| `headers` | no | An array of headers to display in the table |
The following sample shows the absolute minimal example for a report named **Report** in an own data source **datasource**. It just returns **Jon Doe** as data and displays it in the table.
```javascript
{
datasource: [{
key: 'report',
text: 'Report',
run: function(fetcher, groupId, extractor) {
return new Promise(function(resolve, reject) {
const result = extractor({ Name: 'Jon Doe' });
resolve(result);
});
},
extractor: function(json) {
return [{ name: json.Name }];
},
headers: [{
name: "Name",
key: "name"
}]
}]
}
```
### Run
**run: function** is used to return the data and has the following signature: `function(fetcher, groupId, extractor)`.
**Parameters:**
* fetcher: The fetcher object. See below for more details.
* groupId: The GroupID on which to operate. An empty string if called for **all** clients.
* extractor: The extractor function that is defined for the report. See below for more details.
**Returns:** A promise that will eventually resolve to an array of objects.
#### Extractor
**extractor: function** is used to return the data and has the following signature: `function(json)`.
**Parameters**\
json: Some json data from which to extract the needed information.
**Returns**\
An array of objects.
#### Fetcher
All functions return a promise.
| | |
| ---------------------------- | ------------------------------------------------------------ |
| fetch(path) | Fetch data from path |
| getModuleData (module, user) | Get module for "module" data for the specified user |
| fetchGroups() | Return all groups |
| fetchGroup(group\_id) | Return group with `group_id`. Use `*` for unassigned clients |
| fetchClients(group\_id) | Fetch all clients which belong to group `group_id`. |
#### Headers
Headers is an array of objects. Each object has the following members:
| Key | Description |
| ------------ | ---------------------------------------------------- |
| `name` | String. Test to display in the column header |
| `key` | String. Key used to access the value to be displayed |
| `width` | Number. Width of the column |
| `formatter` | Function. Formatter to use |
| `sortable` | Boolean. If this field is sortable |
| `resizable` | Boolean. If this field is resizable |
| `filterable` | Boolean. If this field is filterable |
**Sample:**
```javascript
{
name: "Group ID",
key: "group_id",
width: 250,
formatter: GroupIDFormatter,
sortable: true,
resizable: true,
filterable: true
}
```
#### Formatter
| Key | Description |
| ------------------- | --------------------------------------------------------------------------------------------------- |
| `ClientIDFormatter` | Formats a client ID to a link |
| `GroupIDFormatter` | Formats a group ID to a link |
| `ProgressFormatter` | Creates a progress bar |
| `SoftwareNameLink` | Generates a link for a search on |
| `DateTimeFormatter` | Formats a unix time stamp to a localized string |
| `BooleanFormatter` | Formats a boolean to either **yes** or **no** |
#### Utilities
**DataTime(unix\_timestamp)** = Creates a data time object from a unix time stamp.
### Samples
The following samples provide some common patterns that have been useful so far. Of course, these patterns can be combined with each other.
#### Remove Data Source
Included data source can be removed by overwriting the data source with a `null` value.
To remove PST data source, use `reports.js`:
```javascript
{
pst: null
}
```
#### Remove Report
Included reports can be removed by providing a report with the same key but a `null` value for the run function.
To remove File state report, use `reports.js`:
```javascript
{
file: [{
key: 'file-state',
run: null
}]
}
```
## Included Reports
### Data Sources
* `progress`
* `inventory`
* `file`
* `inventory`
### Reports
| Data Source | Key | Name |
| ----------- | ----------------------------------- | ----------------------- |
| `progress` | `get-client-state` | Client State |
| `progress` | `get-client-module-state` | Module State per Client |
| `progress` | `get-group-assignments` | Group Assignments |
| `progress` | `group-commands` | Group Commands |
| `inventory` | `system-summary` | System details |
| `inventory` | `software-installed-on-client` | Software summary |
| `inventory` | `software-installed-on-each-client` | Software details |
| `inventory` | `drive-mappings-per-client` | Drive mappings |
| `inventory` | `printers` | Printers |
| `file` | `file-state` | File state |
| `file` | `file-state-anonymous` | File state anonymized |
| `pst` | `pst-state` | PST state |
| `pst` | `pst-detail` | PST details |
### Source
These reports are already included in RealMigrator. They are provided here as a reference.
```javascript
{
progress: [{
key: 'get-client-state',
text: 'Client State',
run: function(fetcher, groupId, extractor) {
return fetcher.fetchClients(groupId).then((clients) => {
var progress = [];
for (var i in clients) {
progress.push(clients[i]);
}
return progress;
}).then((progress) => {
var arr = [];
for(var i in progress) {
const x = extractor(progress[i]);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json) {
const progressData = (progress) => {
var percent = 0.0;
var errors = 0;
var keys = progress ? Object.keys(progress) : [];
if(keys.length > 0) {
keys.forEach(key => {
var element = progress[key];
if((element.state !== 'skip') && element.total) {
percent += element.count / element.total;
if (element.errors) {errors += element.errors.length}
} else {
percent += 1.0;
}
});
percent /= keys.length;
}
return [percent, errors];
};
var p = json["Status"] || { };
var state = '';
if (p.state === 0) {
state = 'in progress';
} else if(p.state === 1) {
state = 'finished';
} else if(p.state === 2) {
state = 'error';
}
var percent = 0.0, errors = 0;
if (p.progress) {
[percent, errors] = progressData(p.progress)
}
return [{
group_id: json.Group ? json.Group.ObjectID : '',
group_name: json.Group ? json.Group.DisplayName : '',
client_id: json["ObjectID"],
display_name: json["DisplayName"],
state: state,
message: p ? p.message : "",
percent: percent,
errors: errors,
timestamp: json.Timestamp,
upn: json.UPN,
}];
},
headers: [{
name: "Group ID",
key: "group_id",
width: 250,
formatter: GroupIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group",
key: "group_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Client ID",
key: "client_id",
formatter: ClientIDFormatter,
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "UPN",
key: "upn",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "State",
key: "state",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Message",
key: "message",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Complete",
key: "percent",
formatter: ProgressFormatter,
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Errors",
key: "errors",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Last seen",
key: "timestamp",
width: 150,
sortable: true,
resizable: true,
filterable: true
}]
},{
key: 'get-group-assignments',
text: 'Group Assignments',
run: function(fetcher, groupId, extractor) {
return fetcher.fetchClients(groupId).then((clients) => {
return extractor(clients);
});
},
extractor: function(clients) {
var map = {};
for(var i in clients) {
const client = clients[i];
const memberId = client.GroupID || client.ObjectID;
if (!map[memberId]) {
const groupId = client.Group ? client.Group.ObjectID : '';
const groupName = client.Group ? client.Group.DisplayName : '';
map[memberId] = {
GroupID: groupId,
GroupName: groupName,
MemberID: memberId,
count: 1
};
} else {
map[memberId].count += 1;
}
}
var arr = []
for(var key in map) {
arr.push(map[key]);
}
return arr;
},
headers: [{
name: "Group ID",
key: "GroupID",
formatter: GroupIDFormatter,
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group Name",
key: "GroupName",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Member ID",
key: "MemberID",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Count",
key: "count",
width: 150,
sortable: true,
resizable: true,
filterable: true
}]
}, {
key: 'group-commands',
text: 'Group Commands',
run: function(fetcher, groupId, extractor) {
return fetcher.fetchModuleConfiguration(groupId).then((group_configs) => {
return extractor(group_configs);
});
},
extractor: function(group_configs) {
var arr = []
for(var i in group_configs) {
arr.push({
ObjectID: group_configs[i].ObjectID,
DisplayName: group_configs[i].DisplayName,
Inventory: group_configs[i].Config.inventory,
File: group_configs[i].Config.file,
PST: group_configs[i].Config.pst,
Custom: group_configs[i].Config.custom
});
}
return arr;
},
headers: [{
name: "Group ID",
key: "ObjectID",
formatter: GroupIDFormatter,
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group Name",
key: "DisplayName",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Inventory",
key: "Inventory",
width: 100,
sortable: true,
resizable: true,
filterable: true
}, {
name: "File",
key: "File",
width: 100,
sortable: true,
resizable: true,
filterable: true
}, {
name: "PST",
key: "PST",
width: 100,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Custom",
key: "Custom",
width: 100,
sortable: true,
resizable: true,
filterable: true
}]
}],
inventory: [{
key: 'system-summary',
text: 'System details',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'inventory', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
var system_ram = "";
var disk_model = "", disk_size = "";
var os_type = "", os_architecture = "";
var pc_model = "", pc_manufacturer = "", pc_serial = "";
var processor_name = "", processor_speed = 0, processor_id = "";
var video_processor = "", video_ram = 0;
var tpm_name = "", tpm_status = "";
if (json.wmiData) {
if (json.wmiData.ComputerSystem && json.wmiData.ComputerSystem.length) {
system_ram = json.wmiData.ComputerSystem[0].TotalPhysicalMemory;
}
if (json.wmiData.ComputerSystemProduct && json.wmiData.ComputerSystemProduct.length) {
pc_model = json.wmiData.ComputerSystemProduct[0].Version.length > json.wmiData.ComputerSystemProduct[0].Name.length ?
json.wmiData.ComputerSystemProduct[0].Version : json.wmiData.ComputerSystemProduct[0].Name;
}
if (json.wmiData.DiskDrive && json.wmiData.DiskDrive.length) {
disk_model = json.wmiData.DiskDrive[0].Model;
disk_size = json.wmiData.DiskDrive[0].Size;
}
if (json.wmiData.OperatingSystem && json.wmiData.OperatingSystem.length) {
os_type = json.wmiData.OperatingSystem[0].Caption;
os_architecture = json.wmiData.OperatingSystem[0].OSArchitecture;
}
if (json.wmiData.SystemEnclosure && json.wmiData.SystemEnclosure.length) {
pc_manufacturer = json.wmiData.SystemEnclosure[0].Manufacturer;
pc_serial = json.wmiData.SystemEnclosure[0].SerialNumber;
}
if (json.wmiData.Processor && json.wmiData.Processor.length) {
processor_name = json.wmiData.Processor[0].Name;
processor_speed = json.wmiData.Processor[0].CurrentClockSpeed;
processor_id = json.wmiData.Processor[0].ProcessorId;
}
if (json.wmiData.VideoController && json.wmiData.VideoController.length) {
video_ram = json.wmiData.VideoController[0].AdapterRAM;
video_processor = json.wmiData.VideoController[0].VideoProcessor;
}
if (json.wmiData.TPM && json.wmiData.TPM.length) {
tpm_name = json.wmiData.TPM[0].Name;
tpm_status = json.wmiData.TPM[0].Status;
}
}
arr.push({
client_id: client_id,
display_name: display_name,
system_ram: system_ram,
disk_model: disk_model,
disk_size: disk_size,
os_type: os_type,
os_architecture: os_architecture,
pc_model: pc_model,
pc_manufacturer: pc_manufacturer,
pc_serial: pc_serial,
processor_name: processor_name,
processor_speed: processor_speed,
processor_id: processor_id,
video_processor: video_processor,
video_ram: video_ram,
tpm_name: tpm_name,
tpm_status: tpm_status
})
return arr;
},
headers: [{
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Display Name",
key: "display_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "RAM size",
key: "system_ram",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Disk",
key: "disk_model",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Disk size",
key: "disk_size",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Operating System",
key: "os_type",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Architecture",
key: "os_architecture",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "PC manufacturer",
key: "pc_manufacturer",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "PC model",
key: "pc_model",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Serial #",
key: "pc_serial",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Processor",
key: "processor_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Speed",
key: "processor_speed",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Processor id",
key: "processor_id",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Video Processor",
key: "video_processor",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Video RAM",
key: "video_ram",
width: 140,
sortable: true,
resizable: true,
filterable: true
}, {
name: "TPM type",
key: "tpm_name",
width: 200,
sortable: true,
resizable: true,
filterable: true
}, {
name: "TPM status",
key: "tpm_status",
width: 140,
sortable: true,
resizable: true,
filterable: true
}]
}, {
key: 'software-installed-on-client',
text: 'Software summary',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'inventory', progressCB).then((inventories) => {
return extractor(inventories);
});
},
extractor: function(inventories) {
var map = {};
for(var i in inventories) {
const json = inventories[i];
if(json) {
for (var x in json.installerRegistry) {
for (var id in json.installerRegistry[x]) {
if(!map[id]) {
map[id] = {
packageCode: json.installerRegistry[x][id].packageCode,
name: json.installerRegistry[x][id].name,
version: json.installerRegistry[x][id].version,
count: 0
};
}
map[id].count += 1;
}
}
}
}
var arr = []
for(var key in map) {
arr.push(map[key]);
}
return arr;
},
headers: [{
name: "Name",
key: "name",
width: 400,
formatter: SoftwareNameLink,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Version",
key: "version",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Package Code",
key: "packageCode",
width: 300,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Install Count",
key: "count",
width: 250,
sortable: true,
resizable: true,
filterable: true
}]
}, {
key: 'software-installed-on-each-client',
text: 'Software details',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'inventory', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
for (var x in json.installerRegistry) {
for (var y in json.installerRegistry[x]) {
json.installerRegistry[x][y].group_id = json.GroupID;
json.installerRegistry[x][y].group_name = json.GroupName;
json.installerRegistry[x][y].client_id = client_id;
json.installerRegistry[x][y].display_name = display_name;
arr.push(json.installerRegistry[x][y])
}
};
return arr;
},
headers: [{
name: "Group ID",
key: "group_id",
width: 250,
formatter: GroupIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group",
key: "group_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Name",
key: "name",
width: 400,
formatter: SoftwareNameLink,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Package Code",
key: "packageCode",
width: 300,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Version",
key: "version",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Flags",
key: "flags",
width: 100,
sortable: true,
resizable: true,
filterable: true
}]
}, {
key: 'drive-mappings-per-client',
text: 'Drive mappings',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'inventory', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
if (json.wmiData && json.wmiData.MappdedDrives) {
for (var x in json.wmiData.MappdedDrives) {
arr.push({
group_id: json.GroupID,
group_name: json.GroupName,
client_id: client_id,
name: display_name,
drive: json.wmiData.MappdedDrives[x].Caption,
path: json.wmiData.MappdedDrives[x].ProviderName
})
}
}
return arr;
},
headers: [{
name: "Group ID",
key: "group_id",
width: 250,
formatter: GroupIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group",
key: "group_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Name",
key: "name",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Drive",
key: "drive",
width: 100,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Path",
key: "path",
width: 400,
sortable: true,
resizable: true,
filterable: true
}]
}, {
key: 'printers',
text: 'Printers',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'inventory', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
if (json.wmiData && json.wmiData.Printer && json.wmiData.Printer.length) {
for (var x in json.wmiData.Printer) {
json.wmiData.Printer[x].group_id = json.GroupID;
json.wmiData.Printer[x].group_name = json.GroupName;
json.wmiData.Printer[x].client_id = client_id;
json.wmiData.Printer[x].display_name = display_name;
arr.push(json.wmiData.Printer[x]);
}
}
return arr;
},
headers: [{
name: "Group ID",
key: "group_id",
width: 250,
formatter: GroupIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Group",
key: "group_name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Name",
key: "Name",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Printer Name",
key: "Caption",
width: 300,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Port",
key: "PortName",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Local",
key: "Local",
width: 100,
formatter: BooleanFormatter,
sortable: true,
resizable: true,
filterable: true
}]
}],
file: [{
key: 'file-state',
text: 'File state',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'file', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
const states = ["Check", "Incomplete", "Exclude", "Delete", "In Sync"];
var walk = function (dir) {
for (var f in dir.files) {
dir.files[f].catalog = cat;
dir.files[f].name = f;
dir.files[f].path = dir.path;
dir.files[f].client_id = client_id;
dir.files[f].display_name = display_name;
dir.files[f].state_cooked = states[dir.files[f].state];
dir.files[f].modified = DateTime(dir.files[f].modified);
if (dir.files[f].state === 4) {
dir.files[f].size_synced = dir.files[f].size;
} else if ((dir.files[f].state === 1) && dir.files[f].hashMap) {
dir.files[f].size_synced = dir.files[f].hashMap.size;
} else {
dir.files[f].size_synced = 0;
}
delete dir.files[f].hashMap;
arr.push(dir.files[f])
}
for (var d in dir.dirs) {
walk(dir.dirs[d]);
}
};
for (var cat in json.catalogs) {
walk(json.catalogs[cat]);
};
return arr;
},
headers: [{
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "File",
key: "name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Path",
key: "path",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size",
key: "size",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size Synced",
key: "size_synced",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "State",
key: "state_cooked",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Modified",
key: "modified",
width: 150,
formatter: DateTimeFormatter,
sortable: true,
resizable: true,
filterable: true
}]
},
{
key: 'file-state-anonymous',
text: 'File state anonymized',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'file', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
const states = ["Check", "Incomplete", "Exclude", "Delete", "In Sync"];
var d = new Date();
var t = Math.round(d.getTime()/1000);
function rounder(num, div) {
return Math.round(num/div) * div;
}
function namer(name) {
var dot = name.lastIndexOf(".");
if (dot > 0) { return name.substring(dot+1);}
else return "";
}
function sizer(size) {
return (rounder(size, 1000000000) ||
rounder(size, 1000000) ||
rounder(size, 1000) ||
rounder(size, 10));
}
function dater(date) {
return Math.floor((t-date)/(60*60*24*30));
}
function walk(dir) {
for (var f in dir.files) {
arr.push({name: namer(f), size: sizer(dir.files[f].size),
months_unchanged: dater(dir.files[f].modified),
state: states[dir.files[f].state]});
}
for (var d in dir.dirs) {
walk(dir.dirs[d]);
}
}
for (var cat in json.catalogs) {
walk(json.catalogs[cat]);
};
return arr;
},
headers: [{
name: "File",
key: "name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size",
key: "size",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Months unchanged",
key: "months_unchanged",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "State",
key: "state",
width: 150,
sortable: true,
resizable: true,
filterable: true
}]
}],
pst: [{
key: 'pst-state',
text: 'PST state',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'pst', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
const states = ["Check", "Incomplete", "Exclude", "Delete", "In Sync"];
for (var cat in json.catalogs) {
for (var f in json.catalogs[cat].files) {
json.catalogs[cat].files[f].catalog = cat;
json.catalogs[cat].files[f].path = f;
json.catalogs[cat].files[f].client_id = client_id;
json.catalogs[cat].files[f].display_name = display_name;
json.catalogs[cat].files[f].state_cooked = states[json.catalogs[cat].files[f].state];
json.catalogs[cat].files[f].modified = DateTime(json.catalogs[cat].files[f].modified);
if (json.catalogs[cat].files[f].state === 4) {
json.catalogs[cat].files[f].size_synced = json.catalogs[cat].files[f].size;
} else if ((json.catalogs[cat].files[f].state === 1) && json.catalogs[cat].files[f].hashMap) {
json.catalogs[cat].files[f].size_synced = json.catalogs[cat].files[f].hashMap.size;
} else {
json.catalogs[cat].files[f].size_synced = 0;
}
delete json.catalogs[cat].files[f].hashMap;
delete json.catalogs[cat].files[f].summary;
arr.push(json.catalogs[cat].files[f])
}
};
return arr;
},
headers: [{
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "PST",
key: "name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Path",
key: "path",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size",
key: "size",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size Synced",
key: "size_synced",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "State",
key: "state_cooked",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Modified",
key: "modified",
width: 150,
formatter: DateTimeFormatter,
sortable: true,
resizable: true,
filterable: true
}]
},{
key: 'pst-detail',
text: 'PST details',
run: function(fetcher, groupId, extractor, progressCB) {
return fetcher.getModuleDataForGroup(groupId, 'pst', progressCB).then((inventories) => {
var arr = [];
for(var i in inventories) {
const x = extractor(inventories[i], inventories[i].ObjectID, inventories[i].DisplayName);
arr = arr.concat(x);
}
return arr;
});
},
extractor: function(json, client_id, display_name) {
var arr = [];
const states = ["Check", "Incomplete", "Exclude", "Delete", "In Sync"];
for (var cat in json.catalogs) {
for (var f in json.catalogs[cat].files) {
json.catalogs[cat].files[f].catalog = cat;
json.catalogs[cat].files[f].path = f;
json.catalogs[cat].files[f].client_id = client_id;
json.catalogs[cat].files[f].display_name = display_name;
json.catalogs[cat].files[f].state_cooked = states[json.catalogs[cat].files[f].state];
json.catalogs[cat].files[f].modified = DateTime(json.catalogs[cat].files[f].modified);
if (json.catalogs[cat].files[f].state === 4) {
json.catalogs[cat].files[f].size_synced = json.catalogs[cat].files[f].size;
} else if ((json.catalogs[cat].files[f].state === 1) && json.catalogs[cat].files[f].hashMap) {
json.catalogs[cat].files[f].size_synced = json.catalogs[cat].files[f].hashMap.size;
} else {
json.catalogs[cat].files[f].size_synced = 0;
}
if (json.catalogs[cat].files[f].hashMap) {
json.catalogs[cat].files[f].blockmap_length = json.catalogs[cat].files[f].hashMap.blockmap ? json.catalogs[cat].files[f].hashMap.blockmap.length : 0;
json.catalogs[cat].files[f].hash = json.catalogs[cat].files[f].hashMap.hash;
json.catalogs[cat].files[f].hashmap_index = json.catalogs[cat].files[f].hashMap.nextIndex;
} else {
json.catalogs[cat].files[f].blockmap_length = 0;
json.catalogs[cat].files[f].hash = '';
json.catalogs[cat].files[f].hashmap_index = 0;
}
if (json.catalogs[cat].files[f].summary) {
json.catalogs[cat].files[f].encryption = json.catalogs[cat].files[f].summary.Encryption;
json.catalogs[cat].files[f].format = json.catalogs[cat].files[f].summary.Format;
json.catalogs[cat].files[f].summary_size = json.catalogs[cat].files[f].summary.Size;
json.catalogs[cat].files[f].version = json.catalogs[cat].files[f].summary.Version;
}
delete json.catalogs[cat].files[f].hashMap;
delete json.catalogs[cat].files[f].summary;
arr.push(json.catalogs[cat].files[f])
}
};
return arr;
},
headers: [{
name: "Client ID",
key: "client_id",
width: 250,
formatter: ClientIDFormatter,
sortable: true,
resizable: true,
filterable: true
}, {
name: "PST",
key: "name",
width: 250,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Path",
key: "path",
width: 400,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size",
key: "size",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Header Size",
key: "summary_size",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Format",
key: "format",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Encryption",
key: "encryption",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Version",
key: "version",
width: 80,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Size Synced",
key: "size_synced",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Blockmap length",
key: "blockmap_length",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Next Index",
key: "hashmap_index",
width: 80,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Hash",
key: "hash",
width: 270,
sortable: true,
resizable: true,
filterable: true
}, {
name: "State",
key: "state_cooked",
width: 150,
sortable: true,
resizable: true,
filterable: true
}, {
name: "Modified",
key: "modified",
width: 150,
formatter: DateTimeFormatter,
sortable: true,
resizable: true,
filterable: true
}]
}]
}
```
# Recipes
The following chapter presents main scripts and functions of RealMigrator.
**Recipes** start with **Group based actions**. It is about group membership.
The next one is **Time based actions**. It is about configuration of local time via script.
With **Set bandwidth** it is possible to change the maximum bandwidth or further general network settings.
**Dump Config** is about log and dump of global settings or module configuration data.
**Remove PSTs** shows a possibility to remove PSTs from outlook.
The last script in Recipes is **Self Service**. It is about creation a self-service portal for an own project.
# Group based actions
It is possible to determine a user´s group membership in **mainConfigCheck()** or any module config check function. To do so copy the following link:
```c++
def mainConfigCheck(globalSettings, config) {
var group = globalSettings["Group"];
if (type_name(group)=="Map") {
gkScript.logInfo("Group: " + group["DisplayName"] + "(" + group["ObjectID"] + ")");
if ("Offenbach" == group["DisplayName"]) {
// user is in group named "Offenbach"
}
else if ("b865c07b49ab454bbd1f41c7bb466d27" == group["ObjectID"]) {
// user is in group with id b865c07b49ab454bbd1f41c7bb466d27
}
} else {
// User is unassigned
}
}
```
# Time based actions
Copy the function below to the Global Configuration script. Call it in **getGroupId()**, **mainConfigCheck()** or any module config check function.
```c++
def isNight() {
var d=60*60*24;
var t = gkScript.time();
t += gkScript.utcOffset();
var localTime = t-t/d*d;
// night is before 28.800 = 8 * 60 *60 = 8:00 local time
// and after = 75.600 = 21 * 60 * 60 = 21:00 local time
var night = ((localTime < 28800) || (localTime > 75600))
return night;
}
```
The following script is an example to detect if it is a weekday on the client.
```c++
def isWeekend() {
var isWeekend = false;
try
{
var query = gkScript.wmiQuery("ROOT\\CIMV2", "SELECT * FROM win32_localtime");
var day = query[0]["DayOfWeek"];
gkScript.logDebug("Current day of week: " + to_string(day));
// day 0 = sunday; day 6 = saturday
if (day == 0 || day == 6)
{
isWeekend = true;
}
} catch (e)
{
gkScript.logError("Failed to retrieve day of week: " + e.what());
}
return isWeekend;
}
```
# Set bandwidth
Use **mainConfigCheck()** to change bandwidth settings.
To change some values use the following code:
```c++
def mainConfigCheck(globalSettings, config) {
if (this or that) {
config["bandwidthSettings"]["maxBandWidth"] = 200000;
config["bandwidthSettings"]["minSleepTime"] = 10;
}
}
```
To overwrite the complete bandwidthSettings object use the following code:
```c++
def mainConfigCheck(globalSettings, config) {
if (this or that) {
config["bandwidthSettings"] = [
"maxBandWidth": 10000,
"slowNetworkBarrier": 300,
"slowNetworkUsage": 0.25,
"unusableNetworkBarrier": 200,
"unusableWaitTime": 120,
"minSleepTime": 500,
"useMeteredConnection":true
];
}
}
```
# Dump Config
To log configuration data, to dump global settings or dump module configurations in order to know what to evaluate and what to change, it is possible to dump settings in **mainConfigCheck()** or any module configuration check function.\
To do so copy the following code:
```c++
def mainConfigCheck(globalSettings, config) {
gkScript.logInfo("globalSettings: \n" + to_json(globalSettings));
gkScript.logInfo("config: \n" + to_json(config));
}
```
# Remove PSTs
To remove PSTs from outlook usw the code below. Copy it to a global configuration script. In the **afterFinalize()** function, call **removePstFromOutlook(true)** or **removePstFromOutlook(false)**.\
If the boolean parameter is true, PST files are moved to the recycle bin.
```c++
def afterFinalize() {
removePstFromOutlook(true);
}
def isVirtualOutlook() {
var reg=gkScript.openRegistry();
return (!reg.readString("HKEY_LOCAL_MACHINE\\SOFTWARE\\WOW6432Node\\Microsoft\\Office\\16.0\\Common\\InstallRoot\\Virtual\\VirtualOutlook").empty() || !reg.readString("HKEY_LOCAL_MACHINE\\SOFTWARE\\WOW6432Node\\Microsoft\\Office\\15.0\\Common\\InstallRoot\\Virtual\\VirtualOutlook").empty());
}
def removePstFromOutlook(bool trash) {
var progid = "Outlook.Application";
var ol;
if (isVirtualOutlook()) {
ol = gkScript.createComObject(progid, false);
} else {
ol = gkScript.getComObject(progid);
if(!ol.connected()) {
gkScript.logInfo("failed to get outlook application, starting outlook..");
var reg=gkScript.openRegistry();
var path = reg.readString("HKEY_CLASSES_ROOT\\CLSID\\{0006F03A-0000-0000-C000-000000000046}\\LocalServer32\\")
if (path.empty()) {
path = reg.readString("HKEY_CLASSES_ROOT\\Wow6432Node\\CLSID\\{0006F03A-0000-0000-C000-000000000046}\\LocalServer32\\")
}
if (path.empty()) {
path = "C:\\Program Files (x86)\\Microsoft Office\\Root\\Office16\\OUTLOOK.EXE";
}
var b = path.find("\"");
var e = path.rfind("\"");
if (b < e) {path = path.substr(b+1, e-b-1);}
gkScript.logInfo("Found outlook at path '" + path + "'. Execute it...");
var handle = gkScript.runProcess(path);
gkScript.sleep(15 * 1000);
var hNotepad = gkScript.runProcess("notepad.exe");
gkScript.sleep(5 * 1000);
ol = gkScript.getComObject(progid);
gkScript.terminateProcess(hNotepad, 0);
}
}
if(!ol.connected()) {
gkScript.logError("failed to get outlook application");
return false;
}
var session = ol.getProp("Session");
if(!session.connected()) {
gkScript.logError("failed to get session");
return false;
}
var stores = session.getProp("Stores");
if(!stores.connected()) {
gkScript.logError("failed to get stores");
return false;
}
var count = stores.getProp("Count");
for(var i = 1; i <= count; ) {
var s = stores.invoke("Item", DISPATCH_PROPERTYGET, [i]);
if(s.connected()) {
if(!s.getProp("IsDataFileStore") || (to_string(s.getProp("ExchangeStoreType")) != "3")) {
i += 1;
continue;
}
var path = s.getProp("FilePath");
if(path.find(".pst") == -1) {
i += 1;
continue;
}
gkScript.logInfo("Store " + to_string(i) + " is a data file store type 3 located at " + path + ". Remove it...");
var folder = s.invoke("GetRootFolder", []);
if(folder.connected()) {
session.invoke("RemoveStore", [folder]);
count -= 1;
if (trash) {
gkScript.deleteFile(path, true);
}
} else {
gkScript.logInfo("Failed to get root folder");
i += 1;
}
}
}
ol.disconnect();
return true;
}
```
# Server Module Scripting
The Server Module offers a Script breakout when the client reaches finish state. In order to be called, you need to implement a ***onServerFinish(app)*** function. The ***app*** object is a ***ScriptApplication*** object that offers methods to interact with the finishing client:
```c++
class ScriptApplication {
bool loadModuleData(moduleName, jsonObject); // load module data into jsonObject
bool loadBlobData(path, data, offset, size); // load a Blob store chunk into data
bool getBlobSize(path, size); // get the size of a Blob
string onedriveSession(path); // create an upload session url in the users onedrive
bool onedriveUploadChunk(session, data, offset, totalsize);
// upload a chunk to the user's onedrive
bool onedriveCancelUpload(session); // cancel an upload
setProgress(progressData); // Report progress. The progressData Map object should
// have "total" and "completed" int members and a "message" string.
setError(msg); // Set an error message
}
```
The following script will load the user's PST module data, iterate through the uploaded PSTs and copy them into its Onedrive.
```c++
//
// config_server script
//
def copyBlobToOnedrive(app, file) {
var size = 0ull;
var blockSize = 4 * 1024 * 1024;
var uploadSize;
var offset;
var data;
if (!app.getBlobSize(file["blobPath"], size)) {
app.setError("Failed to get Blob size for '" + file["blobPath"] + "'");
file["status"] = "failed";
return false;
}
file["size"] = size;
var session = app.onedriveSession(file["onedrivePath"]);
if (session.empty()) {
app.setError("Failed to open Onedrive Session for '" + file["onedrivePath"] + "'");
file["status"] = "failed";
return false;
}
for (offset = 0ull; offset < size; offset += uploadSize) {
if (offset + blockSize <= size) {uploadSize = blockSize;}
else {uploadSize = size - offset;}
data = "";
if (!app.loadBlobData(file["blobPath"], data, offset, uploadSize)) {
app.setError("Failed to read chunk '" + file["blobPath"] + "' (" + to_string(offset) + ":" + to_string(uploadSize) + ")");
app.onedriveCancelUpload(session);
file["status"] = "failed";
return false;
}
if (!app.onedriveUploadChunk(session, data, offset, size)) {
gkScript.logInfo("Failed to upload chunk - retrying after sleep.");
gkScript.sleep(30000);
if (app.onedriveUploadChunk(session, data, offset, size)) {
gkScript.logInfo("Retry succeeded.");
} else {
app.setError("Failed to upload chunk '" + file["onedrivePath"] + "# (" + to_string(offset) + ":" + to_string(data.size()) + "/" + to_string(size) + ")");
app.onedriveCancelUpload(session);
file["status"] = "failed";
return false;
}
}
}
file["status"] = "finished";
return true;
}
def onServerFinish(app) {
var pst;
var moduleName = "pst";
var progress = ["total": 0, "completed": 0, "message": "Initializing PST copy...", "files": Map()];
var &files = progress["files"];
app.setProgress(progress);
if (!app.loadModuleData(moduleName, pst)) { return false; }
for (c:pst["catalogs"]) {
for (f:c.second["files"]) {
if (f.second["state"] != 4) {
gkScript.logError("Cannot migrate PST '" + f.first + "': not synced (" + to_string(f.second["state"]) + ")");
continue;
}
var file = Map();
file["blobPath"] = moduleName + f.second["remotePath"];
file["blobSize"] = f.second["size"];
file["blobDate"] = f.second["modified"];
file["onedrivePath"] = f.second["remotePath"] + "_backup";
files[f.first] = file;
gkScript.logInfo("PST: " + to_json(file));
}
}
progress["total"] = files.size()
for (f:files) {
progress["message"] = "Copying " + f.second["blobPath"];
app.setProgress(progress);
if (copyBlobToOnedrive(app, f.second)) {
progress["completed"] += 1;
}
}
progress["message"] = "Finished PST copy.";
app.setProgress(progress);
return true;
}
```
# Self Service
Create a self service portal by creating a HTML page in [Binaries](/beginning/binaries) under `selfservice/index.html`. The self service portal can be reached by own users under ]
A minimal working sample can be found below:
```html
realmigrator Self Service Portal
realmigrator Self Service Portal
```
[^1]: Use an actual project name instead of 'project'
# ChaiScript Reference
> *ChaiScript - an easy to use embedded scripting language for C++* (from )
**ChaiScript Overview** is about general ChaiScript commands.
**GK Script Object Reference** and the including subchapters offer information about various code and script samples for different operations and commands.
# ChaiScript Overview
ChaiScript is an embedded scripting language designed for C++. You can use ChaiScript to expand the functionality of RealmJoin. Use the following script examples and the scripts in the [GK Script Objects](/script/reference) section to learn how to customize your RealMigrator.
To download ChaiScript click the following link:
## Variables
```c++
var i; // uninitialized variable, can take any value on first assignment;
var k = 5; // initialized to 5 (integer)
var &m = k; // reference to k
```
## Built in Types
```c++
var i = 1; // creates an integer
var u = 1u; // creates an unsigned integer
var s = "hello"; // create a string
var f = 1.0; // creates a float
var v = [1,2,3u,4ll,"16", `+`]; // creates vector of heterogenous values
var m = ["a":1, "b":2]; // map of string:value pairs
```
### Working with strings
```shell
chaiCmd.exe
eval> var s="Hello"; // Assignment
Hello
eval> s+= " World !!"; // Concatenation
Hello World !!
eval> s.size(); // Size
14
eval> s.find("World"); // Find
6
eval> s[6]; // reference character
W
eval> s.substr(0, 6); // Substrings
Hello
eval> s.substr(6,-1);
World !!
eval> s.find("o"); // Find - reverse find
4
eval> s.rfind("o");
7
```
## Conditionals
```c++
if (expression) { }
if (expression) { }
else { }
```
## Loops
```c++
// c-style for loops
for (var i = 0; i < 100; ++i) { print(i); }
```
```c++
// while
while (some_condition()) { /* do something */ }
```
```c++
// ranged for
for (x : [1,2,3]) { print(x); }
```
Each of the loop styles can be broken using the break statement. For example:
```
while (some_condition()) {
/* do something */
if (another_condition()) { break; }
}
```
## Functions
```c++
def myfun(x, y) { x + y; } // last statement in body is the return value
def myfun(x, y) { return x + y; } // equiv
```
### Optionally Typed
```c++
def myfun(x, int y) { x + y; } // requires y to be an int
```
## Classes/Objects
### Class definition
```c++
class MyType {
var value;
def MyType() {
this.value = "a";
}
def get_value() {
"Value Is: " + this.value;
}
};
```
### Working with class objects
```c++
var m = MyType(); // calls constructor
print(m.get_value()); // prints "Value Is: a"
m.value += " joke";
print(m.get_value()); // prints "Value Is: a joke"
```
## JSON
| Script | Function |
| ----------- | ---------------------------------------------------------------------------------------------------------------------------------- |
| `from_json` | Converts a JSON string into its strongly typed (map, vector, int, double, string) representations |
| `to_json` | Converts a ChaiScript object (either a object or one of map, vector, int, double, string) tree into its JSON string representation |
# GK Script Object Reference
# Module Reference
gkScript is a chai module that offers different features:
* Subscripts
* [HTTP Reference](/script/reference/http)
* [Web browser](/script/reference/browser)
* [File system](/script/reference/filesystem)
* [Graph Reference](/script/reference/graph-reference)
* [MS Graph Authentication (and token acquisition)](/script/reference/graph-auth)
* [Hash Reference](/script/reference/hash)
* [Registry Reference](/script/reference/registry)
* [COM](/script/reference/com)
* [Self Update Reference](/script/reference/self-update)
## gkScript main object
All gkScript module features are available through a global gkScript object:
```c++
class gkScript
{
string executionType(); // returns "exe", "service" or "docker"
// Helper functions
time_t time();
time_t utcOffset();
string timeString(time_t val, bool convertToLocalTime = true);
int tick(); // tick-count in milliseconds
string expandString(string);
string getKnownFolder(string);
sleep(milliseconds);
string getCommandLine();
vector getArgList();
string getNetworkId(bool appendGateway);
int getConnectionCost(string host);
string getUserSID();
string getUserUPN();
string getMailAddress();
bool waitForNetwork(milliseconds);
map getUrlParameter(string url);
string urlEncode(string);
string pathEncode(string); // same as urlEncode, but does not encode '/'
string hexEncode(string);
string base64Encode(string);
string base64Decode(string);
string toLower(string);
string utf16ToUtf8(string);
string utf8ToUtf16(string);
string jScriptStringEncode(string);
bool matchWildCard(string pattern, string test);
int getCodePage();
string randomFill(size);
bool cryptProtect(string in, bool toLocalMachine, string &out);
bool cryptUnprotect(string in, string &out);
bool exportCertStore(string path, string password);
// Settings store
bool setString(Key, Value);
string getString(Key);
bool setInt(Key, Value);
integer getInt(Key);
// Logging functions
LogLevel getLogLevel();
setLogLevel(level);
logError(string);
logInfo(string);
logDebug(string);
bool exportLog(string);
setPowermode(bool on); // on: prevent idle power off
// WMI
map wmiQuery(string Path, string Query);
// Processes and Threads
bool shellExecute(hwnd, string verb, string path);
int executeProcess(string commandLine, string stdin, string &stdout, string &stderr, unsigned int timeout);
handle runProcess(string commandLine);
handle openProcess(int processId);
bool wait(handle, milliseconds); // can wait for processes, threads, events
bool terminateProcess(handle, int exitCode);
wmQuitProcess(int processId); // send WM_QUIT to all windows of that process
int getExitCode(handle);
handle runThread(string chaiCommand);
bool terminateThread(handle, int exitCode);
// Events
handle createEvent();
setEvent(handle);
resetEvent(handle);
// script tool
ScriptTool ScriptTool(bool with_gkScript);
};
```
## Helper functions
Most helpers are easy to understand. See below for some samples:
### urlEncode - getUrlParameter
```shell
eval> var url = "https://localhost/response?company=" + gkScript.urlEncode("Glück & Kanja") + "&message=" + gkScript.urlEncode("Schöne Grüße");
https://localhost/response?company=Gl%81ck+%26+Kanja&message=Sch%94ne+Gr%81%E1e
eval> var param=gkScript.getUrlParameter("https://localhost/response?company=Gl%81ck+%26+Kanja&message=Sch%94ne+Gr%81%E1e");
[, ]
eval> for (p : param) {print(p.first + ": " + p.second);}
company: Glück & Kanja
message: Schöne Grüße
```
### matchWildCard
| Script | Function |
| ----------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------- |
| `bool gkScript::matchWildCard(string pattern, string test)` |
Tests for a case insensitive match of test against pattern. Pattern may contain wild cards '\*' and '?'
Wait until an internet connection is established and returns true. If a timeout occurs, the function returns false
|
| `gkScript.waitForNetwork(0);` | Query network status |
| `gkScript::getNetworkId()` | Returns the hex encoded Id of the network (the MAC-Address of the Gateway) |
| `gkScript::getNetworkId(true)` | Append the IP address of the Gateway |
```shell
eval> gkScript.getNetworkId()
802AA8F1C3BD
eval> gkScript.getNetworkId(true)
802AA8F1C3BD@172.27.0.1
```
| Script | Function |
| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `gkScript::getConnectionCost(host)` | Returns the cost flags for a connection to the specified host. See for a definition of WCM\_CONNECTION\_COST flags |
## Logging
Logging amount is controlled by LogLevel values:
```c++
enum LogLevel
{
ll_ERROR, // Log errors
ll_INFO, // Log error + info
ll_DEBUG, // Log error + info + debug
ll_OFF, // Log nothing
};
```
Logging functions are part of the gkScript class:
```c++
var logLevel = gkScript.getLogLevel() // gets current LogLevel
gkScript.setLogLevel(ll_ERROR) // sets the LogLevel to Error
gkScript.logError(string) // Log an error message
gkScript.logInfo(string) // Log an info
gkScript.logDebug(string) // Log a debug string
bool gkScript.exportLog(string path) // Export the log to a file
```
## Tray Icon
gkScript offers the following functions to control the tray icon **(client.exe only)**:
```c++
class callbackObject
{
void onTrayClick();
}
bool showTrayIcon(callbackObject);
hideTrayIcon();
```
## WMI
| Script | Function |
| --------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `gkScript::wmiQuery(Path, Query)` |
Executes WMI queries. Path is the object path of the WMI namespace to query, e.g. ROOT\CIMV2 Query is the WQL query to execute.
|
The following code is a WMI query sample:
```c++
def printResults(wmi) {
var count = 0;
for (i:wmi) {
print (" Item " + to_string(count));
++count;
for (j:i) {
if (j.first[0] == '_') {continue;}
if (is_var_undef(j.second)) {print(" " + j.first + ": undefined");}
else{print(" " + j.first + ": " + to_string(j.second));}
}
}
}
print("Win32_DiskDrive");
var r=gkScript.wmiQuery("ROOT\\CIMV2", "SELECT Manufacturer, Model, Size FROM Win32_DiskDrive where Size>0");
printResults(r);
```
## Script Tool
The scriptTool class is a sandbox for ChaiScripts. You can call scripts without messing up your own environment...
| Script | Function |
| ---------------------------- | ----------------------------------------- |
| `gkScript.ScriptTool(true)` | gkScript object inside the sandbox |
| `gkScript.ScriptTool(false)` | Creates a sandbox without gkScript object |
{
} ScriptTool instances do not have an `exit()` command {}
```c++
class ScriptTool
{
bool addScript(string script);
object callFunction(string func);
void reset(); // delete all defined variables and functions
};
```
The following code is a ScriptTool sample:
```shell
eval> var script=gkScript.ScriptTool(true);
eval> script.callFunction("gkScript.getVersion()");
0.2.0.0
eval> script.addScript("def testFunction(val) {return \"testFunction - \" + val;}");
true
eval> script.callFunction("testFunction(\"hello world\")");
testFunction - hello world
eval> script.reset();
eval> script.callFunction("testFunction(\"hello world\")");
[2017-12-01 18:26:11.946] [1668] [error] [Default] ScriptTool::callFunction: Eval exception thrown: Error: "Can not find object: test" during evaluation at (__EVAL__ 1, 1)
eval>
```
# HTTP Reference
## Http transfer
| Script | Function |
| -------------------------------------------------------------------------------------- | --------------------------- |
| `bool gkScript::fetchUrl(string url, string &response, mapheader = [])` | Fetches a URL into a string |
Samples:
```c++
var data = "";
gkScript.fetchUrl("https://www.dropbox.com/", data);
gkScript.fetchUrl("https://www.dropbox.com/", data, ["Accept-Charset": "utf-8", "Accept-Language": "en-US"]);
```
## HttpResponse Object
| Script | Function |
| --------------------------------------------------------------------------------------------------------------------------- | ---------------------------- |
| `HttpResponse gkScript::httpRequest(string url, method httpMethod, mapheader = [], string requestBody = "")` | Creates a HttpResponse class |
The following methods are defined:
```c++
enum http_methods // supported methods for gkScript::httpRequest
{
GET,
POST,
HEAD,
PUT,
PATCH,
DELETE,
TRACE,
OPTIONS,
CONNECT
};
```
### HttpResponse class
```c++
class HttpResponse
{
statusCode; // unsigned int
responseBody; // string
responseHeader; // map of string:string pairs
string getHeader(string header); // get response-header with case insensitive header name
};
```
### Http transfer sample
The following sample is a combination of the script from **Http transfer** and **HttpResponse Object**. It uses both to get the headers of an URL.\
Then a **GET** method with range header to retrieve the first 100 bytes of the URL.
```shell
eval> var response=gkScript.httpRequest("http://www.glueckkanja.com/", HEAD);
eval> for (h: response.responseHeader) {print(" " + h.first + ": " + h.second);}
: HTTP/1.1 200 OK
Accept-Ranges: bytes
Arr-Disable-Session-Affinity: true
Cache-Control: no-cache,public,max-age=1036800
Content-Length: 45170
Content-Type: text/html
Date: Tue, 07 Nov 2017 10:12:53 GMT
ETag: "4612133abe54d31:0"
Last-Modified: Fri, 03 Nov 2017 16:10:11 GMT
Server: Microsoft-IIS/8.0
Strict-Transport-Security: max-age=31536000
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
X-Powered-By: ASP.NET
eval> var response2=gkScript.httpRequest("http://www.glueckkanja.com/", GET, ["Range": "bytes=0-100"]);
eval> response2.responseBody
Starts a web browser object and navigates to an entered URL. The method will return when the browser is closed. |
A simple web browser sample:
```c++
print("start");
gkScript.showBrowser("http://www.google.de")
print("end");
```
## Web Browser Object
| Script | Function |
| ----------------------- | --------------------------------------- |
| `gkScript.WebBrowser()` | Returns a web browser object |
| `WebBrowser::Show` | Defines a callback object and submit it |
Web browser object example:
```c++
class WebBrowser
{
void Show(uri, callbackObject, bool modal = true);
void Show(uri, callbackObject, integer flags);
bool IsVisible(); // true if browser is currently shown
uint64_t GetWindowHandle();
void Close();
void SetFullScreen(bool fullScreen);
bool SetContent(string HTML);
bool Navigate(string uri);
bool EvalJavaScript(string code);
// flags
interger showFullScreen; // show full screen browser. cannot be combined with showFlash
integer showFlash; // show flash-style browser.
integer showFrameless; // no frame, titlebar and close button on browser window
integer showModal; // Show() returns when browser is closed.
}
```
Callback object and submit:
```c++
class callbackObject
{
bool onBeforeNavigate(string url);
void onDocumentComplete(string url);
void onBrowserCallback(string data);
}
```
Web browser sample - callback object:
```c++
class browserController
{
def browserController() {
this.browser = gkScript.WebBrowser();
};
def Show(url) {
this.browser.Show(url, this);
}
def ShowFlash(url) {
var flags = gkScript.WebBrowser.showFlash | gkScript.WebBrowser.showFrameless | gkScript.WebBrowser.showModal;
this.browser.Show(url, this, flags);
}
def onBeforeNavigate(str) {
gkScript.logDebug("onNavigate called " + str);
if (str.find("cat") != -1) {
gkScript.logError("cat alert " + str);
this.browser.SetContent("Cats detected
Shame on you!
We catched you seraching for cats.
");
return false;
}
else if ((str.find("coke") != -1) || (str.find("cola") != -1)) {
gkScript.logInfo("coke redirection " + str);
this.browser.Navigate("http://www.pepsi.com");
return false;
}
return true;
}
def onBrowserCallback(str) {}
var browser;
}
var c = browserController();
c.Show("http://www.google.de");
c.ShowFlash("http://www.google.de");
```
# File System
## gkScript main object
The global gkScript object offers the following filesystem functions:
```c++
class gkScript
{
...
// Local filesystem
bool loadFile(string path, string content);
bool loadFile(string path, string content, size_t offset, size_t size);
string readStdin();
bool saveFile(string path, string content, bool overwrite = false);
bool saveFile(string path, string content, size_t offset);
bool setFileSize(string path, size_t size);
bool setFileAttributes(string path, int attributes);
bool copyFile(string sourcePath, string targetPath, bool overwrite);
bool moveFile(string sourcePath, string targetPath);
bool deleteFile(string path, bool recycleBin = false);
bool createDirectory(string path);
bool deleteDirectory(string path);
vector listDirectory(string path);
bool getFileInformation(string path, DirectoryEntry &info);
map getFileSummary(string path);
map enumDrives();
bool addNetworkConnection(string path, string user, string password, string localDrive = "");
bool deleteNetworkConnection(string path, bool force = false);
...
};
```
## Directory Content
| Script | Function |
| -------------------------------------- | --------------------------------------------- |
| `gkScript::listDirectory(string path)` | Returns an array of **DirecotyEntry** objects |
DirectoryEntry:
```c++
class DirectoryEntry
{
isFolder; // bool
name; // string
path; // string
attributes; // uint32_t
size; // int64_t
created, modified; // time_t
}
```
**listDirectory** sample:
```shell
eval> var list=gkScript.listDirectory("c:");
eval> for (entry : list){print(entry.name);}
$Recycle.Bin
$WINDOWS.~BT
bootmgr
BOOTNXT
Config.Msi
Documents and Settings
MATS
OneDriveTemp
pagefile.sys
PerfLogs
Program Files
Program Files (x86)
ProgramData
Recovery
swapfile.sys
System Volume Information
Users
Windows
```
## Ranged read/write operations
| Script | Function |
| ------------------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| `gkScript::loadFile(path, content, offset, size)` | Can be used to load a part of a file |
| `gkScript::getFileInformation(path, &info)` | Can be used to retrieve the file size |
| `gkScript::saveFile(path, content, offset)` |
Can be used to write a part of the file. The write size is the size of the content string.
Returns a map of key:value pairs with summary information about the file. Supported file types are: - Structured storage file with summary information (.msi, .msg, .doc, .xls, thumps.db,...) - PST files
|
Dump MSI installer cache:
```c++
var dir=gkScript.listDirectory(gkScript.expandString("%windir%\\installer"));
var i=0;
for (entry : dir) {
if (!entry.isFolder) {
var info=gkScript.getFileSummary(entry.path);
if (!is_var_undef(info["MSI Package Code"])) {
++i;
print(entry.name + " (" + entry.path + ")");
print(" Package Code " + to_string(info["MSI Package Code"]));
if (!is_var_undef(info["Subject"])) {print(" " + to_string(info["Subject"]));}
}
}
}
print(to_string(i) + " Packages found.");
```
## Read console input
| Script | Function |
| ------------------------------ | --------------------------------------------- |
| `string gkScript::readStdin()` | Reads from stdin until end of file is reached |
Console input sample:
```
>echo Hello %username%|chaiCmd.exe -c "var in=gkScript.readStdin(); print(\"input: \" + in);"
input: Hello christoph
```
## Read console input
| Script | Function |
| ------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `gkScript::addNetworkConnection(path, user, password, localDrive)` |
Creates a network connection. If user and password are empty, the current user context is used to connect. If localDrive is present, the connection is permanent and mapped to the drive. Use the format "X:" for localDrive.
Removes a network connection. Path can be eithe a UNC path or a local drive.
|
# Graph Reference
### GraphConnection class
All Graph operations are done with the GraphConnection class.
| Script | Function |
| ----------------------------- | ------------------------ |
| `gkScript::GraphConnection()` | Creates a class instance |
**GraphConnection** offers the following features:
* [Authentication, token acquisition](/script/reference/graph-auth)
* OneDrive file operations
```c++
class GraphConnection
{
// connection state
bool isConnected();
string getUserId();
// Authentication
string getAuthorizeUrl(string login_hint);
string getResponseUrl();
bool setAuthorizationResponse(string ResponseUrl);
bool hasAdminScope();
string getAdminConsentUrl();
string getAdminResponseUrl();
bool checkAdminConsentResponse(string ResponseUrl);
// OneDrive
class Drive
{
string id;
string name;
string description;
string url;
int64 sizeTotal, sizeFree;
time_t created;
};
class DriveItem
{
string id;
string driveId;
bool isFolder;
string name;
string path;
string downloadUrl;
int64 size;
time_t created, modified;
};
bool getPersonalDrive(Drive &);
bool getDriveItem(DriveItem &, string driveId, string path);
vector getChildItems(DriveItem);
bool createChildFolder(DriveItem &newFolder, DriveItem parent);
bool renameMoveItem(DriveItem &, string newName, string newParentItemId);
bool deleteItem(DriveItem);
bool uploadFile(DriveItem &, DriveItem parent, string fileName, string localFilePath);
bool uploadFile(DriveItem &, string localFilePath);
bool uploadFile(DriveItem &, string localFilePath, string sessionUrl);
bool setCallback(callbackObject);
};
```
### OneDrive file operations
In order to work with OneDrive, you need a drive object.
| Script | Function |
| -------------------------------------------- | --------------------------------------------- |
| `GraphConnection::getPersonalDrive(Drive &)` | Get the primary OneDrive assigned to the user |
All further file operations work on DriveItems, which could be either folders or files.
| Script | Function |
| -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------ |
| `GraphConnection:: getDriveItem(DriveItem &, string driveId, string path)` |
Opens a DriveItem. To open the root directory of a drive, use the path "/"
|
| `GraphConnection:: getChildItems(DriveItem &)` | Get directory contents |
**List directory sample**
```c++
def listDirectory(path) {
var graph=gkScript.GraphConnection();
if (!graph.isConnected()) {
print("no graph connection.");
return;
}
var drive = graph.Drive();
if (!graph.getPersonalDrive(drive)) {
print("no personal drive.");
return;
}
print("Drive info:");
print(" id: " + drive.id);
print(" name: " + drive.name);
print(" description: " + drive.description);
print(" url: " + drive.url);
print(" sizeTotal: " + to_string(drive.sizeTotal));
print(" sizeFree: " + to_string(drive.sizeFree));
print(" created: " + gkScript.timeString(drive.created));
var root = graph.DriveItem();
if (!graph.getDriveItem(root, drive.id, path)){
print("Could not open path " + path);
return;
}
var list = graph.getChildItems(root);
print("Content of '" + root.path + "':");
for (entry : list){
var str = " ";
if (entry.isFolder) {str += " ";}
else {str += " ";}
str += entry.name;
print (str);
print(" id: " + entry.id);
print(" size: " + to_string(entry.size));
print(" created: " + gkScript.timeString(entry.created));
print(" modified: " + gkScript.timeString(entry.modified));
}
print(to_string(list.size()) + " items total.");
}
listDirectory("/");
```
| Script | Function |
| -------------------------------------------------------------------------------------- | --------------------------------- |
| `GraphConnection::renameMoveItem(DriveItem &, string newName, string newParentItemId)` | Rename and/or move DirectoryItems |
| `GraphConnection::renameMoveItem(item, "newName", "")` | Will rename item to "newName" |
| `GraphConnection::renameMoveItem(item, "", folderXY.id)` | Will move item to folderXY |
| `GraphConnection::renameMoveItem(item, "newName", folderXY.id)` | Will move and rename item. |
**Basic file operations sample**
```c++
var graph=gkScript.GraphConnection();
// check connection
if (!graph.isConnected()) {
print("no graph connection.");
return;
}
// get onderive drive object
var drive = graph.Drive();
if (!graph.getPersonalDrive(drive)) {
print("no personal drive.");
return;
}
// get onedrive root folder
var root = graph.DriveItem();
if (!graph.getDriveItem(root, drive.id, "/")){
print("no root.");
return;
}
print("Opened root " + root.path);
var folderA = graph.DriveItem();
folderA.name = "folderA";
if (graph.createChildFolder(folderA, root)){
print("Created folder " + folderA.path);
}
var folderB = graph.DriveItem();
folderB.name = "the folder b";
if (graph.createChildFolder(folderB, folderA)){
print("Created folder " + folderB.path);
}
// move folderB to root
if (graph.renameMoveItem(folderB, "", root.id)){
print("Moved folderB to " + folderB.path);
}
// cleanup
graph.deleteItem(folderA);
graph.deleteItem(folderB);
```
### File upload
| Script | Function |
| --------------------------------------------------------------------------------------------------- | ------------------------------------------------------------- |
| `GraphConnection::uploadFile(DriveItem &, DriveItem parent, string fileName, string localFilePath)` | Will upload new files |
| `GraphConnection::uploadFile(DriveItem &, std::string localFilePath)` | Will upload existing files |
| `GraphConnection::uploadFile(DriveItem &, std::string localFilePath, std::string sessionUrl)` | Will upload files with an externally generated Upload session |
All upload methods overwrite existing content, and can use a callback object to report upload progress.
| Script | Function |
| ---------------------------------------------- | -------------------------- |
| `GraphConnection::setCallback(callbackObject)` | Will set a callback object |
```c++
class callbackObject
{
void progressCallback(int sizeCompleted, int sizeTotal);
}
```
**File upload sample**
```c++
var graph=gkScript.GraphConnection();
// check connection
if (!graph.isConnected()) {
print("no graph connection.");
return;
}
class Progress
{
def Progress(int Sleeptime, name) {
this.sleepTime = Sleeptime;
this.name = name;
}
def progressCallback(int sizeCompleted, int sizeTotal){
print(this.name + " [ " + to_string(sizeCompleted) + " / " + to_string(sizeTotal) + " ]");
// sleep if we need to upload more than 4 k...
if (sizeTotal - sizeCompleted > 4096) {gkScript.sleep(this.sleepTime);}
}
var sleepTime;
var name;
}
// create local file
var file = gkScript.expandString("%temp%\\upload.txt");
gkScript.saveFile(file, "hello world!", true);
// get onderive drive object
var drive = graph.Drive();
if (!graph.getPersonalDrive(drive)) {
print("no personal drive.");
return;
}
// get onedrive root folder
var root = graph.DriveItem();
if (!graph.getDriveItem(root, drive.id, "/")){
print("no root.");
return;
}
print("Opened root " + root.path);
// upload...
var progress = Progress(2000, file + " -> /uploadtest.txt")
graph.setCallback(progress);
var upload = graph.DriveItem();
if (graph.uploadFile(upload, root, "uploadtest.txt", file)) {
print("Successfully uploaded ");// + file + " to " + upload.path);
}
// cleanup
graph.deleteItem(upload);
gkScript.deleteFile(file);
```
# MS Graph Authentication
## User Authentication
| Script | Function |
| -------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `GraphConnection::isConnected()` |
Will test if a user has a valid token. Note: Script will return false if the client is offline. Check internet acces before using any GraphConnection features.
|
| | If **GraphConnection** returns **true**, `GraphConnection::getUserId()` will return the token principial name. |
In order to request a new token, you need to proceed the following steps:
1. Start a web browser with the uri returned from `GraphConnection::getAuthorizeUrl(login_hint)`.\
The login\_hint can be a UPN if you know the user's UPN, or "".
2. Check if the browser is redirected to an uri starting with the string returned by `GraphConnection::getResponseUrl()`.
3. Put the redirect uri from the browser to `GraphConnection::setAuthorizationResponse(string ResponseUrl)`.\
If the call succeeds, the user received a valid token.
### Authentication sample
```c++
print("start");
class graphAuthentication
{
def graphAuthentication() {
this.set_explicit(true);
this.browser = gkScript.WebBrowser();
this.graph = gkScript.GraphConnection();
if (this.graph.isConnected()) {
var str = this.graph.getUserId();
this.startPage = "Account OK
";
this.browser.SetContent(str);
return false;
}
}
gkScript.logDebug("opening target");
return true;
}
var browser;
var graph;
var startPage;
var responseUrl;
}
gkScript.setLogLevel(ll_DEBUG);
var ga = graphAuthentication();
ga.Show();
print("end");
```
# Hash Reference
## gkScript main object
| Script | Function |
| ------------------------------------------------- | ---------------------------------------------------------------- |
| `string gkScript::hashMD5(string data)` | Use it to MD5-hash the data string. A binary has string returns. |
| `gkScript::hexEncode` or `gkScript::base64Encode` | Get an appropriate string representation |
| `Hash gkScript::createMD5Hasher()` | Will create a hash object |
## gkScript hash object
```c++
class Hash
{
reset();
update(string data);
string finalize();
};
```
#### Hash sample
```shell
eval> var md5 = gkScript.hexEncode(gkScript.hashMD5("The quick brown fox jumps over the lazy dog"));
9E107D9D372BB6826BD81D3542A419D6
eval> var hasher = gkScript.createMD5Hasher();
eval> hasher.update("The quick brown fox");
eval> hasher.update(" jumps over the lazy dog");
eval> gkScript.hexEncode(hasher.finalize());
9E107D9D372BB6826BD81D3542A419D6
```
# Registry Reference
## gkScript Main Object
| Script | Function |
| -------------------------- | ----------------------------- |
| `gkScript::openRegistry()` | Will create a registry object |
## gkScript Registry Object
```c++
enum RegistryTypes {
REG_DWORD,
REG_BINARY,
REG_SZ,
REG_EXPAND_SZ,
REG_MULTI_SZ,
REG_LINK,
REG_NONE,
REG_DWORD_LITTLE_ENDIAN,
REG_DWORD_BIG_ENDIAN,
REG_QWORD,
REG_QWORD_LITTLE_ENDIAN,
};
class Registry
{
int readDWORD(string path); // returns 0 on error
bool writeDWORD(string path, int val);
string readString(string path); // returns "" on error
bool writeString(string path, string val);
bool readValue(string path, string &value, RegistryTypes &type);
bool writeValue(string path, string value, RegistryTypes type);
bool deleteValue(string path);
bool createKey(string path);
bool deleteKey(string path);
vector enumSubkeys(string path);
vector enumValues(string path);
};
```
| Registy paths and root keys |
| --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| For all operations, use fully qualified path strings. |
| Paths strings to values include the value name. |
| To address the the key's unnamed or default value, e.g. "HKLM\Software\\" for the default value of the key "HKEY\_LOCAL\_MACHINE\Software". |
|
Paths must start with one of the following default keys, or it's appreviation: HKCR or HKEY\_CLASSES\_ROOT HKCU or HKEY\_CURRENT\_USER HKLM or HKEY\_LOCAL\_MACHINE HKU or HKEY\_USERS HKCC or HKEY\_CURRENT\_CONFIG HKPD or HKEY\_PERFORMANCE\_DATA
|
| The Registry always opens the 64-bit registry paths. To use the 32-bit path you need explicit addressing, e.g. HKEY\_LOCAL\_MACHINE\SOFTWARE\Wow6432Node |
#### Strings
| Script | Function |
| ---------------------- | --------------------------------------------------------------------------------------------------- |
| `Registry::readString` | Will return UTF8 strings |
| `Registry::readValue` | Will return UTF16 strings for all registry string types (REG\_SZ, REG\_EXPAND\_SZ, REG\_MULTI\_SZ). |
Convert the string returned by `Registry::readValue` with `gkScript.utf16ToUtf8`
#### Registry sample
```c++
// Dump installed software from registry
def dumpSoftware(string user) {
var reg=gkScript.openRegistry();
var path = "HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Installer\\UserData\\" + user + "\\Products";
var software = reg.enumSubkeys(path);
for (s:software) {
var sPath = path + "\\" + s + "\\InstallProperties\\"
var name = reg.readString(sPath + "DisplayName");
var version = reg.readString(sPath + "DisplayVersion");
var t=REG_NONE;
var uninst = "";
reg.readValue(sPath + "UninstallString", uninst, t);
if (!name.empty()) {
print(" " + name);
print(" version: " + version);
print(" uninstall info: " + to_string(t) + " '" + gkScript.utf16ToUtf8(uninst) + "'");
}
}
}
var reg=gkScript.openRegistry();
var users = reg.enumSubkeys("HKEY_LOCAL_MACHINE\\SOFTWARE\\Microsoft\\Windows\\CurrentVersion\\Installer\\UserData")
for (u:users) {
print("Software installed for user " + u);
dumpSoftware(u);
}
```
# COM
## Object creation
| Script | Function |
| --------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `ComObject gkScript.createComObject(string id, bool inProcess)` |
Will create a COM object. - id can be a CLSID or a ProgID - Set inProcess to true for in-process objects, false for local servers.
Will attach to a running COM server. - id can be a CLSID, a ProgID or a Moniker
|
## ComObject class
```c++
enum InvokeTypes {
DISPATCH_METHOD,
DISPATCH_PROPERTYGET,
DISPATCH_PROPERTYPUT,
DISPATCH_PROPERTYPUTREF
};
class ComObject
{
bool connected(); // return true if ComObject is connected to COM server
void disconnect();
value invoke(string name, vector params); // invoke a function
value invoke(string name, InvokeTypes type, vector params); // invoke a function with specified invoke type
value getProp(string prop); // read property
bool setProp(string prop, val); // write property
}
```
Sample:
```shell
eval> var inet = gkScript.createComObject("InternetExplorer.Application", false);
eval> inet.connected();
true
eval> inet.setProp("Visible", true);
true
eval> inet.getProp("HWND");
3736394
eval> inet.invoke("Navigate", ["http://www.google.de"])
eval> inet.invoke("Close", [])
[2018-02-23 11:29:43.892] [9516] [error] [Default] gkScript::getProp can't get id for Close
eval> inet.invoke("Quit", []);
eval> inet.connected()
true
eval> inet.disconnect()
eval> inet.connected()
false
eval>
```
# Self Update Reference
## Installation
gkScript methods:
* Install itself to the machine and set an auto-run key to start on every logon.
* Self-update an installed client.
gkScript also ensures that the installed client is running as a single instance per logon session.
| Script | Function |
| -------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `string gkScript::getVersion()` | Version string of the current client-xxx.exe / chaiCmd-xxx.exe module |
| `string gkScript::moduleFilename(bool fullPath = false)` | Module file name of the current client-xxx.exe / chaiCmd-xxx.exe module. |
| `int gkScript::getPID()` | Process ID of the current instance. |
| `bool gkScript::isInstalled()` |
- Returns false for chaiCmd-xxx.exe - Returns true if client-xxx.exe is present in the target path
|
| `bool gkScript::isInstalledInstance()` |
- Returns false for chaiCmd-xxx.exe - Returns true if client-xxx.exe was started from install-path
|
| `bool gkScript::setupClient(bool install)` |
- Will install itself or deinstall the client - Will fail if current module is chaiCmd-xxx.exe - Will fail of isInstalledInstance() == true
|
| `string gkScript::installPath()` | Returns the installation path |
Sample install code:
```c++
def checkInstallation(serverVersion, app) {
if (gkScript.isInstalledInstance()) {
gkScript.logDebug("we are the installed client, checking auto-update ...");
if (gkScript.getVersion() != serverVersion) {
var path = gkScript.expandString("%temp%\\" + gkScript.moduleFilename());
if (app.loadClient(path)) {
gkScript.logDebug("starting new client for self-update ...");
gkScript.runProcess(path);
exit(0);
}
else {
gkScript.logError("Failed to load: " + path);
exit(2); // comment this line if the outdated version should run as fallback..
}
}
return true;
}
else {
if (gkScript.isClientInstalled()) {
gkScript.logDebug("Client is installed, nothing to do.");
exit (0);
}
else {
if ("chaicmd" == gkScript.toLower(gkScript.moduleFileName().substr(0,7))) {
gkScript.logDebug("Running chaiCmd with bootstrapper.");
return true;
}
gkScript.sleep(10000); // wait for main process to end.
if (!gkScript.installClient()) {
gkScript.logDebug("Failed to install client!");
exit(2);
}
}
}
gkScript.logDebug("starting new client for self-update ...");
gkScript.runProcess(gkScript.installPath() + gkScript.moduleFilename());
exit (0);
}
```