Deployer 1.16.0

Helpful links on CICD

• ServiceNow Documentation - Production deployment tips

Reference

Configuration Options

• x_snc_deployer.active

  Controls if the Deployer application and its REST APIs are active.

• x_snc_deployer.logging.verbosity

  Controls the verbosity of log output produced by the Deployer application.

• x_snc_deployer.required_target_version

  Specificies the minimum required version of the Deployer application on the target instance to perform remote operations on the target instance.

Roles

• x_snc_deployer.admin

  A Deployer admin user has access to all configuration and process assets of the Deployer application. The role contains the deployer manager role.

• x_snc_deployer.api

  The api user is entitled to execute REST API calls against the REST APIs exposed by the Deployer app. The admin and security_admin roles are required since the execution of ATF tests depends on the roles of the user triggering their execution.

• x_snc_deployer.developer

  A developer can view all deployer related data and can create new Repsitory Application Status records.

• x_snc_deployer.manager

  A Deployer manager user can trigger deployments and has read access to all configuration assets and the source control based dependency tree.

• x_snc_deployer.viewer

  A Deployer viewer user has read access to the Deployer configuration assets and has read access to the source control-derived dependency tree of a given application.

REST APIs

Deployer Target API

The API provides endpoints to be used while installing applications.

• Get Status

  Retrieve status information on all applications installed on the instance.
  Method: GET
  URI: /api/x_snc_deployer/target/get_status
  

• Run Installation Scripts

  Executes the AppInstall() functions of the given app and its dependencies.
  Method: GET
  URI: /api/x_snc_deployer/target/run_install_scripts
  

Tables

• Deployment (x_snc_deployer_deployment)

  The table is derived from table Task (task).
  

• Deployment Task (x_snc_deployer_deployment_task)

  The table is derived from table Task (task).
  

• Instance (x_snc_deployer_instance)

  The table is not derived from any other table.

• Pipeline (x_snc_deployer_pipeline)

  The table is not derived from any other table.

• Pipelines to Instances (x_snc_deployer_m2m_pipline_instance)

  The table is not derived from any other table.

• Repository Application Status (x_snc_deployer_repository_application_status)

  The table is not derived from any other table.

• Repository Credentials (x_snc_deployer_repository_credentials)

  The table is not derived from any other table.

UI Actions

• Cancel (x_snc_deployer_deployment)

  Cancels the deployment.

• Cancel Task (x_snc_deployer_deployment_task)

  Cancels the deployment task.

• Create Deployment (x_snc_deployer_instance)

  Creates a new deployment based on the current Repository Application Status.

• Create Deployment (x_snc_deployer_repository_application_status)

  Creates a new deployment based on the current Repository Application Status.

• Get New Status For Same Version (x_snc_deployer_repository_application_status)

  Gathers the status of the current application version again by creating a new status record.

• Get Status For New Version (x_snc_deployer_repository_application_status)

  Checks the status of the current application for a new version by creating a new status record.

• Mark as Complete (x_snc_deployer_deployment)

  Marks the deployment as failed.

• Mark as Failed (x_snc_deployer_deployment)

  Marks the deployment as failed.

• Mark Task as Complete (x_snc_deployer_deployment_task)

  Marks a manual task as complete

• Mark Task as Failed (x_snc_deployer_deployment_task)

  Marks a manual task as failed

• Prepare (x_snc_deployer_deployment)

  Prepares the deployment tasks.

• Retry (x_snc_deployer_deployment)

  Creates a new deployment based on the chosen parameters of the failed or cancelled deployment.

• Retry Task Execution (x_snc_deployer_deployment_task)

  Attempts to retry the execution of the task.

• Show Task List (x_snc_deployer_deployment)

  Displays a wiki-formatted task list for the deployment.

• Skip Task (x_snc_deployer_deployment_task)

  Skips the deployment task and continues with the next task.

• Start (x_snc_deployer_deployment)

  Starts the deployment.

• Update Local CMDB (x_snc_deployer_instance)

  Loads the installed applications from the target instance and updates the local CMDB.

• Validate (x_snc_deployer_instance)

  Validates if the instance is configured correctly and if it is up and running.

• View (x_snc_deployer_repository_application_status)

  Displays the dependency tree.

• View Status (x_snc_deployer_pipeline)

  Displays the pipeline status.

Script Includes

• AppGetDependencies

  Retrieves information about the app's dependencies to other apps and required versions.

• AppGetProperty

  Gets a system property of this application.

• AppInstall

  This script installs Deployer.

• AppSetDefaults

  Sets all system properties to default values.

• AppSetProperty

  Sets a system property of this application.

• CleanUpDependencyObject

  Removes all fields from a dependency record object that are not required for preparing and executing deployments.

• CleanUpPackageObject

  Removes all fields from a package record object that are not required for preparing and executing deployments.

• CleanUpSysDBObjectObject

  Removes all fields from a syS_db_object record object that are not required for preparing and executing deployments.

• CleanUpSysDictionaryObject

  Removes all fields from a sys_dictionary record object that are not required for preparing and executing deployments.

• Debug

  Produces a log output in the application log using the DevTools Debug function.

• DeployerClientAPI

  Provides functions to be called from UI scripts.

• DevTools

  Implements the extension point for DevTools.
  |
  Methods:
  * initialize()
  * GetParentRecord()
  * GetLinkDirectory()
  * GetFormatConfig()
  * GetApplicationStatus()
  * RenderDebugDump()
  * GetInstancePipelineName()
  * AppVersionBaselineOnFinish()

• DevToolsGetApplicationStatus

  The function DevToolsGetApplicationStatus is called by the DevTools extension point instance class and returns the application status object with additional status information to be displayed on the "DevTools Application Overview" page.

• DevToolsGetFormatConfig

  The DevToolsGetFormatConfig function is called by the DevTools extension point instance class and returns an object with field formatting information.

• DevToolsGetInstancePipelineName

  The function DevToolsGetInstancePipelineName is called by the DevTools extension point instance class and returns the pipeline name of an instance based on the given actual name.

• DevToolsGetLinkDirectory

  The DevToolsGetLinkDirectory function is called by the DevTools extension point instance class and returns an extended (or modified) link directory object that serves as the basis for the DevTools Link page.

• DevToolsGetParentRecord

  The DevToolsGetParentRecord function is called by the DevTools extension point instance class and returns a parent record for the given record if it can be determined.

• DevToolsRenderDebugDump

  The function DevToolsRenderDebugDump is called by the DevTools extension point instance class and returns additional debug information provided by the application.

• GetApplicationListFromInstance

  Returns the array of applications from an instance.
  This is a deprecated proxy of the function TargetInstanceGetApplicationList().

• GetApplicationStatus

  Retrieves an array of objects representing the applications available on the target instance.

• GetCredentialForRepository

  Retrieves a credential record for the given repository based on the configured Repository Credential records. If no match is found or no valid repository url is provided, the function returns false.

• GetDeploymentTasks

  Returns the status object including the required deployment tasks based on the given status and instance objects.

• GetLatestRepoDependencyList

  Returns the latest available dependency list for the given repository and version or false if no status is available.

• GetLatestRepoDependencyStatus

  Returns an object containing the latest available dependency list and tree for the given repository and version or false if no status is available.

• GetLatestRepoDependencyTree

  Returns the latest available dependency tree for the given repository and version or false if no status is available.

• GetLinkDirectoryCICD

  Returns a link directory with useful links regarding CICD topics in ServiceNow.

• GetPipelineStatus

  Retrieves the status information for a given pipeline.

• GetRepoDependencyList

  Returns an array of objects representing the packages representing the dependency list based on the given repository url and the desired version.

• GetRepoDependencyTree

  Returns the dependency tree based on the given repository url and the desired version.

• GetRepositoryCredentialRecord

  Retrieves the repository credential record for the given repository based on the given path or Sys Id. If no match is found or no valid repository url is provided, the function returns false.

• GetRepositoryType

  Returns the type identifier for a given repository url. The function returns false of the type could not be determined.

• IdentifyDataDeletion

  Identifies the tables (sys_db_object) and columns (sys_dictionary) which are going to be deleted based on the current list of applications on an instance and the to-be-installed list of applications.

• IsActive

  Returns true if the application is active, otherwise false.

• IsBranchDeployable

  Returns if the given branch name indicates whether the branch is deployable. The branch must either be named "master" or contain a semantic version.

• Log

  Produces a log output in the application log.

• LogError

  Produces an error log output in the application log.

• LogWarning

  Produces a warning log output in the application log.

• SourceControlAPI

  The class acts as an interface to source control repositories in GitHub or GitLab.
  |
  Methods:
  * initialize()
  * GetRequestCount()
  * Error()
  * ParseRepoUrl()
  * GetBranchesFromRepo()
  * GetFilesFromRepo()
  * GetFileContent()
  * GetPackageRecordFromRepo()
  * GetDependencyRecordsFromRepo()
  * GetSysDBObjectRecordsFromRepo()
  * GetSysDictionaryRecordsFromRepo()
  * ProtectBranchReadOnly()

• TargetInstanceATFTestSuiteRun

  Runs the ATF test suite on the given target instance. Returns the progress id of the running suite if successful, or false in case of error.

• TargetInstanceCICDCheckProgress

  Checks the progress of a running ATF test suite on the given target instance.

• TargetInstanceGetApplicationList

  Returns the array of applications from the given target instance.

• TargetInstanceGetStatus

  Returns the array of applications from the given target instance.

• TargetInstanceInstanceScanRunSuiteOnApp

  Runs the ATF test suite on the given target instance. Returns the progress id of the running suite if successful, or false in case of error.

• TargetInstanceRequest

  Requests the given end point of the Deployer Target API on the given target instance.
  The credentials used for the request are taken from the credentials associated with the target instance.

• TargetInstanceRunInstallScripts

  Runs install scripts on the given target instance for the given application.

• TargetInstanceSourceControlApplyChanges

  Triggers the action "Apply Remote Changes" for the given application for the current branch on the target instance.

• TargetInstanceSourceControlImport

  Imports an application from source control on the target instance.

• TargetInstanceUpdateLocalCmdb

  Updates the local CMDB based on data gathered from the target instance.

• TargetInstanceValidate

  Returns true if the given instance is correctly configured and accessible.

• TriggerNewRepositoyApplicationStatus

  Triggers the background process to create a new repository application status based on a given repository url and version.

• ValidateInstance

  Deprecated proxy for the new TargetInstanceValidate() function for backward compatibility.

Dependencies

• DevTools

  Compatible Versions: 1.103.0
  Repository url: https://github.com/saschawildgrube/servicenow-devtools.git
  Uses numerous functions and classes from DevTools - the GitHubAPI and GitLabAPI classes are of specific importance. Version 1.94.0 is required due the new functions to update the CMDB. Version 1.96. is required due to new functions in DocumentationAPI. 1.99.0 is required due to MID Server support. 1.103.0 is required for branch protection support.
  

Technical Debts

• Function ValidateInstance() is deprecated - replaced by TargetInstanceValidate()

  The function ValidateInstance() is depreacted. The function TargetInstancevalidate() should be used instead. The new function follows a naming convention to group all functions that communicate with the Deployer Target API.
  

License

Copyright 2023-2024 by Sascha Wildgrube

Licensed under the Apache License, Version 2.0 (the "License")

You may not use Deployer except in compliance with the License.

You may obtain a copy of the License at: https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Release Notes

1.16.0 - 2024-11-25

1. The local CMDB is now updated immediately after an application has been installed during a deployment.
2. When identifying the version to be deployed Deployer will now prefer the oldest compatible version available in a repsitory.
3. Added function SortBranches().
4. Reduced flakyness of test "Deployer - GetLatestRepoDependencyTree".

1.15.0 - 2024-10-22

1. DevTools 1.103.0 is required for branch protection support.
2. Deployer now protects new version branches during application version baselining.

1.14.0 - 2024-10-17

1. Vancouver is no longer actively supported.
2. Deployer is fully tested with Xanadu.
3. A MID Server can now be used to connect to a target instance.
4. Failing requests sent to a target instance are now logged as errors.
5. Added menu item "Technical Users".
6. When preparing a deployment GetDeploymentTask() erroneously created manual import tasks for non-Custom Application dependencies (e.g. Store Apps). This has been fixed.
7. When a task is cancelled in-flight, the remote operation will still finish but the execution of the next pending task is now suppressed.

1.13.0 - 2024-08-26

1. DevTools 1.99.0 is required due to MID Server support in GitHubAPI, GitLabAPI and HttpRequest().
2. Deployer is now fully tested with Xanadu.
3. A MID Server can now be specified for a Repository Credential.
4. Added function GetRepositoryCredentialRecord().
5. Repository Credentials now have a proper display value.
6. Improved ATF test "Deployer - Deployer Target API - Run Install Scripts".
7. Changed term "CICD Instance" to "Controller Instance".

1.12.1 - 2024-07-05

1. Packages other than Custom Applications (sys_app) were erroneously treated as apps that can be installed from a source control repository when preparing a deployment. This has been fixed. Such packages are now ignored when creating the tasks.
2. In the Repository Application Status view, an error message was shown if at least one package is contained that is out of scope. This has been fixed.

1.12.0 - 2024-07-05

1. DevTools 1.94.0 is required due to the new functions to update the CMDB.
2. Deployer 1.12.0 requires the same version or higher on the Target Instance due to the change in the x_snc_deployer.api role and the new capabilities of the "Get Status" REST API endpoint.
3. The local CMDB is updated to reflect the installed applications on the Target Instance after a deployment.
4. The local CMDB is updated to reflect the availability of installable applications after creating a Repository Application Status.
5. Added the UI Action "Update Local CMDB" on the Target Instance form.
6. The x_snc_deployer.api role now also contains the security_admin role - The admin and security_admin roles are required since the execution of ATF tests depends on the roles of the user triggering their execution.
7. The "Get Status" REST API endpoint now also provides the list of dependencies and the current branch for each application.
8. Under some circumstances in the Repository Application Status Dependency List view the error banner was not displayed. This has been fixed.
9. If an instance url is specified without trailing slash, the Studio url in the manual task for importing a new application from source control was incorrect. This has been fixed.
10. Repository App Status lists are now sorted by "Updated On" by default.

1.11.0 - 2024-06-06

1. Utah is no longer actively supported.
2. Deployer has been fully tested with Vancouver and Washington.
3. DevTools 1.86.0 is now required because GitLabAPI and GitHubAPI can now handle repositories with more than 100 branches.
4. A Repository Application Status can now be generated for all branches starting by "dev".
5. Renamed UI Action "Refresh" to "Get New Status For Same Version".
6. The UI Action "Get New Status For Same Version" is no longer visible when the Repository Application Status is not yet finished.
7. Renamed UI Action "New Version" to "Get Status For New Version".
8. The UI Action "Get Status For New Version" now proposes the next minor version based on the previous version.
9. When generating a Repository Application Status dependencies to plugins and store apps are now correctly identified as out-of-scope and displayed as such.
10. When generating a Repository Application Status a more precise error is reported if no corresponding repository credential could be identified.
11. When generating a Repository Application Status a dependency without a repository url is now correctly identified as an error and the status is not ready for deployment.
12. Added warning about cascade deletes in case a table is about to be deleted during deployment.
13. Added detailed instructions and a url to start with for the manual task of importing an application from source control for the first time.
14. When updating a repository credential record, GitLab repository credentials are deactivated if no API Url has been specified.
15. Refactoring in ExecuteDeploymentTask() to use more DevTools functions.

1.10.0 - 2024-02-29

1. DevTools 1.82.0 is now required.
2. Deployer 1.10.0 requires the same version or higher on the target instance due to updated identification of data deletion.
3. The detection of data deletion now considers "cloned" dictionary records on TPP tables (like tables derived from cmdb_ci).
4. The Deployment form no longer reloads if the "Close Notes" have been edited.
5. The Deployment Task form now reloads when the status changes to update the UI Action buttons.
6. The Deployer - Repository Viewer page was not displayed properly when using Polaris dark theme. This has been fixed.
7. The display value of a "Repository Application Status" now indicates that the date/time is in UTC.
8. Modified ATF test for GetLatestRepoDependencyList() to avoid test flakyness.

1.9.1 - 2024-02-15

1. Fixed a bug related to generating the result urls in deployment tasks to navigate to test and scan results on the target instance.
2. In the Pipeline status view page, "null" was displayed if a platform app scope is configured for a target instance, but the app is not installed on the target instance. This has been fixed.

1.9.0 - 2024-02-14

1. A data deletion policy can be specified for each target instance and is enforced upon deployment.
2. Introducing the concept of Pipelines.
3. Deployer 1.9.0 requires the same version or higher on the target instance due to the newly introduced identification of data deletion.
4. DevTools 1.81.0 is now required because of GetAppDefaultTestSuite().
5. A repository application status can now be established for the current state in a dev branch.
6. Deployment Tasks to run CodeSanity checks and ATF test suites now contain a link to the results on the target instance.
7. The checkbox "Enforce Downgrade" on the Deployment form was not read-only by default. This has been fixed.
8. The "Prepare" button is now hidden if no instance has been selected yet for a deployment.
9. The "Run Installation Scripts" REST API endpoint no longer requires the "async" parameter - to ensure full backward compatibility.
10. The Deployment and the Repository Application Status forms now reload when there is a status change.
11. The actual start and end date and time is recorded for every deployment.
12. The "Show Task List" UI Action on the Deployment form is now only visible in draft state.
13. Deployments can now be cancelled while in ready state.

1.8.0 - 2024-01-16

1. Deployer is fully tested in Vancouver.
2. Tokyo is no longer actively supported.
3. DevTools 1.79.0 is now required because of StringStripTrailingSlash() and DevToolsWorkerAPI::InstallApp().
4. Deployer 1.8.0 now requires the same version or higher on the target instance due to the asynchronous execution of installation scripts.
5. Deployer can now properly handle a repository url path with a trailing slash.
6. The "Prepare" UI Action is no longer available for new deployments. Some options are instance specific and can only be selected once the deployment has been saved with a selected target instance.
7. Added table x_snc_deployer_deployment_task to the link directory.
8. When deleting a Deployment, associated Deployment Tasks are now deleted, too.
9. Improved feature description and installation instructions in the manual.

1.7.0 - 2023-11-17

1. Added more supported automatic deployment tasks: Installing applications from source control via switch branch/apply remote changes.
2. Now supporting instance specific manual pre and post deployment activities.

1.6.0 - 2023-11-10

1. Deployer now requires Deployer 1.6.0 or higher on the target instance because the technical user requires additional roles and the "Get Status" endpoint delivers additional information about the applications - including the Sys ID of the default ATF Test Suite.
2. Deployer now requires DevTools 1.75.1 because HttpRequest() now handles exceptions thrown by RESTMessageV2::execute(). This allows for proper error handling when communicating with the target instance.
3. Introducing deployment tasks which represent the individual steps of a deployment - supporting both manual and automated activities.
4. Supported automatic deployment tasks: Running installation scripts, running CodeSanity checks, and executing the default ATF Test Suite.
5. A (manual) smoke test procedure can be specified per instance.
6. Introduced the x_snc_deployer.developer role to allow developers to create a new Repository Application Status to check if their baselined applications are ready for deployment.
7. The user can specify if the deployment should be automated (where possible) or be conducted manually.
8. A deployment cannot start while another deployment on the same target instance is in progress.
9. Added related list of deployments to "Repo App Status" and "Target Instance" forms.
10. Added UI action "Create Deployment" both on the "Repository Application Status" and the "Target Instance" forms.
11. The x_snc_deployer.api role now contains the sn_cicd.sys_ci_automation role which is required to operate the sn_cicd API endpoints.
12. The x_snc_deployer.api role now contains the admin role which - due to a platform deficiency - is required to run ATF tests remotely.

1.5.0 - 2023-10-30

1. DevTools 1.74.0 is now required due to a bugfix in GetDependencyListFromTree().
2. Added state model to deployments to represents the status properly.
3. Added UI action "New Version" to create a new Repo App Status for a new version.
4. Users with the x_snc_deployer.viewer role now have read access to the basic_auth_credentials table.
5. Users with the x_snc_deployer.manager role now also inherit the credential_admin role and hence have write access to the discovery_credentials table.
6. Users with the x_snc_deployer.manager role can now edit all fields on the deployment record.
7. The related link "Show task List" now only shows after the deployment has been created.
8. If logging.verbosity is set to "debug" HtmlRenderRepoDependencies() displays the raw data of the repository status.
9. Improved status display in HtmlRenderRepoDependencies().
10. Added "Credentials" menu item.
11. The system property "x_snc_deployer.required_target_version" now defines the minimum version of the Deployer app that must be installed on the target instance.
12. Menu items "Log" and "Test Suite" are only visible for x_snc_deployer.admin users.

1.4.0 - 2023-09-22

1. A deployment can now be configured to either enforce downgrades - to exactly match the versions contained in the repositories - or to leave more current versions on the target instance.
2. A deployment can now be configured to refresh application versions that are already installed on the target instance (or not).
3. A deployment can now be configured to skip the execution of CodeSanity checks on the target instance.
4. A deployment can now be configured to skip the execution of the default ATF Test Suites of applications on the target instance.
5. Deployer Target API / Get Status now also returns the application status.
6. Added function GetApplicationListFromInstance() to retrieve an array of applications and customization applications from a target instance.

1.3.0 - 2023-09-18

1. Introduced the "Deployment" table to keep track of deployments.
2. Added the "Show Task List" UI action to show a wiki-formatted task list for the deployment.
3. Introduced the "Instance" table to keep configuration and credential references for target instances.
4. The "Validate" UI action can be used to verify if an instance is properly configured and if the credentials enable the execution of API calls on the instance.
5. Added function GetDeploymentTasks() to determine the required deployment tasks based on a given application deployment status and a target instance.
6. GetLatestRepoDependencyTree() now contains the field "repository_application_status" which contains the Sys ID of the status record from where the information was gathered.

1.2.0 - 2023-07-31

1. Added function GetLatestRepoDependencyStatus().
2. Added function CleanUpPackageObject() to avoid not necessary fields in the objects returned by functions determining the application repository status.
3. Avoiding flakyness for ATF tests for GetLatestRepoDependencyList() and GetLatestRepoDependencyTree().
4. Improved feature and installation documentation in manual.

1.1.0 - 2023-07-25

1. Deployer is now fully tested in Utah.
2. San Diego is no longer actively supported.
3. DevTools 1.65.0 is now required because of the contained GitLabAPI and GitHubAPI classes and the new GetDependencyListFromTree() function.
4. Deployer now supports the analysis of the deployment readyness of an application version stored in a source code repository.
5. The exposed functions allow pipeline automation apps to use the repository application status information as input.
6. Added x_snc_deployer.viewer, x_snc_deployer.manager and x_snc_deployer.admin roles.
7. The Deployer menu is now accessible to users with the x_snc_deployer.viewer role.
8. Removed the menu item "Repository Viewer (ad-hoc)".
9. Added function GetCredentialForRepository().
10. Added function GetRepositoryType().
11. Added function SourceControlAPI::GetRequestCount().
12. Added function IsBranchDeployable().
13. Added function GetLatestRepoDependencyTree().
14. Added function GetLatestRepoDependencyList().

1.0.0 - 2023-02-24

First baselined version