DataSanity 1.2.0

Welcome to DataSanity - created and maintained by Sascha Wildgrube.

Overview

DataSanity contains Instance Scan checks that verify data quality.

Features

The DataSanity app contains Instance Scan checks that validate data records. The checks are aimed to reveal data problems, inconsistencies and invalid records.

The checks also enforce the application of the persona role concept, described in the whitepaper A mature Development and Deployment Process - How to sleep well when your 200 developers deploy apps every day.

Results should be reviewed on a regular basis and countermeasures should be taken. Bad data leads to incorrect reports, bad decisions, security issues and unnecessary work.

Disclaimer

DataSanity is NOT an officially supported ServiceNow product.

DataSanity does NOT come with any kind of warranty. If you use it, you own it!

System Requirements

Installation

  1. Create an account on GitHub - if not done already.
  2. Create a personal access token for your GitHub account.
  3. Add credentials to access GitHub - use "Basic Auth".
  4. Fork the repository https://github.com/saschawildgrube/servicenow-devtools.
  5. Go to Studio and import the DevTools application from source control.
  6. Perform all installation steps for the DevTools application documented here: https://www.wildgrube.com/servicenow-devtools/ 
  7. Fork the repository https://www.wildgrube.com/servicenow-datasanity/.
  8. Go to Studio and import the DataSanity application from source control.
  9. Execute installation scripts.
    Run the following script as a background script in scope x_snc_datasanity:
  10. x_snc_devtools.InstallApp("x_snc_datasanity");

Instance Scan checks contained in the DataSanity app

Reference

Configuration Options

Roles

Tables

This app does not add any tables.

Script Includes

  • AppGetDependencies

    Retrieves information about the app's dependencies to other apps and required versions.

  • AppGetProperty

    Gets a system property of this application.

  • AppInstall

    This script installs DataSanity.

  • AppSetDefaults

    Sets all system properties to default values.

  • AppSetProperty

    Sets a system property of this application.

  • Debug

    Produces a log output in the application log using the DevTools Debug function.

  • DevTools

    Implements the extension point for DevTools.

  • DevToolsGetApplicationStatus

    The function DevToolsGetApplicationStatus is called by the DevTools extension point instance class and returns the application status object with additional status information to be displayed on the "DevTools Application Overview" page.

  • DevToolsGetFormatConfig

    The DevToolsGetFormatConfig function is called by the DevTools extension point instance class and returns an object with field formatting information.

  • DevToolsGetInstancePipelineName

    The function DevToolsGetInstancePipelineName is called by the DevTools extension point instance class and returns the pipeline name of an instance based on the given actual name.

  • DevToolsGetLinkDirectory

    The DevToolsGetLinkDirectory function is called by the DevTools extension point instance class and returns an extended (or modified) link directory object that serves as the basis for the DevTools Link page.

  • DevToolsGetParentRecord

    The DevToolsGetParentRecord function is called by the DevTools extension point instance class and returns a parent record for the given record if it can be determined.

  • DevToolsRenderDebugDump

    The function DevToolsRenderDebugDump is called by the DevTools extension point instance class and returns additional debug information provided by the application.

  • IsActive

    Returns true if the application is active, otherwise false.

  • Log

    Produces a log output in the application log.

  • LogError

    Produces an error log output in the application log.

  • LogWarning

    Produces a warning log output in the application log.

Dependencies

Technical Debts

No technical debt has been documented for this application.

License

Copyright 2024 by Sascha Wildgrube

Licensed under the Apache License, Version 2.0 (the "License")

You may not use DataSanity except in compliance with the License.

You may obtain a copy of the License at: https://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.

Release Notes

1.2.0 - 2024-08-06

  1. DevTools 1.97.0 is now required.
  2. Added check "DataSanity - Users without name".
  3. Added check "DataSanity - Contracts without name".
  4. Added check "DataSanity - Companies with duplicate names".
  5. Check "Users must have a non-empty user name (User ID)" no longer creates findings for locked out users.
  6. Check "DataSanity - Companies without names" has been renamed to "DataSanity - Companies without name".
  7. Refactored check "DataSanity - Groups with non-persona roles" to use DevTools' IsPersonaRole() and GetRoleRecord() functions.

1.1.2 - 2024-06-06

  1. Fixed ATF test for "DataSanity - Groups with non-persona roles" - which ran into issues when executed via REST API.

1.1.1 - 2024-06-05

  1. The check "DataSanity - Invalid technical users" passed all tests but caused an evaluator error when executed as part of a full instance scan triggered by the GUI. The evaluator error indicated that the method startsWith() was not defined (although it is). Yet another example of random scripting behavior. However this has been fixed.
  2. Renamed the check "DataSanity - Scheduled Jobs run by technical users" to "DataSanity - Scheduled Jobs not run by technical users" for consistency.

1.1.0 - 2024-06-05

  1. DevTools 1.87.0 is now required.
  2. Added check "DataSanity - Scheduled Jobs run by technical users".
  3. Added check "DataSanity - Invalid technical users".
  4. The check "DataSanity - Groups with non-persona roles" now allows the admin and security_admin roles to be assigned to groups directly.

1.0.1 - 2024-05-26

  1. Check "DataSanity - Users without user name (User ID)" only considers users in sys_user (and ignores extended tables like customer_contact).
  2. Check "DataSanity - Roles assigned directly to users" now ignores the roles snc_internal and snc_external (which are introduced by the "Explicit Roles" plugin) as these roles are directly assigned to users automatically.
  3. Improved ATF test for check "DataSanity - Groups with non-persona roles" by setting the scope of the test data role.

1.0.0 - 2024-05-17

First baselined version