Page tree
Skip to end of metadata
Go to start of metadata

Introduction

ServiceNow provides service management software as a service. It specializes in IT services management, IT operations management and IT business management.

Company offerings are centered around the creation of a “service model” that corrects the root cause of service issues and enables self-service. ServiceNow's tasks, activities and processes occur as cloud services, overseen as part of a comprehensive managed workflow that supports real-time communication, collaboration and resource sharing. ServiceNow has service management offerings for IT, human resources, security, customer service, software development, facilities, field service, marketing, finance and legal enterprise needs.

Use case(s)

  • Read and replicate all of ServiceNow's data tables (e.g. Contract Management, Product Catalog, Procurement  etc.). This would allow users to extract and load all of their ServiceNow data into specified multi-table or multi-file sink.
  • Retrieve data from one of ServiceNow's data tables, allowing users to transform and enrich the data before potentially joining it with other datasets.

User Storie(s)

  • As a data pipeline developer, I should be able to import all of ServiceNow data tables so that I can analyze ServiceNow data
  • As a data pipeline developer, I should be able to specify ServiceNow table name (e.g. ast_service, proc_po, proc_po_lineitem) for which I can retrieve and transform the data
  • As a data pipeline developer, I should be able to specify Client ID and Client secret that would allow me to extract the table records from ServiceNow
  • As a data pipeline developer, I should be able to see any errors from ServiceNow REST API calls so that I can resolve those issues

Plugin Type

  • Batch Source
  • Batch Sink 
  • Real-time Source
  • Real-time Sink
  • Action
  • Post-Run Action
  • Aggregate
  • Join
  • Spark Model
  • Spark Compute

Configuration

ServiceNow batch source plugin will support two different modes:

  1. ServiceNow Reporting mode - This mode would get all the ServiceNow's data tables for a specific process/application (e.g. Contract ManagementProduct CatalogProcurement  etc.)
  2. ServiceNow Table mode - This plugin would get only the data associated with specified entity (e.g. ast_service, proc_po, proc_po_lineitem)

ServiceNow Reporting plugin

This plugin should be used when the user would like to retrieve all of the ServiceNow data tables associated with Leads and Activities entities: 

User Facing NameTypeDescriptionDefault valueNotes
Client IDstringServiceNow Client ID

Client SecretstringServiceNow Client secret

Application Namedrop-downServiceNow Application NameNoneValues: Contract ManagementProduct CatalogProcurement etc.
REST API EndpointstringREST API endpoint URL

User namestringServiceNow user name

PasswordstringServiceNow password

Start datestringStart date for reports data

End datestringEnd date for reports data

Type of Valuedrop-down

The type of values to be returned. The type can be one of two values:
`Actual` - will fetch the actual values from the ServiceNow tables,
`Display` - will fetch the display values from the ServiceNow tables.

ActualValues: Actual , Display


ServiceNow Table plugin

This plugin should be used when the user would like to extract the data associated with a single ServiceNow table (e.g. ast_service)


User Facing NameTypeDescriptionDefault valueNotes
Table NamestringServiceNow table name to fetch
Tables from Contract ManagementProduct CatalogProcurement, Financial Management etc.
Client IDstringServiceNow Client ID

Client SecretstringServiceNow Client secret

REST API EndpointstringREST API endpoint URL

User namestringServiceNow user name

PasswordstringServiceNow password

Start datestringStart date for the report

End datestringEnd date for the report

Type of Valuedrop-down

The type of values to be returned. The type can be one of two values:
`Actual` - will fetch the actual values from the ServiceNow tables,
`Display` - will fetch the display values from the ServiceNow tables.

ActualValues: Actual , Display


Design / Implementation Tips

  • None yet

Approach(s)

Properties

Security

Limitation(s)

Future Work

Test Case(s)

  • Test case #1 - TBD
  • Test case #2 - TBD

References

  • No labels

7 Comments

  1. Prateek DubleBhooshan Mogal

    As discussed offline this is just a single plugin with 2 options ( Reporting Mode / Single Table Mode) to use in a Bulk or reporting mode.  In a Reporting mode do we need to add a filed to indicate the table name field so it can be used by the Multi* sinks.  Can you modify the wiki to reflect the same. 

    1. Shivakumar Gokaram - The expectation is that the user does not need to fill in table name in the Reporting mode, however they would be able to provide the set name (e.g. Procurement, Contract Management etc.). We will then pull all the data tables associated with the specified set.

      I have also update the Wiki to reflect that it will be a single plugin with two data extraction modes.

  2. Prateek Duble - Similar to SFMC , since these plugins output different schemas depending on the tables we need an additional field to indicate which table this record belongs so the multi*sinks can process.  Let me now if i am missing something on the working of multi*sinks

    1. Shivakumar Gokaram - Can you please look at BigQuery Multi-table source? It is the similar concept here. The user specifies a whitelist and the table name gets passed in a schema field.

  3. Prateek Duble Bhooshan Mogal


    Queries 

    1. Application Name - List :  Is this a hard coded list or how do we get list of applications.  We checked around and could not find a credible API, there are some articles that talk about looking at a sys table.    If its hardcoded please provide a finite list of Applications. 
    2. Application to Table Mappings :  I saw links for list of tables in application.  Do we hard code the table names or provide as configuration files. 
    3. Modes ( Reporting and Table Mode) - Both Reporting mode and table mode use the same API's to retrieve data.  In Reporting mode we would multiple the table API's for each of the table's thats part of the application and retrieve data.   Service now provides a Reporting API but thats more analytics oriented and not to confuse with the plugin's reporting mode.  Confirming our understanding.
  4. Prateek Duble Bhooshan Mogal Mikkin Patel


    Handling schema issues. As per the Requirements the Source plugin should support the multi* option. To determine the schema of each table we query the first field and then based off it determine the schema. But in some cases the schema is different between the first field and the next one and fails. How do we handle this scenario?

    1. We worked around this issue by using a new API paramter - Type of Value ( detailed above).