Page tree
Skip to end of metadata
Go to start of metadata

You are viewing an old version of this page. View the current version.

Compare with Current View Page History

« Previous Version 29 Next »


Checklist

  • User Stories Documented
  • User Stories Reviewed
  • Design Reviewed
  • APIs reviewed
  • Release priorities assigned
  • Test cases reviewed
  • Blog post

Introduction 

CDAP Data pipeline and Wrangler 

Goals

There are four goals which needs to be achieved to improve error handling:

  • Add a framework to centralize error messages in wrangler and pipeline

  • Add a framework to add error codes to error messages so that developers can figure out the source of error message without looking at stacktrace

User Stories

  1. As a CDAP pipeline developer, if a pipeline contains plugin configurations which are invalid, I will like it to fail early with appropriate error message.

  2. As an ETL engineer, if I run into error situation while applying directives, I will like to see appropriate error message which clearly indicates the error.

Scenarios

  • Scenario 1: Error codes in Wrangler

    • Scenario 1.1

Alice wants to wrangle data using CDAP's Wrangler tool. As part of that, Alice wants to connect to Database Source using Wrangler Connection. While attempting to do that, Alice is seeing a cryptic error message while testing the connection that she does not know how to resolve just by looking at the error message. In order figure out cause of the issue and recommended action, Alice will like to browse error code catalog with the error code displayed along with the error message.

    • Scenario 1.2

Alice is applying transformations to the data on the fly using CDAP's Wrangler tool. However, while applying transformations, Alice sees an error message that does not suggest recommended action to fix the issue. She wants to browse the error code catalog to figure out recommended action to resolve the issue.

  • Scenario 2: Centralized Error messages in Wrangler

    • Scenario 2.1

Alice wants to wrangle data using CDAP's Wrangler tool. While applying transformations to the connected source data on the fly, Alice tries to parse boolean column as csv. However, Alice observes that built-in directives to parse boolean data as csv and avro returns different error messages. Alice would like to see standard error message from both the directives when column type is incompatible.

  • Scenario 3: Error codes in Pipeline

    • Scenario 3.1

As a plugin developer, Bob has deployed a plugin to CDAP. Alice is a pipeline developer who is building a pipeline using CDAP data pipeline studio. While building the pipeline,  built data pipeline using GUI of CDAP pipeline studio. After deploying the data pipeline, the pipeline failed with NPE without any meaningful error message. To debug this issue further, Alice wants to share error code, rather than whole stack trace, with Bob to figure out which method caused NPE exception.

  • Scenario 4: Centralized Error messages in Pipeline

    • Scenario 4.1

Alice who is a data pipeline developer, develops data pipelines using CDAP. While building data pipelines, Alice observes that the error messages for common error types such as invalid configuration error are not consistent. Alice would like to see consistent error messages for common errors.

Design


Approach


Impact on UI


Test Scenarios

Test ID

Test Description

Expected Results













Bug Fixes 

Releases

Release 6.1.0

Related Work

Future work

  • No labels