Page tree
Skip to end of metadata
Go to start of metadata

This page documents the incompatible changes between CDAP 4 and CDAP 5.

ChangeImpactCDAP 5 Behavior

No centralized management of dataset modules, types, jars

  • Cannot use dataset classes that are not included in the same application artifact
  • Explore service won't able to instantiate dataset instance for RecordScannable / RecordWritable implementations
    • Only affect Table based dataset, not affecting FileSet based dataset
  • Dataset classes should always be inside the application artifact jar
  • Need to provide cdap-hive jar to be added to Hive service to query Table
  • Explore integration will be off in cloud runtime
Default tx control for program will be explicit (except Flow)
  • Transactional operations need to be wrapped with Transactional.execute
  • In cloud runtime, calling Transactional.execute will result in exception
  • Would still works if runs with default profile
No custom log pipeline for programs in cloud
  • Logs collection will be done via cloud service directly, and won't go through CDAP
  • It will still works for runs using default profile
No dynamic access to artifact store
  • API will be removed / deprecated
  • New API only available for cdap system extensions
  • Only services running with default profile have access
No Stream / Flow / Service supports in cloud  

 

 

  • No labels

1 Comment

    • API changes
      • Product Impact: on the application, will require recompiling apps
    • Responsibility changes from dataset (access via libraries)
      • Product Impact: Users needs package the to redeploy the app
    • Explore
      • Asses the the impact 
      • Product requirements: Keep the UI in CDAP for explore
    • Transaction Behavior 
      • Product Impact: Default transaction control will change 
    • Upgrade 
      • Product Requirements: Error out if the apps and pipelines are not upgraded
    • Flows
      • Product Impact: Deprecate flows and streams in 5.0
    • Renaming impact
    • Extension apps
      • No Product impact
      • Technical impact: 
        • Will Run in system namespace 
        • Upgrade step to migrate the data
        • Isolation needs to be provided 
          • App will need to be update to account for different namespaces 
        • Apps that need change
          • Data Prep
          • Rules Engine
          • Tracker
          • Navigator
    • Action Items:
      • Figure out how does explore work in 5.0 and what are the trade offs
      • Figure out the impact of renaming impact packages