Page tree
Skip to end of metadata
Go to start of metadata

CDAP 3.x

CDAP 3.0 through CDAP 3.6 releases use multiple scripts to interface with different CDAP functionality.

ScriptLocationFunctionExample Use
cdap.bat$CDAP_HOME/bin (SDK)Manage CDAP Standalone on Windowscdap.bat start
cdap.sh$CDAP_HOME/bin (SDK)Manage CDAP Standalone on MacOS/Linuxcdap.sh start
cdap-cli.bat$CDAP_HOME/bin (SDK)Execute the CDAP CLI on Windowscdap-cli.bat list artifacts
cdap-cli.sh

$CDAP_HOME/bin (SDK)

$CDAP_HOME/cli/bin (Distributed)

Execute the CDAP CLI on MacOS/Linuxcdap-cli.sh list artifacts
common.sh$CDAP_HOME/*/bin (Distributed)Functions for distributed CDAPsource $CDAP_HOME/master/common.sh
common-env.sh$CDAP_HOME/*/bin (Distributed)Default environment variables for distributed CDAPsource $CDAP_HOME/master/common-env.sh
service$CDAP_HOME/*/bin (Distributed)Distributed CDAP start scriptn/a (script cannot be directly executed)
config-tool$CDAP_HOME/ui/bin (Distributed)Convert CDAP XML configuration files to JSONconfig-tool --cdap
auth-server-env.sh$CDAP_HOME/security/conf (Distributed)Environment variables for CDAP Auth Serversource $CDAP_HOME/security/conf/auth-server-env.sh
kafka-server-env.sh$CDAP_HOME/kafka/conf (Distributed)Environment variables for CDAP Kafka Serversource $CDAP_HOME/kafka/conf/kafka-server-env.sh
master-env.sh$CDAP_HOME/master/conf (Distributed)Environment variables for CDAP Mastersource $CDAP_HOME/master/conf/master-env.sh
router-env.sh$CDAP_HOME/gateway/conf (Distributed)Environment variables for CDAP Routersource $CDAP_HOME/gateway/conf/router-env.sh
ui-env.sh$CDAP_HOME/ui/conf (Distributed)Environment variables for CDAP UIsource $CDAP_HOME/ui/conf/ui-env.sh
svc-auth-server$CDAP_HOME/security/bin (Distributed)Symlink to manage CDAP Auth Server$CDAP_HOME/security/bin/svc-auth-server start
svc-kafka-server$CDAP_HOME/kafka/bin (Distributed)Symlink to manage CDAP Kafka Server$CDAP_HOME/kafka/bin/svc-kafka-server start
svc-master$CDAP_HOME/master/bin (Distributed)Symlink to manage CDAP Master$CDAP_HOME/master/bin/svc-master run \ co.cask.cdap.data.tools.UpgradeTool upgrade
svc-router$CDAP_HOME/gateway/bin (Distribtued)Symlink to manage CDAP Router$CDAP_HOME/gateway/bin/svc-router start
svc-ui$CDAP_HOME/ui/bin (Distributed)Symlink to manage CDAP UI$CDAP_HOME/ui/bin/svc-ui start
tx-debugger.bat$CDAP_HOME/bin (SDK)Execute Transaction Debugger on Windowstx-debugger.bat
tx-debugger.sh$CDAP_HOME/bin (SDK)Execute Transaction Debugger on MacOS/Linuxtx-debugger.sh

Functionality is spread across the file-system and it is difficult to know which script to use for which functionality. The service script, while not run directly, changes its functionality based on the way it is run, rather than arguments. This makes the script rather fragile and difficult to modify without knowing how it will possibly effect other services. Each distributed component also has a copy of the common.sh and common-env.sh files, making them identical, but not common across the components.

CDAP 4.x

CDAP 4.0 introduces a single script which unifies and simplifies the interface with CDAP. Rather than multiple scripts, the cdap script should support all of the necessary functionality for running CDAP in either distributed mode or standalone mode within the CDAP Sandbox. This reduces the maintenance cost of these scripts while providing a single location for end users to perform CDAP actions.

"sdk" was renamed to "sandbox" in CDAP 4.2. Existing "cdap sdk" commands still work, but give a deprecation notice to use "cdap sandbox" instead.


The entirety of the scripts above have been rolled into a small set of scripts.

ScriptLocationFunctionExample Use
cdap.bat$CDAP_HOME/bin (Sandbox)Interface with CDAP on Windowscdap.bat sandbox start
cdap

$CDAP_HOME/bin (Sandbox),

$CDAP_HOME/*/bin (Distributed)

Interface with CDAP on MacOS/Linuxcdap sandbox start
functions.sh

$CDAP_HOME/bin (Sandbox),

$CDAP_HOME/*/bin (Distributed)

Provide shell functions for MacOS/Linux (Sandbox) or Linux (Distributed)source $CDAP_HOME/bin/functions.sh

CDAP does not provide a set of usable functions on Windows due to the limitations of the BAT file format.

Using the cdap script

The cdap script is merely a wrapper which takes certain arguments and passes them to the correct functions in the function.sh shell function library. Any commands which are not recognized by the script are passed to the CDAP CLI and executed. The table below shows some common uses which will mirror the previous functionality. However, it is not intended to be exhaustive and will not include every command or option.

CommandContextOld CommandNotes
cdap sandbox start --foregroundSandboxcdap.sh start --foregroundCDAP Sandbox command arguments remain unchanged
cdap sandbox --helpSandboxcdap.sh --help
cdap cli list artifactsDistributed, Sandboxcdap-cli.sh list artifacts
cdap auth-server startDistributed$CDAP_HOME/security/bin/svc-auth-server start
cdap classpathDistributed$CDAP_HOME/master/bin/svc-master classpathAlways returns the CDAP Master CLASSPATH
cdap upgradeDistributed

$CDAP_HOME/master/bin/svc-master run co.cask.cdap.data.tools.UpgradeTool upgrade

Using run from a service script other than svc-master would result in an invalid CLASSPATH
cdap upgrade hbaseDistributed

$CDAP_HOME/master/bin/svc-master run co.cask.cdap.data.tools.UpgradeTool upgrade_hbase


cdap debug transactionsDistributed, Sandbox$CDAP_HOME/bin/tx-debugger.sh (SDK only)

Previously, the script was only available on Sandbox

WARNING: Token file must exist or provided on command line

*cdap versionDistributed, Sandbox
Show cdap_version output
*cdap statusDistributed, Sandbox
Show status of CDAP processes

*cdap start

*cdap stop

Distributed, Sandbox

Distributed shows usage "cdap $service start"

Sandbox starts or stops

*cdap support-bundleDistributed, Sandbox
Logs, jmap, jstack, environment vars + below
*cdap infoDistributed, Sandbox

Display info about CDAP: versions, has internet (curl cask.co), data directory structure (Sandbox only), CLASSPATH, JAR checksums for CDAP, JDK

Items denoted with an asterisk * are to be completed

Shell function library

All CDAP functions are provided in a library shell file, which can be sourced to easily interact with CDAP from the command line, especially useful for integrating CDAP into new deployment or management systems. Below is a list of functions which may be useful to integrators.

FunctionInputOutputDescription
cdap_home
location of CDAPReturns the location of CDAP Home to STDOUT
cdap_get_confproperty name, file to read, default response (optional)value of property or default responseReturns 1 if property isn't found and no default is set
cdap_kinitCDAP_PRINCIPAL (optional), CDAP_KEYTAB (optional)
Initializes Kerberos ticket using variables or cdap-site.xml configuration
cdap_set_javaJAVA_HOME (optional)exports JAVASets location to java binary in JAVA variable, uses JAVA_HOME, if set
cdap_serviceservice, action, arguments (optional)
Executes the given service with the correct environment
cdap_start_java

Executes a CDAP Java service with the given environment
cdap_run_classclass name, arguments (optional)
Executes a non-service Java class with arguments using the CDAP Master environment
cdap_contextCDAP_HOMEcontextReturns the CDAP_HOME context (sdk or distributed) to STDOUT
cdap_versionCDAP_HOME, component (optional)versionReturns the CDAP_HOME version to STDOUT


CDAP 5.x

CDAP 5.x retains the CDAP 4.x scripts.  The only significant change is the removal of the "sdk" commands, which were deprecated in 4.2.  The reference tables above have been updated to reflect this.


  • No labels