From time to time, applications have need to access to small piece of sensitive data, such as pass-phases, cryptographic keys or access tokens. Those information should be stored and managed by CDAP in a secure way. At runtime, authorized applications should be able to have access to those information.
New API will be introduce to CDAP to allow applications accessing sensitive data in a secure way.
New REST APIs will be provide to mirror the
delete capability as exposed through the
SecureStoreManager interfaces as shown above.
The REST API can only runs on
HTTPS and only authorized user can access them. Permissions will also be enforced based on the following roles:
MANAGE- Can perform
READ_ONLY- Can perform
ALL- Can perform all operations
Permission is enforced at per key level. For example, a user can only list or get keys that he has access to.
SecureStoreManager will be implemented using the standard JKS or JCEKS keystore to store the sensitive data. The keystore can be protected with a key in the CDAP master keystore, which CDAP already requires the user to provide in order to have SSL enabled. Since program will be executed in the same JVM as the SDK process, accessing to the sensitive data directly through the proper Guice binding that binds the
SecureStore interface to the actual implementation.
Hadoop with KMS
On Hadoop cluster with KMS enabled, CDAP can provide an implementation of
SecureStoreManager through the Hadoop
KeyProvider API, assuming the cluster is configured to use
KMS implementation of
The CDAP master process will host the HTTPS server for providing the REST API support. The master process is also responsible for acquiring and refreshing the KMS delegation token when launch programs to be run on YARN.
Implementation of the
SecureStore used by the program at runtime can be implemented by adapting calls to the underlying Hadoop
Hadoop without KMS
When KMS is not available, CDAP can provide an implementation of
SecureStore with an architecture similar to
Integration with Hydrator
The Hydrator UI can use the REST API to get list of names and provides a dropbox/auto-complete box for the user to pick which to use when configuring the plugin.
Plugin can access to the secure data store through the
SecureStore API exposed through the context object. Plugin will get the name through the configuration and gets the actual sensitive data at runtime.