Page tree
Skip to end of metadata
Go to start of metadata

Checklist

  • User Stories Documented
  • User Stories Reviewed
  • Design Reviewed
  • APIs reviewed
  • Release priorities assigned
  • Test cases reviewed
  • Blog post

Introduction 

After adding support for Authorization in CDAP, we are authorizing operations on different cdap entities through AuthorizationEnforcer in the following way:

Namespace.create()
public synchronized void create(final NamespaceMeta metadata) throws Exception {
  // Namespace can be created. Check if the user is authorized now.
  Principal principal = authenticationContext.getPrincipal();
  authorizationEnforcer.enforce(instanceId, principal, Action.ADMIN);
  ....
  ...
}

 

The AuthorizationEnforcer is also used to create filters for entities which are accessible to a user. This is typically used in list operations:

Namesapce.list()
public List<NamespaceMeta> list() throws Exception {
  List<NamespaceMeta> namespaces = nsStore.list();
  Principal principal = authenticationContext.getPrincipal();
  final Predicate<EntityId> filter = authorizationEnforcer.createFilter(principal);
  return Lists.newArrayList(
    Iterables.filter(namespaces, new com.google.common.base.Predicate<NamespaceMeta>() {
      @Override
      public boolean apply(NamespaceMeta namespaceMeta) {
        return filter.apply(namespaceMeta.getNamespaceId());
      }
    })
  );
}

 

This process is very cumbersome because of the following reasons:

  1. A developer will first need to make sure he has all the needed modules injected to use AuthorizationEnforcer and AuthenticationContext. 
  2. Create and initialize an instance of AuthorizationEnforcer and AuthenticationContext
  3. Use them at various places in their code to enforce Authorization.

Moreover, we require authorization enforcement all over cdap code base which requires the above to be done at a lot of places.

Also, in coming releases, we want to support enabling/disabling operations depending on user privileges. For example, if a user does not have "ADMIN" privilege on a program, the "START" button in the UI should be disabled for that user. A similar feature is needed in the CLI too, where the auto complete should only show options for which the user has privileges. Although this requirement is out of scope for 4.0 we would like to design the Authorization Enforcement mechanism to support this in the future.

This calls for some redesign of the authorization enforcement to simplify and standardize authorization enforcement across CDAP code base and also provide a good base for further UI enhancements. The design in the following section presents an approach to solve the above issues.

Goals

  1. Simplify and standardize the process of authorization enforcement in CDAP.
  2. Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)
  3. Enable/Disable operations on the fly for the logged in user (Out of scope for 4.0)

User Stories 

  1. As a CDAP developer, I would like to easily enforce authorization for a code block with minimal code in a standard way.
  2. As a CDAP security admin, I would like to enforce authorization on all entities in CDAP.
  3. As a CDAP security admin, I would like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Scenarios

  • Scenario #1

    Bhooshan is a ninja CDAP developer who is working on a cool new feature, adding lots of new publicly exposed APIs and he wants to support authorization enforcement for his new feature. He looks around in CDAP code base and finds out that he will need to get some classes through the injector and put conditional checks at various places in his new code. He feels overwhelmed by all these new changes and will like to have a simplistic and standard way to add authorization.

  • Scenario #2

    Derek is an IT Operations Extraordinaire at a corporation that uses CDAP. He just upgraded CDAP to use the new authorization features. He granted all the appropriate privileges to different users of CDAP in his corporation and left for the day. When he comes back to the office next day he sees a new ticket assigned to him created by Mr. Little Fingers who is angry because Mr. Tim can see all the logs and metrics which his CDAP programs are generating. Derek is furious and after some digging, he identifies that not all entities in CDAP support authorization and he would like to enforce authorization on all entities in CDAP. 

  • Scenario #3 (Out of scope for 4.0)

    Derek (IT Operations) has been appreciating the authorization feature of CDAP as it has restricted unintentional usage of CDAP entities among the users in his corporation. Although he does not like that users can see the UI button to run a CDAP program even though the user does not have privileges for it. This creates unnecessary confusion among users. He thinks that it will be helpful if a user can only see operations for which he has privileges. 

Design

In our current code base, we do authorization enforcement where the entity id on which enforcement is being done is one of the following: 

 

No.UsageExample
1Entity Id is class member
StreamQueueReader.dequeue
2Entity Id is method parameter
ArtifactRepository.deleteArtifactProperties
3Entity id is parent
ArtifactRepository.addArtifact
4Entity id is constructed from multiple parametersStreamFetchHandler.fetch
5Entity id is a stringDatasetInstanceService.create

 

To represent all the above possibilities we propose the following annotation.

AuthEnforce
/**
 * Annotation for a method that needs Authorization
 * <p>
 * {@link AuthEnforce#entity()}: Specifies the entity on which authorization will be enforced.
 * It can either be a variable name of the EntityId or an array of Strings from which the entity id can be constructed.
 * This variable will be first looked up in the method parameter and if not found it will be looked up in the class
 * member variable.
 * <p>
 * {@link AuthEnforce#enforceOn()}: Class name of one of the CDAP entities on which enforcement will be done. If you
 * want to enforce on the parent of the entity specify that EntityId class here
 * <p>
 * {@link AuthEnforce#privileges()}: Array of Action to be checked during enforcement
 */
@Retention(RetentionPolicy.RUNTIME)
@Target(ElementType.METHOD)
public @interface AuthEnforce {

  // Specifies the entity on which authorization will be enforced. It can either be a variable name of the EntityId or
  // an array of Strings from which the entity id can be constructed. These variable names will be first looked up in the 
  // method parameter and if not found it will be looked up in the class member variable.
  String[] entity();

  // Class name of one of the CDAP entities on which enforcement will be done. If you want to enforce on the parent
  // of the entity specify that EntityId class here
  Class enforceOn();

  // Array of Action to be checked during enforcement
  Action[] privileges();
}

 

entity: Specifies the entity on which authorization will be enforced. It can either be a variable name of the EntityId or an array of Strings from which the entity id can be constructed. This variable will be first looked up in the method parameter and if not found it will be looked up in the class member variable.

enforeOn: Specifies the entityId on which authorization enforcement will be done. It is possible to pass ProgramId in "entity" and enforce authorization on NamespaceId by passing NamespaceId.class here. This allows developers to enforce on parent of an entity. In the case of "entity" being an array of Strings, enforceOn specifies the entityId which will be created.

privileges: The privileges to check for during authorization enforcement on this entity. This is an array of Actions to allow enforcing multiple checks.


Advantage of Authorization Annotations:

  • Authorization annotation will remove the need of adding AuthorizationEnforcer and AuthenticationContext all over the CDAP code base to do authorization.
  • Although, this is not an immediate goal and hence its not documented in the design but later we can also remove the need of injecting Authorization and Authentication module for authorization. This will be possible by making the implementation of these two classes singleton and using them in during the class rewrite.
  • Standardize the way we do authorization enforcement across CDAP
  • Drives us towards Aspect Oriented Programming. Since, authorization annotation will allow us to add additional behavior to existing code without modifying the code. This will also make our code base more modular.
  • Allow us to enable/disable actions in UI/CLI for a logged in user on the fly.

 

Any method that needs authorization enforcement can be annotated with the above annotation. Below are some examples:

 

@AuthEnforce(entity = "artifactId", enforceOn = ArtifactId.class, privileges = {Action.ADMIN})
public void writeArtifactProperties(Id.Artifact artifactId, final Map<String, String> properties) throws Exception {
  artifactStore.updateArtifactProperties(artifactId, new Function<Map<String, String>, Map<String, String>>() {
	...
  });
}
@GET
@Path("/{stream}/events")
@AuthEnforce(entity = {"namespaceId", "stream"}, enforceOn = StreamId.class, privileges = {Action.READ})
public void fetch(HttpRequest request, final HttpResponder responder,
                  @PathParam("namespace-id") String namespaceId,
                  @PathParam("stream") String stream,
                  @QueryParam("start") @DefaultValue("0") String start,
                  @QueryParam("end") @DefaultValue("9223372036854775807") String end,
                  @QueryParam("limit") @DefaultValue("2147483647") final int limitEvents) throws Exception {

 

 

The MainClassLoader which extends an InterceptableClassLoader and is responsible for loading CDAP Master can intercept all the CDAP classes (we will look for package name to start with co.cask.cdap) which need class rewrite by looking for the presence of AuthEnforce annotation. During the class rewrite phase all methods which have AuthEnforce annotation will be rewritten where the first line of the method will be authorization enforcement something like below:

public void writeArtifactProperties(Id.Artifact artifactId, final Map<String, String> properties) throws Exception {
  authorizationEnforcer.enforce(artifactId.toEntityId(), authenticationContext.getPrincipal(), Action.ADMIN);
  artifactStore.updateArtifactProperties(artifactId, new Function<Map<String, String>, Map<String, String>>() {
	...
  });
}

 

The authorization enforcement line which will be generated during class rewrite will always be the the first line of the method with AuthEnforce annotation. There are some places in our code base where we do enforcement in middle of the function and we will need to refactor it for standardization. 

 

Note: AuthorizationEnforcer.enforce() will not be deprecated even after introducing AuthEnforce annotation as after class rewrite we will generate AuthorizationEnforcer.enforce() and also program container will use them rather than the annotation.

API changes

New Programmatic APIs

AuthEnforce annotation as mentioned above.

Deprecated Programmatic APIs

None

New REST APIs

None

Deprecated REST API

None

CLI Impact or Changes

None

UI Impact or Changes

None

Security Impact 

We will replace authorization enforcement all across CDAP with the new AuthEnforce annotation. We expect to write a good Authorization Integration test suite which we can depend on to make sure nothing getting affected by these changes. 

Impact on Infrastructure Outages 

None

Test Scenarios

Test IDTest DescriptionExpected Results
1Test AuthEnforce where entity is a variable name of EntityId in method parameterAuthorization should be enforced
2Test AuthEnforce where entity is String[] which are variable names of strings in method parameter from which entity can be constructedAuthorization should be enforced
3Test AuthEnforce where entity is a variable name of EntityId in class member variableAuthorization should be enforced
4Test AuthEnforce where entity is String[] which are variable names of strings in class member from which entity can be constructedAuthorization should be enforced
5Test AuthEnforce where entity is a variable name of EntityId in method parameter and the same variable name also exists as class member variableAuthorization enforcement should be done for the entity in method parameter
6Test AuthEnforce where enforcement is done on the parent of the specified entityAuthorization should be enforced
7Test AuthEnforce where given entity does not exist in either the method parameters or class member variableClass rewrite should fail causing cdap master to fail to start
8Test AuthEnforce where enforcement is done on an entity which is not the parent of the specified entityClass rewrite should fail causing cdap master to fail to start
 // TODO: Add more test cases here as they become apparent 

Releases

Release 4.0.0

  • Goal 1: Simplify and standardize the process of authorization enforcement in CDAP.
  • Goal 2: Support Authorization all across CDAP (Logs, Metrics, Stream Views, Metadata, Preferences, Kafka, Explore)

Release 4.1.0

  • Goal 3: As a CDAP security admin, I will like users to access/view only the operation for which they have appropriate privileges. (Out of scope of 4.0)

Related Work

  • CDAP-7454 - Getting issue details... STATUS
  • CDAP-7455 - Getting issue details... STATUS
  • Standardize authorization enforcement across CDAP so we can use AuthEnforce annotation. For example, there are some handlers which do Authorization enforcement. Ideally, we will want them to happen in the admins.  

 

Future work

Enabling/Disabling action in UI and CLI

The future work involves taking advantage of authorization annotation to enable/disable actions in UI and CLI for a user on the fly. Below we discuss an initial design to showcase that this is possible with authorization annotation.

We will add a unique name called actionId to every annotation. This will serve as a map between UI and the backend so that for every action in UI we will know what privileges needs to be enforced and on which entity.

 

Annotation for EntityIds
@AuthEnforce(

     entity()

     enforceOn()

     privileges()
 
     actionId("uniqueActionName")
)

 

PrivilegeInspector: When CDAP  Master starts privilege inspector can inspect all the authorization annotation to create a map of  privileges needed for a given action.

Querying Privileges: Once the action to required privilege map is build by the PrivilegeInspector this can be served to the  UI. We will introduce new endpoints which will map every page in UI (like ns page, app page etc) which can be used to construct entityId on which authorization needs to be enforced and the UI will also pass a set of unique action names for which privileges needs to be checked.

For example, let's say a user is on a program page  (/ns/appid/program/checkactions)

/programs end point can be hit with  /program {ns, appid} {set of actions: start, stop}

Now the handler can use the map created by the privilegeInspector to know what privileges are needed for the requested action.


Authorization for listing entities through filter

Another enforcement type supported by  AuthorizationEnforcer is filtering. This is used in most of our listing APIs to list only the entities on which user has privilege. Supporting annotation for AuthorizationEnforcer.createFilter will be a good idea to take another step towards standardization in authorization enforcement across CDAP. But this is something which we will like to do in future due to time constraints of 4.0 and also because this is not needed to enable/disable action in UI/CLI.

 

 

 

 

 

 

 

  • No labels

9 Comments

  1. How do we determine the location (line of code) in a method annotated with @AuthEnforce where authorization enforcement will take place? Will it always be the first line of the method?

     

     

    Also, does this require that the entity be composed of only the parameters of the method (mentioned in the String [] entity)?

  2. Rohit Sinha would be good to breakdown stories that would be completed in each release. Don't think we are addressing all the stories in a single release. In general stories are good. 

  3. Using this annotation, it is not clear to me how the principal is obtained. How does this interact with impersonation? 

    1. The principal is not specified in the annotation as it can be obtained during the annotation processing part (class rewrite). When we rewrite a class which has annotation we just get hold of the AuthenticationConext field and call getPrincipal on it.

      The annotation generate a authorizationEnforcer.enforce command just as a user will be writing it. I am not sure as I understand the question completely as its interaction with impersonation. The behavior will be same as what it is currently where if an action is being performed in an impersonated namespace the getPrincipal call will give the appropriate user for whom the privilege needs to be checked.

      I think the document does not clearly say that there is no changes in the behavior of authorization enforcement with the introduction of annotation. All the annotation does is give a clear, concise and standard way of specifying what enforcement is needed. How its being done is exactly same as what we have today. During the class rewrite the code generated will do the exact same thing which is being done now. I will update the document to say this clearly.

      1. Oh I think I misread, I thought the principal gets injected at class rewrite time. That would only work in program containers. 

        But actually the generated code will assume that the class has a member named authorizationContext (or a getter for that) and call that. That would work. Will you enforce the existence of such a getter using an interface? Say, every class that gets rewritten must implement an interface like AuthContextProvider? 

         

        1. Andreas Neumann The generated code will insert two new fields in the class. One for AuthorizationEnforcer and another for AuthenticationContext and also generate two setter methods for these which will be marked with Inject annotation. Guice will be responsible for injecting these. Please see https://github.com/google/guice/wiki/Injections#method-injection for more details on this.

          This does mean that we expect the binding for these to be defined. If bindings are not defined master will fail to start due to guice binding issue which we think is acceptable. Ideally, we want to move to an approach where we can eliminate the need of the binding being defined. For example, by making these two classes  static singleton but this is out of scope for 4.0. Please let us know your opinion on this. 

  4. Not sure whether I like the idea that code running in program containers performs authorization differently than code running in system containers. What is an example of authorization in program containers?

    1. I discussed with Terence Yim to get an idea of the authorization being done in program containers and see if we can make these two similar. We discussed the following:

      The way we do authorization enforcement in program container and master is similar. The differentiation comes in the way we are specifying what enforcement needs to be done. This is because in program container the entity on which enforcement needs to be done is known and all we need to know is what action to check for so we have ReadOnly, WriteOnly annotation. Whereas as in this case we don't know the entity on which we want to enforce and hence it needs to be specified by the user. Yes, we can make this a bit similar by allowing user to specify ReadOnly, WriteOnly annotation etc in place of actions being specified in AuthEnforce annotation and then the AuthEnforce will take only the entity on which the authorization needs to be performed. The issue with this approach is that this might lead to cases where the developer  will  specify  one annotation and forgot to specify another. Although, we can check this during class rewrite and fail but this will be a runtime failure. In our current design where single AuthEnforce annotation takes all the required information the failure to specify something will cause compile time failure early on and hence will be easier to use.

      Please let me know if you have any feedback or further concerns about this.

       

      1. This makes sense.