1. SPECS (Team) Avatar SPECS (Team)
  2. SPECS
  3. specs-utility-security-tokens

Wiki

Clone wiki

specs-utility-security-tokens / Home

View History

Table of Contents

Overview

The specs-pki-tokens project provides infrastructure for dealing with Public Key Infrastructure (PKI) tokens. A PKI token is a digitally signed piece of information intented for transmitting assertions from the token issuer to relying parties. The token carry claims (statements) about a specific subject made by the token issuer.

The specs-pki-tokens project consists of following modules:

  • security-token-service: a RESTful web service that issues the PKI tokens
  • pki-tokens-client: library providing support for requesting, parsing and validating of PKI tokens

PKI tokens are designed with two goals in mind:

  • to enable offline validation of tokens without calling the central validation service. The advantages of offline validation are shorter time and less traffic and load on the Security token service
  • to be small enough to fit into HTTP header so the token could be simply put into HTTP header with every request when calling various web services

PKI tokens resemble SAML assertions to a certain extent but use much simpler token model than SAML and use the JSON encoding syntax. Therefore PKI tokens are not intended as a full replacement for SAML assertions, but rather as a token format to be used when ease of implementation or compactness are important.

PKI tokens are digitally signed which ensures the authenticity and integrity of tokens. Authenticity means that the receiver of the token can be assured that the token really comes from the Security token service, integrity means that the receiver can be sure that the token has not been altered during transmission.

PKI tokens are not encrypted, therefore they must be transmitted only through secure connection. It is safe to put the token into HTTP header but it is not recommended to put them in URL query string.

Token Structure

PKI tokens consists of three parts:

  • header
  • payload
  • signature

The header contains token's metadata, payload is a collection of claims about specific subject and signature is digital signature of header and payload combined.

Header

The token's header is a JSON object that contains the metadata about the token as a collection of key/value pairs. The header is digitally signed.

The header contains following parameters:

Param Name Description
sigAlg signature algorithm Signature algorithm which was used for creating the digital signature. The algorithm choice is configurable and can be set in the Security token service's configuration.
iat issued at The time at which the token was issued.
exp expiration date Expiration date of the token in milliseconds format.
iss issuer The entity which issued the token and digitally signed it.
scf signing certificate fingerprint The fingerprint of the signing certificate which was used to sign the token.

The following example shows what the header of the token looks like: 

{
  "sigAlg" : "SHA256withRSA",
  "iat" : 1411072932201,
  "exp" : 1411076532201,
  "iss" : "specs-demo",
  "scf" : "01:18:BD:FE:5A:AF:DC:64:21:F5:07:93:7C:87:50:F6:5E:4C:75:B0"
}

Payload

The token's payload is a collection of claims written in JSON format. A claim is a statement made by the issuer (the Security token service) about a specific subject. A claim consists of a name and value. Claim name is a string and claim value can be of any JSON type.

An example of a claim, containing user attributes. Key names are shortened to make the token as small as possible:

{
 "user" :
 {
   "id" : "d3c23310-18be-11e4-8c21-0800200c9a66",
   "un" : "test.user",
   "fn" : "Test",
   "ln" : "User",
   "em" : "test.user@specs.org",
   "ro" : ["SPECS_USER"]
 }
}

Signature

The signature part contains the digital signature of token's header and payload combined together.

Signing and Encoding the Token

A token is encoded into compact ASCII string format suitable to be put in a HTTP header. The token consists of three parts: header, payload and signature which are concatenated together and separated with a dot separator. All parts are encoded using Base64 encoding scheme. The structure of an encoded token is as follows:

<header>.<payload>.<signature>

Signing and encoding the token takes the following steps:

  1. The header's JSON representation is encoded using Base64 encoding scheme.

  2. The JSON representation of the claims collection (tokens' payload) is compressed into GZIP format. The compressed byte sequence is encoded into ASCII string format using Base64 encoding scheme.

  3. The JSON representation of the token's header is concatenated with the JSON representation of the token's claims collection.

  4. The concatenated JSON data is encoded into byte sequence using UTF-8 encoding.

  5. The resulting byte sequence is digitally signed with signing private key using specific signature algorithm. As a result we get digital signature byte sequence which is encoded into ASCII string format using Base64 encoding scheme.

  6. All three Base64 encoded parts (header, payload and signature) are concatenated together with a dot separator.

The following example shows signing and encoding a token with the header and payload given above:

Base64 encoding the token's header gives:

eyJzaWdBbGciOiJTSEEyNTZ3aXRoUlNBIiwiaWF0IjoxNDExMDcyOTMyMjAxLCJleHAiOjE0MTEwNzY1MzIyMDEsImlzcyI6InNwZWNzLWRlbW8iLCJzY2YiOiIwMToxODpCRDpGRTo1QTpBRjpEQzo2NDoyMTpGNTowNzo5Mzo3Qzo4Nzo1MDpGNjo1RTo0Qzo3NTpCMCJ9

The payload compressed to GZIP format and Base64 encoded:

H4sIAAAAAAAAAE2OTQuDMBBE/8uejWySIqmngngvTT2JFBu3RfCLJB6K+N+7Qg+9DW8ew2zghrYfA+T1BvGzEORQBfLFQSGBvmPQaae0liikeZKQkk7COCUFGkSF6M5tlrG7TuxGCjFdeYHB6wB3BpyH6bfMmcZ/8RIWciGd/ZsrP/MTsNeysI/Kljdo9mb/AhWBiz6lAAAA

The header and payload JSON representation are concatenated together: 

{"sigAlg":"SHA256withRSA","iat":1411072932201,...}{"user":{"id":"d3c23310-18be-11e4-8c21-0800200c9a66","un":"test.user",...}}

The resulting JSON data is encoded into byte sequence using UTF-8 encoding and digitally signed. The signature byte sequence is Base64 encoded which yields: 

PcUIw0kojidGqKMDPIiGE8g4N8XBas/ZtppcwhfqlLd60FMyDiMKti9iQ6pfyxox7telRiraKO4SvremR7FgaF8XDYqPJ09/VoPR/iPA5v0rX7VbpZOy0C/mjCKoIKrN7MfNgKJkvwxQh9Iih8SoShFff19buwHEt8bjcA1y9bsHRi+M+Ovt3yVM2hNWX7EazA59T8gUWhbPcKNsqw0VMvwlEBCxod0vt2eVrVHgTVUzlNPgqaNOCVJlwflvPQGRaVlW2otJ3jhf/r4OkEPm2Wq+rSQ9843cXCNr+kEJnlr0vXmwvXC6zXEu2e+5PvBV/2tiQs3nDFI8aq+hb8jzIA==

Finally all three Base64 encoded parts are joined together using the dot separator. The size of the resulting token is 754 bytes.

Decoding and Validating the Token

The token is decoded in the following steps:

  1. The encoded token is split into three parts (header, payload and signature) using the dot separator.

  2. The header part is decoded from the Base64 encoding scheme, we get the header in the JSON format which is deserialized into Header object.

  3. The payload part is decoded from the Base64 encoding scheme. The resulting byte sequence is decompressed, we get the payload in the JSON format which is deserialized into Payload object.

  4. The signature part is decoded from the Base64 encoding scheme.

To make sure that the token is valid three things have to be validated:

  • token signature
  • token expiration date
  • token revocation status

A digital signature provides proof of the originator of the token and the integrity of the signed data against tampering or corruption. To verify the signature, the client needs the signing certificate (precisely, the public key) of the token's issuer (Security token service). The fingerprint of the signing certificate and the signature algorithm are specified in the token's header. The header and payload in JSON format which were decoded and decompressed from the token are concatenated together. This concatenated string is then used for digital signature verification.

The token's expiration date is given in the token's header. The expiration date is then compared against the current time on the client machine. For this to work correctly, it is important that the time difference between the client and the token's issuer is not too big.

By checking the token revocation status we make sure that the token hasn't been revoked. The revocation of tokens is handled by token revocation list (TRL) which resembles the certificate revocation list (CRL). When a token is revoked it is put on the TRL maintained by the Security token service. The client pulls this list at the startup and then periodically updates it on predefined time intervals.

Architecture

PKI tokens infrastructure consists of three modules:

  • pki-tokens-core
  • security-token-service
  • pki-tokens-client

The module pki-tokens-core is a library that provides the core classes and methods for dealing with tokens, digital signatures. The module security-token-service is a RESTful web service that issues the tokens and maintains the token revocation list. The module pki-tokens-client is a library intended to be used by applications and web services to provide support for PKI tokens.

The following diagram depicts the architecture of the PKI tokens infrastructure:

The architecture of the PKI tokens infrastructure

PKI Tokens Core Module

The module pki-tokens-core is a library with core classes, token definition, signing and validation functionality and is used by the security-token-service and pki-tokens-client modules.

Class Token

The Token has the following structure:

class Token {
  Header header;
  Payload payload;

  class Header {
    String signatureAlgorithm;
    Date issuedAt;
    Date expiryDate;
    String issuer;
    String signingCertFingerprint;
  }

  class Payload {
    Map<String, Object> claims;
  }
}

The Token provides following API:

String getTokenId()
Header getHeader()
Payload getPayload()
String getEncodedValue()
String sign(TokenSigner signer)
static Token decode(String encodedToken, TokenSigner tokenSigner)
String dump()

Creating a new Token

A new token is created with a no-arg constructor Token(). Then we add claims to the token's payload with the method addClaim():

Token token = new Token();
token.getPayload().addClaim(String name, Object value)

A claim has a name of the type String and a value which can be any type but must be serializable to JSON.

Signing the Token

To sign the token we call the token's method sign() by providing the TokenSigner object:

String sign(TokenSigner signer)

The TokenSigner represents an entity which signs the tokens. It is constructed by calling the constructor with following parameters:

  • signerName: name of the signer, this value is added to the token's header.
  • keystoreFile: key store where the signing certificate is stored.
  • keystorePass: password of the signing key store
  • signingCertFingerprint: fingerprint of the signing certificate stored in the key store
  • signingPrivateKeyPass: password of the signing private key (corresponding to the signing certificate) stored in the key store 
TokenSigner signer = new TokenSigner(
    SIGNER_NAME,
    SIGNING_KEYSTORE_FILE,
    SIGNING_KEYSTORE_PASS,
    SIGNING_CERT_FINGERPRINT,
    SIGNING_PRIVATE_KEY_PASS
);

The sign() method first creates the token's header and sets all the fields (issue date, expiration date, signature algorithm and signer name). The header is then serialized to JSON format and encoded with Base64 encoding scheme. The dates are represented in milliseconds format. The token's payload is serialized to JSON format, compressed to GZIP format and encoded to Base64. The encoded header and payload are then digitally signed using the Java security API:

Signature sigInstance = Signature.getInstance(sigAlgorithm);
sigInstance.initSign(signer.getSigningPrivateKey());
sigInstance.update(message);
byte[] signature = sigInstance.sign();
byte[] signatureEncoded = Base64.encode(signature);

The signature byte array is encoded to Base64. All three parts (header, payload, signature) are concatenated with the dot separator and we get the encoded token which is the return value of the sign() method.

After signing the token the sign() method sets the token ID by computing the SHA-256 hash of the token's signature byte array and converting it to a string value using hexadecimal encoding:

MessageDigest md = MessageDigest.getInstance("SHA-256");
md.update(signature);
byte[] digest = md.digest();
tokenId = DatatypeConverter.printHexBinary(digest);

Decoding and Validating Tokens

The class Token provides the method decode() which decodes the token from the encoded value and validates it:

static Token decode(String encodedToken,
                    VerificationCertProvider verifCertProvider,
                    RevocationVerifier revocationVerifier)

The method accepts three parameters:

  • encodedToken: the encoded token
  • verifCertProvider: an instance of the VerificationCertProvider which provides the signing certificate needed to verify the token's signature
  • revocationVerifier: an instance of the RevocationVerifier which is responsible for checking if the specified token has been revoked

The interface VerificationCertProvider defines following method:

X509Certificate getCertificate(String fingerprint)
The method getCertificate returns signing certificate with the specified fingerprint.

The interface RevocationVerifier defines following method:

boolean isRevoked(String tokenId)
The method isRevoked checks if the token with specified ID has been revoked.

The decode() method splits the encoded token into three parts: header, payload and signature. Then it decodes the header part, checks the expiration date and retrieves the signature algorithm used for creating the signature. The digital signature is verified using the Java security API:

Signature sigInstance = Signature.getInstance(header.getSignatureAlgorithm());
sigInstance.initVerify(tokenSigner.getSigningCertificate());
sigInstance.update(encodedTokenBytes, 0, pos2);
boolean isVerified = sigInstance.verify(signature);

If the signature is valid the token's payload is decoded and decompressed from the payload part of the encoded token. The token ID is computed by calculating the SHA-256 hash of the token's signature (decoded from Base64) and converting the hash to a string value using the hexadecimal encoding. Finally a new Token instance is created, populated with data and returned.

Security Token Service

The security-token-service is Java RESTful web service that provides following functionality:

  • issuing PKI tokens
  • revoking tokens
  • generating token revocation lists and providing them to clients
  • providing signing certificate to clients for verifying the digital signatures

Issuing PKI Tokens

A client can request a token by calling the method:

POST /pkitokens
Consumes: application/json
Produces: text/plain

The request must specify the subject of the token to issue. The method creates a PKI token containing the claims about the specified subject. The STS retrieves required data from the specs-db database, creates the token with corresponding claims, digitally signs and encodes it and returns it to the client. The issued token is registered in the STS database where it is hold till the token expiration.

Revoking the Tokens

A token can be revoked by calling the following method:

DELETE /pkitokens/{tokenId}

The method marks the specified token in the database as revoked and stores its revocation date. When a new token revocation list will be generated this token will be put on the list and later pulled to the clients. The revoked token remains in the database till the token expiration date extended by some safety time interval. This safety time interval is required because of the potential time lag between the STS and client time. The safety time interval must be greater than maximum tolerable time lag. Without such safety time interval, following situation could have happened: a token is revoked just before its expiration. When the token expires, it is removed from the database and is no longer included on the token revocation list. The client updates its local token revocation list cache but is not notified about this token revocation. The client's clock is a little behind the STS's clock. Because of this time difference, the token is still valid on the client.

Generating Token Revocation List

The STS periodically generates token revocation list (TRL) which contains a list of revoked not expired tokens. There are two types of revocation lists, both are generated every time:

  • full revocation list
  • delta revocation list

The full revocation list contains all revoked tokens which are not yet expired at the time of revocation list creation. The full TRL is usually pulled by the clients only once when the client's local TRL is being initialized.

The delta revocation list contains only the tokens which have been revoked since the last delta list generation. The aim of delta revocation list is to reduce the size of revocation list, amount of network traffic and load on the STS. The client to be up-to-date has to download the full revocation list at initialiation time and then periodically download all the delta revocation lists. In case the client misses or fails to download just one delta list the local TRL cache has to be cleared and initialized again by downloading the full TRL. The TRLs are numbered sequentially so the client knows which delta it has to download, that is the last TRL the client has downloaded number plus 1. If the delta TRL with requested number is obsolete the server returns the full TRL.

Example of a full token revocation list:

{
  "tokens" : [
    {
      "id" : "4C6CC09D7EB229F944D21B5E781CC8B45046E5AB52B1DE1A5D2FEED498DB401C",
      "exp" : 1409239406240
    }
  ],
  "id" : 140823,
  "created" : 1409237139613,
  "type" : "full"
}

A token revocation list can be retrieved by calling the following method:

GET /trl?id={id}&type={type}
Produces: application/json

Query parameters:

  • id: sequential number of the TRL
  • type: type of the TRL, can be full or delta

Providing Signing Certificates

The STS uses the signing private key for signing the tokens. For verifying the digital signatures the clients need corresponding signing certificate (public key). The signing certificate can be downloaded from the STS using the following method:

GET /certificates/{fingerprint}
Produces: application/x-pem-file

The method loads the specified certificate from the key store and returns it in the PEM format.

Replacing the Signing Certificate

The STS supports replacing the signing certificate without any service interruption. For example, when the signing certificate is about to expire, it has to be replaced or renewed. Just simply replacing the old certificate with a new one wouldn't work correctly because the tokens that were issued just before the switch are still valid after the switch but clients that would want to validate such token wouldn't be able to obtain the certificate used for signing the token to verify the signature.

To properly handle such situation the STS adds the signing certificate fingerprint into the token header. Using this fingerprint, clients are able to obtain the proper signing certificate and use it to validate the token signature. The signing certificate which is nearing its expiration date has to be replaced in time so that the last issued token expires before the certificate used for signing expires.

The STS stores signing certificates in a key store. The key store holds the active signing certificate and potential retired signing certificates. The active one is used for signing, others can be retrieved by clients for validating the signatures.

The procedure for replacing the signing certificate is as follows:

  1. Obtain/generate a new signing certificate.

  2. Determine the certificate fingerprint. Using the OpenSSL, this can be done with the following command: 

    openssl x509 -noout -in signing-cert.pem -fingerprint

  3. Import the new certificate into the STS signing key store: 

    openssl pkcs12 -name "<fingerprint>" -in signing-cert.pem -inkey signing-key.pem -out signing-keystore.p12 -export

  4. Update the signing certificate fingerprint in the STS configuration file (parameters signingCertFingerprint and signingPrivateKeyPass.

PKI Tokens Client Module

The pki-tokens-client is a client library that provides support to Java applications for using PKI tokens. There are two use cases:

  • obtaining a PKI token
  • decoding and validating a PKI token and retrieving required information from it

Use Cases

Obtaining a PKI Token

In the first use case an application has to obtain a PKI token which it needs to call some web service which requires a PKI token for example to authorize the request or to get some information from the token needed to fulfill the request. For this use case the pki-tokens-client library provides class PkiTokenRetriever with the following API:

PkiTokenRetriever(String stsAddress,
                  String trustStoreFile, String trustStorePass,
                  String keyStoreFile, String keyStorePass,
                  VerificationCertProvider verifCertProvider
                  );
Token obtainToken(String username, String password, int slaId);

The PkiTokenRetriever is constructed using the following parameters:

  • stsAddress: address of the STS
  • trustStoreFile, trustStorePass: trust store file path and password. The trust store contains the CA certificate chain (the issuer of the STS server certificate). The trust store is needed to validate the STS server certificate when establishing the secure connection with the server.
  • keyStoreFile, keyStorePass: key store file path and password. The key store contains the client certificate and private key which are needed to authenticate to the STS.
  • verifCertProvider: an instance of VerificationCertProvider which is needed to obtain signing certificate

The method obtainToken accepts three parameters: username, password and slaId. It calls the STS and requests a PKI token for specified user and the SLA. The method decodes the received token and returns the Token object.

Decoding and Validating a PKI Token

The second use case happens when for example some web service receives a PKI token attached to a HTTP request (usually in a HTTP header). The web service has to decode and validate the token, and retrieve some information from the claims. For this use case the pki-tokens-client library provides the class PkiTokenValidator with the following API:

PkiTokenValidator(VerificationCertProvider verifCertProvider, RevocationVerifier revocationVerifier)
Token decodeAndValidate(String encodedToken)

The PkiTokenValidator is constructed using the following parameters:

  • verifCertProvider: an instance of VerificationCertProvider which is needed to obtain signing certificate
  • revocationVerifier: an instance of RevocationVerifier which is needed to check the token revocation status

The method decodeAndValidate decodes and validates the provided encoded token and returns the Token object. From the Token object the service can get the token's payload and claims it contains.

VerificationCertProvider Implementations

The pki-tokens-client module provides two implementations of VerificationCertProvider interface:

  • VerificationCertProviderP12
  • VerificationCertProviderWS

The VerificationCertProviderP12 retrieves requested signing certificate from a local trust store. The trust store is maintained manually by the administrator who obtains the signing certificate(s) and imports it into the trust store. No connection with the STS is required.

The VerificationCertProviderWS retrieves requested signing certificate from the STS by calling its RESTful API and caches it locally. Afterwards the certificates are retrieved from the local cache.

RevocationVerifier Implementations

The pki-tokens-client module provides two implementations of RevocationVerifier interface:

  • TRLCache
  • OnlineRevocationVerifier

The TRLCache maintains a local token revocation list (TRL) cache. At the initialization it downloads the full TRL, after that it periodically pulls delta TRLs with the recent changes (tokens revoked since the last update). Also, it periodically cleans up the TRL - removes all tokens from the list which have expired. The tokens revocation status is checked against local TRL copy. This approach is fast - the validation can be accomplished locally. The disadvantages are time lag between token revocation and client local cache and complexity of token revocation list synchronization.

The OnlineRevocationVerifier checks token revocation status by calling the STS RESTful API for each token. This approach is most accurate - there is no time lag between token revocation and client local cache. The disadvantage is that it requires a call to the STS for each validation which takes some time, causes a lot of network traffic and load on the STS.

Command Line Client Application

The pki-tokens-client provides command line application which can be used for obtaining PKI tokens, decoding and validating them, checking if specific token is revoked, printing the TRL. The application can be started from the command line with the following parameters: sts-address, truststore-file and truststore-pass:

java -cp "libs/*" org.specs.pkitokens.client.ConsoleClient --sts-address=https://localhost:8443/sts --truststore-file=specs-truststore.jks --truststore-pass=password

When started the application provides following commands:

obtain <username> <password> <sla-id>
validate <token>
isRevoked <token-id>
printTRL
exit

PKI Tokens Servlet Filter

The pki-tokens-client library provides Java servlet filter called PkiTokensFilter that can be plugged into any Java web application or Java RESTful web service. The filter intercepts every HTTP request before it is processed, checks if a PKI token is present, decodes and validates the token and puts the Token object into HttpServletRequest object. From the HttpServletRequest object the Token is available to servlets processing the request which are able to get information stored in the token.

The filter can also be used for the authorization. For example, the token can include an authorization claim that specifies all SPECS services which the user is eligible to access based on the SLA. The filter extracts this list of services from the token and checks if the request is authorized.

The filter can be deployed in the web application deployment descriptor file (web.xml). First we declare the filter:

<filter>
   <filter-name>PkiTokensFilter</filter-name>
   <filter-class>org.specs.pkitokens.client.PkiTokensFilter</filter-class>
   <init-param>
      <param-name>configFile</param-name>
      <param-value>pki-tokens-config.xml</param-value>
   </init-param>
</filter>

Then we map the filter to specific servlet or URL pattern, for example:

<filter-mapping>
   <filter-name>PkiTokensFilter</filter-name>
   <servlet-name>Jersey REST Services</servlet-name>
   <dispatcher>REQUEST</dispatcher>
</filter-mapping>

References

[1] M. Jones, J. Bradley, N. Sakimura, JSON Web Token (JWT), draft-ietf-oauth-json-web-token-25, July 4, 2014, https://tools.ietf.org/html/draft-ietf-oauth-json-web-token-25

[2] Jones, M., J. Hildebrand, JSON Web Encryption (JWE), draft-ietf-jose-json-web-encryption-31, July 4, 2014, https://tools.ietf.org/html/draft-ietf-jose-json-web-encryption-31

[3] M. Jones, J. Bradley, N. Sakimura, JSON Web Signature (JWS), draft-ietf-jose-json-web-signature-31, July 4, 2014, https://tools.ietf.org/html/draft-ietf-jose-json-web-signature-31

[4] R. Housley, Cryptographic Message Syntax (CMS), September 2009, http://tools.ietf.org/html/rfc5652

[5] OpenStack Identity API v3, https://github.com/openstack/identity-api/blob/master/v3/src/markdown/identity-api-v3.md

[6] Bartek Kupidura, July 10, 2013, Understanding OpenStack Authentication: Keystone PKI, https://www.mirantis.com/blog/understanding-openstack-authentication-keystone-pki/

[7] Wikipedia, Claims-based identity, http://en.wikipedia.org/wiki/Claims-based_identity

Updated