In the previous blogs, we’ve covered the foundations of API design, development, and testing. Now it’s time to delve into the crucial “Manage” phase of API management. This is where you’ll truly harness the power of your APIs – controlling their behavior, optimizing their performance, and ensuring rock-solid security before releasing them to external developers.

By the end of this blog, you will have understood below areas of API Lifecycle Management:

  1. Need for managing API and role of policy editor
  2. Flows of a complete API Proxy Call (i.e. preflow and postflow)
  3. Different categories of policies
  4. Applying policies to flows

Table of Contents

  1. Table of Contents
  2. Key Objectives of the “Manage” Phase
  3. Introducing the Policy Editor
    1. Traffic Management Policies
    2. Data Transformation Policies
    3. Security Policies
    4. Extension Policies
  4. Stages of API Call Lifecycle
    1. Understanding API Call Flows
    2. Preflow
    3. Conditional Flow
    4. Postflow
  5. Applying Policies to Flows: Strategic Injection Points
  6. Exercises: Applying Policies
    1. Implement Traffic Management
    2. Achieve Data Transformation
    3. Extend your API
    4. Secure your API
    5. Working with variables
  7. The Benefits of Effective API Management
  8. Conclusion
  9. What is next?

Key Objectives of the “Manage” Phase

  1. Traffic Control: Intelligently managing the flow of requests to your APIs is essential for preventing overload, ensuring smooth performance, and protecting your backend systems.
  2. Data Mediation: Transforming data on-the-fly to ensure compatibility between your APIs and the various apps that consume them. This is especially important when you have legacy systems with different data formats.
  3. Security: Protecting your APIs from unauthorized access, malicious attacks, and data breaches is a non-negotiable priority.

Introducing the Policy Editor

The nerve center of the “Manage” phase is the Policy Editor. This is typically a user-friendly interface within your API management platform. It allows you to apply policies that dictate how your APIs behave. Below is how you navigate to policy editor:

Go to “Configure->APIs” to select the API to which you want to apply the policies:

Once it opens, select “Policies” from top right corner:

There you are now in policy editor, all you need to do is click “Edit” on top right and you should be set to add, remove or configure the policies.

Let’s break down the key types of policies you’ll encounter.

Traffic Management Policies

  • Rate Limiting: Prevent API abuse by setting limits on how many requests a developer or application can make over a certain period.
  • Quota Enforcement: Enforce subscription plans by setting quotas for resource usage depending on a developer’s subscription tier.
  • Spike Arrest: Protect your backend from sudden traffic surges that could lead to outages.

Data Transformation Policies

  • Request/Response Modifications: Alter incoming requests or outgoing responses to match your API’s requirements or a client’s needs.
  • Data Format Conversion: Convert data between formats like JSON and XML for seamless integration with different systems.

Security Policies

  • Authentication: Enforce different methods like API keys, OAuth, or JWT to verify the identity of API consumers.
  • Authorization: Control which resources and operations an authenticated user or app can access.
  • Encryption: Secure data in transit using protocols like TLS/SSL.
  • Threat Protection: Implement policies to detect and block common web attacks like SQL injection or cross-site scripting (XSS).

Extension Policies

Extension Policies in API management allow developers to extend the functionality of API proxies beyond built-in capabilities by writing custom code and referencing them in a policy. The code can be written in Python or Javascript. Not just this, they enable advanced transformations, custom security measures, and integration with external systems, providing flexibility to address unique requirements.

Stages of API Call Lifecycle

Understanding API Call Flows

A complete API call involves several potential stages:

Broadly the flow is divided into two flows: Request Flow and Response Flow

Each flow is then divided into different steps the call goes through to get a makeover through policies laid in its path at each step, wherever applied.

Preflow

Processes the request before reaching the API proxy’s backend routing logic.

Conditional Flow

As the name suggest, whatever policy is applied under conditional flow, comes into play only if a condition is satisfied, also it breaks out of conditional flow as soon as first condition is met. If there is no condition maintained it will executed after Preflow

Postflow

Executes after the Preflow or Conditional flow (if configured)

Now based on the diagram you can infer below sequence of flow execution.

Sequence Request Flow Sequence Response Flow
1 Proxy Request PreFlow 7 Target Response PreFlow
2 Proxy Request Conditional Flows (Optional) 8 Target Response Conditional Flows (Optional)
3 Proxy Request PostFlow 9 Target Response PostFlow
4 Target Request PreFlow 10 Proxy Response PreFlow
5 Target Request Conditional Flows (Optional) 11 Proxy Response Conditional Flows (Optional)
6 Target Request PostFlow 12 Proxy Response PostFlow

PostClientFlow Response (Optional)

Policy Execution Flow

So, for every request and response, we have certain processes to ensure that our users have the experience we want them to have. Let’s review each step in our exercise.

Now, not every policy can be injected wherever you like; it needs to make sense at the step, and it needs to have all the information available to it to be able to modify proxy behavior.

Below we touch upon possible strategic injection points for different types of policies. These are not set in stone but a general guideline.

Applying Policies to Flows: Strategic Injection Points

Knowing when to apply specific policies is key:

  • Preflow/Postflow: Ideal for authentication, authorization, rate limiting, logging, and data transformation.
  • Request Flows: Implement request validation, threat detection, and content filtering.
  • Response Flows: Data masking, response modification, and error handling.

Exercises: Applying Policies

While its not possible to touch upon each and everyu policy available, we will touch upon a few policies which can be part of overall test case.

In this excercise we will try to achieve below behaviour for our NorthWindAPI proxy.

Description Policy Category Policy Name
Application should not be able to make calls from particular IP. Traffic Management Access Control
Application should not be able to make more than 10 calls an hour and 240 calls a day. Traffic Management Quota and Spike Arrest
Application should be able to reset quota to temporary value (3) though header parameter. Traffic Management Reset Quota
Application should not be able to call proxy without correct API Key Security Verify API Key
Convert XML payload to JSON Mediation XML to JSON
Get rid of JSON root node before calling target endpoint. Extension Javascript
Return description of new product (available in request) as response header Mediation Extract Variable

Policy Enforcement Objectives

You will generally get similar behaviour detailed and you need to implement policies to achieve same.

Below step is going to be repetetive so I will go over it once here. Whenever you want to add a policy, Click “Edit” and then make decision in below order.

In Step 2, you need to hit “+” to add the policy to the flow. After step 6, save the proxy and do not forget to deploy the changes.

In below sections, I will talk & demo directly about Step 4 (wherever applicable) and Step 5 mostly. Also, though its not possible to go over each and every policy, we will try to cover most prominent ones. (And may be a few non-prominent ones in an extended blog at the end of blog series)

Implement Traffic Management

We will implement 4 behaviours on our API Proxy.

  1. Caller should not be able to call our API more than 240 times per day.
  2. Caller should not be able to call our API more than 10 time per minute.
  3. Caller should be able to reset the allowed count to 7 times per minute based on a parameter passed in header of the request call.
  4. Caller should not be able to call our API from anywhere else other than one machine (fix IP).

To implement feature#1 we will add “Quota Policy”. Below is the code, you modify or replace in the editor.

<!-- can be used to configure the number of request messages that an app is allowed to submit to an API over a course of unit time -->
<Quota async="false" continueOnError="false" enabled="true" type="calendar" xmlns="http://www.sap.com/apimgmt">
 	<!-- specifies the number of requests allowed for the API Proxy -->
 	<Allow count="240"/>
 	<!-- the interval of time for which the quota should be applied -->
 	<Interval>1</Interval>
	<!-- used to specify if a central counter should be maintained and continuously synchronized across all message processors --> 
	<Distributed>true</Distributed>
 	<!-- Use to specify the date and time when the quota counter will begin counting, 
		regardless of whether any requests have been received from any apps -->
 	<StartTime>2015-2-11 12:00:00</StartTime>
	<!-- if set to true, the distributed quota counter is updated synchronously. This means that
		the update to the counter will be made at the same time the API call is quota-checked -->
	<Synchronous>true</Synchronous>
 	<!-- Use to specify the unit of time applicable to the quota. Can be second, minute, hour, day, or month -->
 	<TimeUnit>day</TimeUnit>
</Quota>

if you exceed 240 calls, you will get below error. You can try making 6 calls to API from postman.

To implement feature#3 we will add “Reset Quota” policy, we will also provide it a condition based on which it need to reset the Quota value to a new value, allowing more calls than number of calls defined in Quota policy from feature#1. Below is XML to be overwritten in editor.

<ResetQuota async="true" continueOnError="false" enabled="true" xmlns="http://www.sap.com/apimgmt">
	<!-- the name of the quota policy that you wish to override -->
	<Quota name="Quota">
		<!-- identifier for a customer/client -->
		<Identifier>
			<!-- the new value of quota -->
			<Allow>7</Allow>
		</Identifier>
	</Quota>
</ResetQuota>

Also make sure “Reset Quota” Policy is ahead in sequence to “Quota” Policy.

Now if you pass a param “reset” with value “yes” to API call. it will keep allowing you 7 more calls as long as Quota is exhausted and value of “reset” is 7.

Note: This is just for demo, if you do not change value of “reset” other than to “yes”, you will not ever exhaust the quota, as it will keep getting reset. To test “Quota” and “Reset Quota” policy correctly change value of “reset” to “no” after first call.

To implement feature#3 we will add “Spike Arrest” policy. Below is the code, you modify or replace in the editor. Spike Arrest policy is essential to avoid DDoS attacks where a URL is flooded with lot of calls in a short span, consuming the badwidth for the site, making it crash and a hefty bill to customer. Ths Spike arrest policy makes sure such surges are avoided by defining number of calls which can be made in smaller time frames. In our case we will restrict this call number to 10 calls per minute (10pm). Out of 240 calls for the day, you will be able to execute max 10 calls within in any minute.

<!-- This policy throttles the number of requests processed by an API Proxy protecting against performance lags and downtime -->
<SpikeArrest async="true" continueOnError="false" enabled="true" xmlns="http://www.sap.com/apimgmt">
<!-- define the rate here -->
	<Rate>10pm</Rate>
	<UseEffectiveCount>true</UseEffectiveCount>
</SpikeArrest>

If you exceed the 10 per minute rate, you will get below error.

To Implement feature#4, we will use “Access Control” policy. In this policy you can define IP details which should be allowed to call this API. All others will be blocked. Default editor setting allow all IPs, but we will use this in the editor. Note: keep IP as 0.0.0.0 for our testing. We will correct it after to allow traffic from our machine.

<AccessControl async='true' continueOnError='false' enabled='true' xmlns='http://www.sap.com/apimgmt'>
  <IPRules noRuleMatchAction='DENY'>
    <MatchRule action='ALLOW'>
      <SourceAddress mask='32'>0.0.0.0</SourceAddress>
    </MatchRule>
  </IPRules>
</AccessControl>

if you test now, this is what you get:

Now if I replace 0.0.0.0 in policy editor with my machine IP. The error will go away.

Now that we have achieved all 4 features for our API successfully. You can play around with the values and other policies to test the behaviour.

Make sure your policies are in below order, else it will not result in expected behaviour. The checks and validations need to be in sequence to achieve desired results. Moreover, I have applied all the policies in Request Preflow as all these validations are need to be done as soon as API call is received.

Achieve Data Transformation

Now lets focus on working Request and Response payloads. Below are the feature we will try to achieve with our API call flow by the end of this part.

  1. We will send an XML payload to our NorthWind Proxy, however our target endpoint expects a JSON. So we need to convert payload from XML to JSON.

We will use “XML to JSON” policy here

Update the policy xml with below:

<!-- This policy converts an XML payload to JSON structure -->
<XMLToJSON async="false" continueOnError="false" enabled="true" xmlns='http://www.sap.com/apimgmt'>
	<Options>
        <OutputSuffix/>
        <OutputPrefix/>
        <AttributePrefix/>
        <AttributeBlockName/>
        <TextNodeName/>
        <TextAlwaysAsProperty>false</TextAlwaysAsProperty>
        <NamespaceSeparator/>
        <DefaultNamespaceNodeName/>
        <NamespaceBlockName/>
        <NullValue/>
        <RecognizeNull>false</RecognizeNull>
        <RecognizeNumber>true</RecognizeNumber>
        <RecognizeBoolean>true</RecognizeBoolean>
        <TreatAsArray/>
    </Options>
	<!-- The variable to which the converted JSON should be assigned to -->
	<OutputVariable>request</OutputVariable>
	<!-- The variable that we want to convert to JSON -->
	<Source>request</Source>
</XMLToJSON>

Now if you save and deploy your proxy and execute it, you will get this error:

The reason is that the target endpoint expect JSON without root node as below:

{
  "odata.type": "ODataDemo.FeaturedProduct",
  "ID": 12,
  "Name": "Aerated Drinks",
  "Description": "Non-diet Aerated Drinks",
  "ReleaseDate": "2022-01-01T00:00:00",
  "DiscontinuedDate": null,
  "Rating": 9,
  "Price": 3.14
}

However our XML to JSON conversion is generating below JSON:

{
    "root": {
        "odata.type": "ODataDemo.FeaturedProduct",
        "ID": 11,
        "Name": "Aerated Drinks",
        "Description": "Non-diet Aerated Drinks",
        "ReleaseDate": "2022-01-01T00:00:00",
        "DiscontinuedDate": {},
        "Rating": 3,
        "Price": 3.14
    }
}

So now we are going to use Javascript to get rid of this root node, this will also demo another category of policies i.e. “Extension Policies”.

Extend your API

As we saw previously the issue is that our target endpoint does not accept JSON with a root node, so we will use Javascript to get rid of root node, before passing the payload to target endpoint.

Add Javascript policy from “Extension policy” section. Fill in the details. Note that I am adding this policy to target endpoint PreFlow as I need to get the JSON right before target endpoint call.

Once you hit “Add”, replace your default JS policy XML code with this:

<!-- this policy allows us to execute java script code during execution of an API Proxy -->
<Javascript async="false" continueOnError="false" enabled="true" timeLimit="200" xmlns='http://www.sap.com/apimgmt'>
	<!-- contains reference to any library scripts that help the main code file -->
	<!-- contains the name of the main code file -->
	<ResourceURL>jsc://jsNoRoot.js</ResourceURL>
</Javascript>

Now Create a Javascript file by same name as shown in “ResourceURL”:

Replace the default JS code with this.

var inputJSON = JSON.parse(context.getVariable('request.content'));
var outputJSON = inputJSON.root;
context.setVariable('request.content', JSON.stringify(outputJSON));

Lets save and deploy the proxy and execute it.

Now we see the call was successful and our product has been created successfully.

Secure your API

So far you may have we are calling the API without any authorization in place. Anyone who has API proxy URL, can call it without any issues. That also subject our proxy to misuse. We need to put some authorization in place to allow only applications who are subscribed to our API. Here we will enforce the usage of API key verification on the calling application, so that unauthorized apps cannot consume our proxy.

The API key you will get after you Publish your product to API Business Hub, which I cover in next blog. You can still read this section and go over it again after you go through next blog about “Engage” phase.

From you “Security Policies” section add the policy “Verify API Key” to Proxy Endpoint PreFlow as we want API key validated before we rout this request to our target endpoint.

Replace default policy xml to below:

 <!--Specify in the APIKey element where to look for the variable containing the api key--> 
<VerifyAPIKey async='true' continueOnError='false' enabled='true' 
xmlns='http://www.sap.com/apimgmt'>
	<APIKey ref='request.header.APIKey'/>
</VerifyAPIKey>

Now if we save and deploy our proxy, this is what we will get.

The reason is that we have not provided any API key in our request yet. If you notice in the policy, our policy engine is looking for API key in header with key name “APIKey”. Lets add this header and execute the request again.

API key you will get after you publish your API as a product in API Business Hub. So you might want to wait for next blog for same to get those details.

FYI - at this stage the Proxy endpoint preflow looks like this as far as sequence of policies is concerned.

Cool, we are almost there. Just one more topic I want to touch upon. So far we have done all the modifications and policy additions to the request flow. However once target endpoint is called, you can enforce difference policies in response flow as well. In the next section we will see how?

Working with variables

Our objective here is to extract the “description” coming in the request and pass this back to caller in response “header”.

To achieve this we need to do two things:

  1. Extract the description from request and store it in a variable.
  2. Once we get response from target endpoint, add the description to list of response headers.

For step 1, we will use “Extract Variables” policy, it will applied at target endpoint PreFlow:

replace default code with this:

<!-- Extract content from the request or response messages, including headers, URI paths, JSON/XML payloads, form parameters, and query parameters -->
<ExtractVariables async="true" continueOnError="false" enabled="true" xmlns='http://www.sap.com/apimgmt'>
	<!-- the source variable which should be parsed -->
	<Source>request.content</Source>
	<!-- Specifies the XML-formatted message from which the value of the variable will be extracted -->
	<JSONPayload>
	<!-- Specifies variable to which the extracted value will be assigned -->
	    <Variable name="description" type="string">
		    <!-- Specifies the XPath defined for the variable -->
		    <JSONPath>$.root.Description</JSONPath>
	    </Variable>
	</JSONPayload>
</ExtractVariables>

For Step 2, we will use “Assign Message” Policy to set headers. This policy is used to set header, URL parameters and payload properties.

Note this that we are setting header after getting the response so this time when we are creating the policy, make sure the Stream selected in outgoing response as shown below:

Note in the Diagram where does this policy sit:

Replace the default code with this:

<!-- This policy can be used to create or modify the standard HTTP request and response messages -->
<AssignMessage async="false" continueOnError="false" enabled="true" xmlns='http://www.sap.com/apimgmt'>
 
	<!-- Sets a new value to the existing parameter -->
	<Set>
		<Headers>
		    <Header name="Content-Type">application/json</Header>
		    <Header name="description">{description}</Header>
		</Headers>
	</Set>
	<IgnoreUnresolvedVariables>false</IgnoreUnresolvedVariables>
	<AssignTo createNew="false" type="response">response</AssignTo>
</AssignMessage>

Save and Deploy. Execute the request again. You will notice a new header in response called “description” with the value from request “Description”.

So, now you see you can use variable to store and use the values.

There is too much to unpack here in context of policies, I have tried to cover basics to enable you to understand working of different parts of API management policies. But you will need to go through official SAP API Management documentation to be able to implement different policies. I plan to write extended blog after this series ends. to cover othe common topics like implementing OAuth2 etc. Do subscribe to get email alerts when they drop.

The Benefits of Effective API Management

By mastering the “Manage” phase of API management, you’ll reap numerous benefits:

  • Security: Reduced vulnerabilities
  • Performance: Optimized response times and streamlined operations
  • Scalability: Ability to handle growing usage demands
  • Compliance: Adherence to industry regulations and security standards
  • Developer Experience: Enhanced ease of use and onboarding for external developers

Conclusion

The “Manage” phase is where you transform your APIs into strategic assets. Through the power of your Policy Editor, you’ll optimize traffic flow, fortify security, and seamlessly adapt to the needs of your users. Investing in API management leads to secure, performant, and developer-friendly APIs that drive success.

Remember, continuous monitoring and analysis will help you refine your API management strategies over time.

What is next?

Now that you’ve mastered API management, it’s time to unlock the final gateway to your API’s success. In our next blog, we’ll delve into the “Publish” phase of API management. We’ll explore how to create a world-class developer portal for easy API discovery, seamless onboarding, and how to effectively promote your APIs to your target audience.

Stay tuned!