Monday, April 14, 2014

How to remove custom transport headers in a WSO2 ESB Connector

When u send an HTTP request, as same as you set specific headers for the request, you also will get a set of headers in your response as well no matter the response is successful or an error response.

You may get some custom headers along with the standard HTTP headers. In this post, I will be describing on how you should remove those custom headers if you don't want to send them to your end user.

First off all, you should know what are considered as standard headers. You can have a very useul article and along with the necessary list of headers in the below link.

List of HTTP header fields

The following Screenshot shows you a GET request sent from Postman that's made to Google Calendar REST API. You can see the headers section that there are some headers resulted that are not in the above list of standard HTTP headers.

























You can add the below section as a set of properties in your synapse Sequence to be executed after making the call to the endpoint. So, the custom headers can be removed easily.

<!-- Remove response custom header information --> 
<header name="X-Frame-Options" scope="transport" action="remove" /> 
<header name="X-XSS-Protection" scope="transport" action="remove" /> 
<header name="Alternate-Protocol" scope="transport" action="remove" /> 
<header name="X-Content-Type-Options" scope="transport" action="remove" />

Hope it was useful to you.

How to set HTTP headers in a connector for WSO2 ESB

HTTP header fields are components of the message header of requests and responses in the Hypertext Transfer Protocol (HTTP). They define the operating parameters of an HTTP transaction. Most of the API calls requires to set some headers before making each API calls no matter its GET, POST or any other request type. 

In this post, I am going to focus on the headers that needs to be set for HTTP calls for Google Calendar REST API. 














API:                                   Google Calendar
Reference Link:                    https://developers.google.com/google-apps/calendar/v3/reference/
Headers need to be set :  Authorization , Content-Type

What do those headers mean?

Authorization : Authentication credentials for HTTP authentication
Content-Type: The MIME type of the body of the request

Now, let's see how to set those headers within the synapse template of the connector.It's quiet simple and the below code explain them to you.

<template name="test" xmlns="http://ws.apache.org/ns/synapse">
<!-- Set a parameter to get the access token  -->
<parameter name="accessToken"
description="The access token  of the Google Calendar API" />
<sequence>
<!-- Get the access token from the user  through synapse-->
<property name="uri.var.accessToken" expression="$func:accessToken" />

  <!--Set the access token as the Authorization transport header as below --> 
<property name="Authorization"
expression="fn:concat('Bearer ', get-property('uri.var.accessToken'))"
scope="transport" type="STRING" />

 <!-- Setting the Content-Type transport header as below-->
<property name="Content-Type" value="application/json" scope="transport"
type="STRING" />

<call>
<endpoint>
<!-- you can call the required end point from here  -->
</endpoint>
</call>
</sequence>
</template>

As you see above while setting the Authorization header i have used a concatenation function. That's because the authorization header should be set along with the ''Bearer" field concatenated with the access token. 

That's all about it. You can set any HTTP header by following the similar approach. Hope it was useful to you...

Tuesday, April 8, 2014

Using Script mediators to build a Json payoad with optional parameters in a connector for WSO2 ESB

When making requests to an API, it's very important to identify the parameters that are required (mandatory) and that are optional. It's also essential to know the type of those parameters and the proper way of sending them to the API such as if the api supports REST and we are making a REST/json request, we must ensure that the request that we are sending is a valid json string.

In this post, I wanted to focus bit more on how to handle body parameters that are optional. The concept beind using optional parameters are, it says, a user need not mandatory send those values to call that function but if they wish to set those values as well, then the function should support them. 

As in my other similar posts, let me take a real word example to demonstrate this.

API:                     Google Calendar
Function:             CalendarList: insert
Reference Link: https://developers.google.com/google-apps/calendar/v3/reference/calendarList/insert

The above mentioned request is an HTTP POST request to create a new Calendar List Entry.According to the API documentation, it expects a list of values to be sent through the request body and the request body consists of both required and optional parameters. 

So, these should be handled carefully within the synapse template of the connector. The concept is clear. We are going to set a payload factory to send the request body parameters and the required parameters can be directly hard coded. Also, the optional parameters will be added to the request body checking the condition that the parameter holds a value (it should be not null and not empty to be added to the payload).

The below shown code from the synapse template will clearly explain you how this task is handled.

Note that this function has id as the only request body parameter that is considered to be required and all the rest of the parameters are optional.

The request body parameters should be declared as below;

<!-- Required parameters -->
<parameter name="id" description="Identifier of the calendar." />

<!-- Optional parameters -->
<parameter name="defaultReminders" description="A list of authenticated users for this calendar. Each item has method and minutes." />
<parameter name="foregroundColor" description="The foreground color of the calendar in the format '#ffffff'." />
<parameter name="isHidden" description="Whether the calendar has been hidden from the list." />
<parameter name="notificationSettings" description="The method used to deliver the notification. " />
<parameter name="summaryOverride" description="The summary that the authenticated user has set for this calendar." />

Property mediators should be set to the above parameters to map with the proxy as below within the <sequence>.

<property name="uri.var.defaultReminders" expression="$func:defaultReminders" /> 
<property name="uri.var.foregroundColor" expression="$func:foregroundColor" /> 
<property name="uri.var.isHidden" expression="$func:isHidden" /> 
<property name="uri.var.notificationSettings" expression="$func:notificationSettings" /> 
<property name="uri.var.summaryOverride" expression="$func:summaryOverride" />

Then the payload factory can be added with the required (mandatory) parameters as shown below.

<!-- Building request with mandatory params -->
<payloadFactory media-type="json">
<format>
{
"id":"$1"
}
</format>
<args>
<arg expression="get-property('uri.var.id')" />
</args>
</payloadFactory>

Finally, the script mediator can be used to add the optional request body parameters on checking the conditions as below.

<script language="js">
<![CDATA[        
//request body param variables
var defaultReminders = mc.getProperty('uri.var.defaultReminders');
var foregroundColor = mc.getProperty('uri.var.foregroundColor');
var isHidden = mc.getProperty('uri.var.isHidden');
var notificationSettings = mc.getProperty('uri.var.notificationSettings');
var summaryOverride = mc.getProperty('uri.var.summaryOverride');                                    
//getting the json payload of the message context.
var payload = mc.getPayloadJSON();
//if values set for defaultReminders Array, then add it to the payload
if ( defaultReminders != null && defaultReminders != "") { 
payload.defaultReminders = eval("("+defaultReminders+")");
}
//if a value is set for foregroundColor , then add it to the payload
if (foregroundColor != null && foregroundColor != "") {
payload.foregroundColor = foregroundColor;
}
//if isHidden parameter is set, add it to the payload
if (isHidden != null && isHidden != "") {
payload.hidden = isHidden;
}
//if values set for notificationSettings Object, then add it to the payload
if ( notificationSettings != null && notificationSettings != "") { 
payload.notificationSettings = eval("("+notificationSettings+")");
}
//if values set for summaryOverride , then add it to the payload
if (summaryOverride != null && summaryOverride != "") {
payload.summaryOverride = summaryOverride;
}
//add the payload to the message context
mc.setPayloadJSON(payload);
]]>
</script>

Finally you can use the call mediator by setting required or optional URL parameters to call the function appropriately.

Hope it was useful to you.

How to parse Complex data types or arrays through payload in a synapse template of a connector for WSO2 ESB?

Here I am going to focus on a REST API where we should be able to pass objects or arrays as a rest request in json. Let me start with a real word example as shown below.

API:                     Google Calendar
Function:             CalendarList: insert
Reference Link: https://developers.google.com/google-apps/calendar/v3/reference/calendarList/insert
Sample REST Request:

{
    "id": "836g6ropqm741sqsjl7abcde@group.calendar.google.com",
    "defaultReminders": [
        {
            "method": "email",
            "minutes": 60
        }
    ],
    "notificationSettings": {
        "notifications": [
            {
                "type": "eventCreation",
                "method": "email"
            },
            {
                "type": "eventResponse",
                "method": "email"
            }
        ]
    }
}

Note that the above highlighted values are complex types such as defaultReminders being a list and notificationSettings being an object. The proxy will accept only the first level parameters. So that the objects and arrays will be accepted in the proxy as one single parameter as shown below.

<proxy xmlns="http://ws.apache.org/ns/synapse"
       name="googlecalendar_createCalendarEntry"
       transports="https,http"
       statistics="disable"
       trace="disable"
       startOnLoad="true">
   <target>
      <inSequence>        
         <property name="id" expression="json-eval($.id)"/>       
         <property name="defaultReminders" expression="json-eval($.defaultReminders)"/>
         <property name="notificationSettings" expression="json-eval($.notificationSettings)"/>         
         <googlecalendar.createCalendarEntry>
            <id>{$ctx:id}</id>         
            <defaultReminders>{$ctx:defaultReminders}</defaultReminders>           
            <notificationSettings>{$ctx:notificationSettings}</notificationSettings>           
         </googlecalendar.createCalendarEntry>
         <respond/>
      </inSequence>
      <outSequence>
         <send/>
      </outSequence>
   </target>
   <description/>
</proxy>
              
Now the challenge comes on how we are going to add them to the payload factory inside the synapse template. Of course we have to use a script mediator because the above highlighted parameters are considered as optional parameters according to the API. Unlike handling string values, integers or even booleans, a valid json string does not accept "" surrounded within the values of complex types like arrays and objects but they should be sent within {} or []. 

To achieve the task, we must use eval function in JavaScript.

The eval() function evaluates or executes an argument.If the argument is an expression, eval() evaluates the expression. If the argument is one or more JavaScript statements, eval() executes the statements.
.
Note how it should be handled within a script mediator as shown below.

<script language="js">
        <![CDATA[      
               var defaultReminders = mc.getProperty('uri.var.defaultReminders');   
               var notificationSettings = mc.getProperty('uri.var.notificationSettings');
                            
var payload = mc.getPayloadJSON();
if ( defaultReminders != null && defaultReminders != "") { 
                 // if not null, add optional defaultReminders Array 
payload.defaultReminders = eval("("+defaultReminders+")");
}
if (notificationSettings != null  && notificationSettings != "") { 
                 // if not null , add optional notificationSettings JSON Obj
payload.notificationSettings = eval("("+notificationSettings+")");
}
mc.setPayloadJSON(payload);
]]>
</script>

Hope it was useful to you...

Sunday, April 6, 2014

Using Script mediators to build a url with optional parameters in a connector for WSO2 ESB

I thought to write another useful post that is related to connector development of WSO2 ESB. You may have seen many request URLs in APIs that mention a set of parameters. It's very important to identify the required and the optional parameters. In such cases we should know some efficient techniques on how to handle them inside the synapse template mediator.

Let me focus some real world REST API calls to explain these to you in this post. I use Linkedin API Reference for this post.

Handling request of a GET request with no URL parameters


Reference Link: 
https://developer.linkedin.com/documents/connections-api

Task:
Returns a list of 1st degree connections for a user who has granted access to his/her account.

E.g:
http://api.linkedin.com/v1/people/~/connections

If the request URL is as above where you see no any parameters mentioned but it  returns the authenticated user's list of first degree connection details, you can simply hard code the URL in the call mediator as below.

<template xmlns="http://ws.apache.org/ns/synapse" name="getConnections">
   <sequence>
      <call>
         <endpoint>
            <http method="get" uri-template="http://api.linkedin.com/v1/people/~/connections" />
         </endpoint>
      </call>
   </sequence>
</template>


Handling request of a GET method with required URL parameters


Reference Link: 
https://developer.linkedin.com/documents/connections-api

Task:
Returns the 1st degree connection's details with the specified id  for a user who has granted access to his/her account.

E.g:
http://api.linkedin.com/v1/people/id=12345/connections

Now we have a URL parameter that should be expected from the user through the proxy. It can be handled as below.

<template xmlns="http://ws.apache.org/ns/synapse" name="getConnections">
   <parameter name="id" description="Id of the member of whose connections are requested to be retrieved." />
   <sequence>
      <property name="uri.var.id" expression="$func:id" />
      <call>
         <endpoint>
            <http method="get" uri-template="http://api.linkedin.com/v1/people/id={uri.var.id}/connections" />
         </endpoint>
      </call>
   </sequence>
</template>


Handling request of a GET method with set of optional URL parameters


Reference Link: 

Task:
Returns the 1st degree connection's mentioned field details with the specified id  for a user who has granted access to his/her account.

E.g: 
http://api.linkedin.com/v1/people/~/connections:(headline,first-name,last-name)

This request is similar to retrieving  a list of 1st degree connections for a user who has granted access to his/her account. But this returns only the fields that are mentioned within the braces separated with commas.request will display only the headline,first-name and the last-name of the connections that are of 1st degree connections for the user who has granted access to his/her account. For an instance the above shown.

Let's assume the end users send the comma separated values to the above request as shown below through the proxy (assume it's an XML request),

<fieldSelectors>headline,first-name,last-name,id</fieldSelectors>

then the given URL can be constructed from the synapse template as shown below using a script mediator. Note that, I have mentioned how to construct it using javascript.

<template xmlns="http://ws.apache.org/ns/synapse" name="getConnections">
   <parameter name="fieldSelectors" description="Fields that are required tobe retrieved from the search result." />
   <sequence>
      <property name="uri.var.fieldSelectors" expression="$func:fieldSelectors" />
      <property name="uri.var.query" value="" />

      <script language="js"><![CDATA[
         //getting the comma separated values to a string variable
         var fieldSelectors = mc.getProperty('uri.var.fieldSelectors');

         //query variable is initially empty
         var query =mc.getProperty('uri.var.query');
   
         //if user has given some values to fieldSelectors variable through the proxy, then condition becomes true

         if (fieldSelectors != null && fieldSelectors != "") 
         {

          query=':'+'('+ mc.getProperty('uri.var.fieldSelectors')+')';

         }        
         mc.setProperty('uri.var.query', query);]]></script>

      <call>
         <endpoint>

            <!--since the fieldSelectors part of the URL is optional, it will  be set if an only f the user passes values to it. Else {uri.var.query} will be empty  -->
            <http method="get" uri-template="http://api.linkedin.com/v1/people/~/connections{uri.var.query}" />

         </endpoint>
      </call>
   </sequence>
</template>

Note: 

CDATA block within the script mediator must be used to avoid special characters to be encoded within the script mediator.

If you have any number of optional URL parameters in the request links, you can easily use script mediators as above in order to construct them as you wish. For better understanding see the below example as well where there are more number of parameters, how they should be concatenated with "&" symbol within the script mediator.

Reference Link: 

Task:
Covers all the URL GET requests of below shown calls of the connections API calls of Linkedin.

E.g: 

http://api.linkedin.com/v1/people/~/connections
http://api.linkedin.com/v1/people/id=12345/connections
http://api.linkedin.com/v1/people/~/connections:(headline,first-name,last-name)
http://api.linkedin.com/v1/people/url=http%3A%2F%2Fwww.linkedin.com%2Fin%2Flbeebe/connections

<template xmlns="http://ws.apache.org/ns/synapse" name="getConnections">

   <!--declarations of parameters -->
   <parameter name="id" description="Id of the member of whose connections are requested to be retrived." />
   <parameter name="publicProfileUrl" description="public url of the requested profile." />
   <parameter name="fieldSelectors" description="Fields that are required to be retrieved from the search result." />
   <parameter name="start" description="Start location within the result set for paginated returns." />
   <parameter name="count" description="The number of profiles to return.Values can range between 0 and 25." />
   <parameter name="modified" description="Values are updated or new." />
   <parameter name="modifiedSince" description="Value as a Unix time stamp of milliseconds since epoch." />

   <sequence>

      <!-- Setting the properties accepting the values passed through the proxy -->

      <property name="uri.var.id" expression="$func:id" />
      <property name="uri.var.publicProfileUrl" expression="$func:publicProfileUrl" />
      <property name="uri.var.fieldSelectors" expression="$func:fieldSelectors" />
      <property name="uri.var.start" expression="$func:start" />
      <property name="uri.var.count" expression="$func:count" />
      <property name="uri.var.modified" expression="$func:modified" />
      <property name="uri.var.modifiedSince" expression="$func:modifiedSince" />
      <property name="uri.var.query" value="" />

      <!-- Script Mediator block -->

      <script language="js"><![CDATA[//retrieving the values from the message context

         var apiUrl = 'http://api.linkedin.com'+'/v1/people/';
         var id = mc.getProperty('uri.var.id');
         var publicProfileUrl = 
                    mc.getProperty('uri.var.publicProfileUrl');               
         var fieldSelectors = mc.getProperty('uri.var.fieldSelectors');
         var query ='';
         var start = mc.getProperty('uri.var.start');
         var count = mc.getProperty('uri.var.count');
         var modified = mc.getProperty('uri.var.modified');
         var modifiedSince = mc.getProperty('uri.var.modifiedSince');

         if (id != null && id != "") 
         {
          query = apiUrl + 'id=' + mc.getProperty('uri.var.id') + '/connections';
         } 
         else if(publicProfileUrl != null && publicProfileUrl != "") 
         {
            //public profile url should be sent as a URL encoded string
            var encoded_publicProfileUrl =
                  encodeURIComponent(mc.getProperty('uri.var.publicProfileUrl'));                                               
            query = apiUrl + 'url=' + encoded_publicProfileUrl + '/connections';
         } 
          else 
          {
            query = apiUrl + '~/connections';
          }
      
          if (fieldSelectors != null && fieldSelectors != "") 
          {
            query = query + ':' + '(' + 
                          mc.getProperty('uri.var.fieldSelectors') + ')';        
          } 
  
          //since oauth2 access token should also be passed in every request url of Linkedin  
            query = query + '?oauth2_access_token=' + 
                          mc.getProperty('uri.var.accessToken');      
          //if 'start' optional param is provided append it to the URL with a '&' symbol
            if (start != null && start != "") 
            {
              query = query + '&start=' + mc.getProperty('uri.var.start');
            }
          //if 'count' optional param is provided append it to the URL with a '&' symbol
            if (count != null && count != "") 
            {
              query = query + '&count=' + mc.getProperty('uri.var.count');
            }
         //if 'modified' optional param is provided append it to the URL with a '&' symbol
            if (modified != null && modified != "") 
            {
              query = query + '&modified=' + 
                          mc.getProperty('uri.var.modified');      
            }
         //if 'modifiedSince' optional param is provided append it to the URL with a '&' symbol
            if (modifiedSince != null && modifiedSince != "") 
            {
              query = query + '&modified-since=' + 
                          mc.getProperty('uri.var.modifiedSince');
            }
  
         //set the query to message context's uri.var.query property 
              mc.setProperty('uri.var.query', query);]]></script>
      <call>
         <endpoint>
            <!--Since the entire URL is constructed above,it's just a matter of calling the endpoint as below. -->

            <http method="get" uri-template="{uri.var.query}" />

         </endpoint>
      </call>
   </sequence>
</template>



Hope it was useful to you. If so, please leave a comment.

Sunday, March 16, 2014

How to enable wire log in WSO2 ESB Server?

When working with WSO2 ESB server implementing connectors to connect to several APIs, one of the main problem I faced was the difficulty in monitoring the server while a request is sent to it from a connector. It was realized that things would be much easier if we can see how and what are the parameter going towards the ESB and what are coming from the ESB server while we make each request. 

Here comes the usage of enabling wire log in WSO2 ESB server. It's pretty simple but I hope the outcome will be so helpful to identify the behavior of the server.

Follow the simple steps. But before that make sure that you have the latest and properly running ESB in your local machine. If you want to know basic steps of running the WSO2 ESB, please follow the blog post here

If your ESB server is already running, make sure to follow the steps below and restart the server to apply the changes and monitor the enables wire log.

Enable the wire log as instructed below

Go to the extracted folder of the running instance of your WSO2 ESB server and move to the below shown location.

~\wso2esb-4.8.0\wso2esb-4.8.0\repository\conf

Now, open the log4j.properties file in some text editor like Notepad ++

find for the word "wire" and locate to the below shown lines of the file.



















Now, uncomment the highlighted two lines that are shown in the above image. Save and close the file. You are done.

As I have already mentioned, if you already have the server instance running, do restart it to apply the changes. 

How to restart your ESB Server?

Follow the coupe of screenshots below






















Hope it was useful to you. If so, please leave a comment.

Saturday, February 8, 2014

How to test Google Contacts functions through Google Contacts API using Postman?

In my previous post, I have explained you how you can install Postman- Rest- Client extension in your Google Chrome Web browser and why we need such tool. In the same post, I mentioned that I will be explaining it more in another post on how to use postman to test a real world API and make sure its functionalists are working as expected. So, here we go guys.

This post is based on testing Google Contacts API functions (not all but few of them) in Postman-Rest-Client tool.

Prerequisites

  • As u already know, we need to have Google Chrome running in your machine with the Postman extension installed in it.
  • You must have a Google Contacts account. Don't worry, if you do have a gmail account, you will be having a Google Contacts account as well. All you need to do is to login to it at least once using the following url.
  • Of course, you need to have Contacts in your account to start testing any function given in the api.
  • You need to refer the Google Contacts API documentation well before starting to test them. The below link will direct you to the latest version of it.
  • The most important thing is you need to obtain an authorization access token from Google Developers' OAuth 2.0 Playground. Follow the below link to obtain it and note i will explain you how to get the access token step by step.

Great. Now, let's go from point zero :) 

I assume everyone has a gmail account so that a Google Contacts account too. So, let me explain you how to obtain an access token from Google Developers' OAuth 2.0 playground.

Obtaining an Autherization access token from Google Developers' OAuth 2.0 playground for Google Contacts


1.Go to the link below


2.



3.



4.




5.




6.





Note that the obtained access token expires within the time that is mentioned above so that you have to use that for testing before it expires. But once it's expired, you can refresh and obtain  a new access token to continue testing.

So, I hope it's clear up to this level. Next is simply to get a function, identify its parameters etc and test for it's response through postman-Rest client tool.

Before that, we have to confirm the function that we are going to test, don't we? Refer the below link and focus on the function that is highlighted in the below screenshot that is taken from the Google Contacts API documentation page.





















Check the below shown screenshot to identify the function correctly.
















How to test the Retrieve All contacts function in postman?


Open Postman-Rest-API plugin in your Google Chrome Web Browser and Click Reset button to clear any existing tests that you have been working with. Then set the below values in front of the below shown fields accordingly.

Set the url in the below format:-
https://www.google.com/m8/feeds/contacts/<your email address>/full

Add a Header call Authorization and add the value as shown below (note that your access token shouldn't be expired at the time you are using it to test):-

Bearer <the access token you obtained above>

Select the type of the http call as :-

GET

and finally click Send button to obtain the response.

for an example, refer the below screenshot




















If you didn't enter an email in the request url, then the default email will be used which means the account that you have currently logged in.

I hope the post was clear enough for you to start with Postman to test API functions. Hope it was useful to you :) See you in another interesting post ...