ewd-feder8 Installation & Reference Guide

Transcription

1 ewd-feder8 Installation & Reference Guide Version 1 25 July M/Gateway Developments Ltd ewd-feder8 Page 1

2 What is ewd-feder8? ewd-feder8 is a Node.js module that allows you to create a customisable platform for federated access across and between a set of HTTP/REST servers. ewd-feder8 is built on top of the ewd-xpress module which, in turn, is an Express-based platform that makes use of the ewd-qoper8 master/worker architecture with either the GT.M or Cache databases providing local persistence and session management within the federating platform. This architecture makes ewd-feder8 highly scalable. It is also highly customisable, for example: incoming HTTP requests can be re-routed to one or more of the federated Web Service/REST servers. Routing to more than one servers simultaneously is achieved by defining groups of servers. You may define as many groups as you like, and groups can overlap. responses from a group of servers are automatically aggregated and, when the last one is received, the aggregated response is returned to the original client. you can optionally customise ewd-feder8 by adding your own Node.js module that allows you to intercept and modify/manipulate not only incoming requests to ewd-feder8, but also incoming responses from the back-end federated servers. Installing ewd-feder8 ewd-feder8 is installed using: npm install ewd-feder8 Configuring and Running ewd-feder8 ewd-feder8 extends the ewd-xpress module, so you need to create a startup file that defines how you want to configure ewd-xpress and starts it up. Within the startup file you also define: - aliases and configuration details for the remote HTTP/REST servers that you want to federate - groupings of remote servers - the name/path for your customisation module, if applicable You'll find an example in the ewd-feder8 /examples directory (look for the file ewd-feder8.js) You then simply run your startup file, eg: node mystartupfile If you are using Caché as a database with ewd-feder8 running on a Linux platform, depending on the permissions granted when you installed Caché, you may need to invoke the startup file as sudo, e.g.: sudo node mystartupfile 2016 M/Gateway Developments Ltd ewd-feder8 Page 2

3 Example ewd-feder8 startup file Here's a simple example: var config = { managementpassword: 'keepthissecret!', servername: 'EWD Feder8 Platform', port: 8082, poolsize: 1, database: { type: 'cache' ; var feder8_params = { server: { placeholder: { host: ' myewd1: { host: ' myewd2: { host: ' destination: { dest1: ['myewd1', 'myewd2'] ; var feder8 = require('ewd-feder8'); feder8.start(config, feder8_params); The config object at the start of this example sets up the configuration for ewd-xpress: - it will listen on port one ewd-qoper8 worker will be used - the worker will connect to the Caché database - the management password can be used to allow access from the ewd-xpress-monitor application. The feder8-params object defines the servers that we want to federate. Each has a name or alias (eg 'placeholder', 'myewd1') and defines its base URL. The destination sub-object defines destination groups, each of which is an array of server names. If the example above is saved as /ewd3/my-feder8.js and started using: cd /ewd3 sudo node my-feder8 Then you should see the following: 2016 M/Gateway Developments Ltd ewd-feder8 Page 3

4 sudo node feder8-test [sudo] password for robtweed: webserverrootpath = /home/robtweed/ewd3/www/ Worker Bootstrap Module file written to node_modules/ewd-qoper8-worker.js ======================================================== ewd-qoper8 is up and running. Max worker pool size: 1 ======================================================== ewd-feder8 is now ready for use. To test it we should use an HTTP/REST client such as Chrome Advanced REST Client or Postman. Simple Use of ewd-feder8 as a proxy At its most simple, ewd-feder8 can be used as a proxy for another HTTP/REST server. The example startup file allows us to demonstrate this - take the "placeholder" server, which was defined/configured as follows in the startup file: placeholder: { host: ' This is a generally-available public server that is useful for REST testing. Try accessing it directly via your REST client: URL: Method: GET Content-type: application/json You should get a response similar to this: { "userid": 1, "id": 1, "title": "sunt aut facere repellat provident occaecati excepturi optio reprehenderit", "body": "quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto" Now try accessing it indirectly via ewd-feder8: URL: Method: GET Content-type: application/json Note: Change the IP address and port to match that of your ewd-feder8 You should get the same response, but you'll see activity being logged in your ewd-feder8 console window, e.g.: 2016 M/Gateway Developments Ltd ewd-feder8 Page 4

5 no available workers sent qoper8-start message to 9538 process.argv[2] = ewd-xpress.worker workermodule: ewd-xpress; worker Session Garbage Collector has started in worker 9538 master process received response from worker 9538: {"type":"workerprocessstarted","ok":9538 new worker 9538 started and ready so process queue again checking if more on queue worker 9538 received message: {"type":"ewd-qoper8-express","path":"/placeholder/posts/ 1","method":"GET","headers":{"host":" :8082","content-type":"application/ json","params":{"0":"posts/1","destination":"placeholder","query":{,"body":{,"application":"ewdfeder8/lib/worker","expresstype":"ewd-feder8" handler module loaded for ewd-feder8/lib/worker master process received response from worker 9538: {"type":"ewd-qoper8- express","finished":true,"message":{"userid":1,"id":1,"title":"sunt aut facere repellat provident occaecati excepturi optio reprehenderit","body":"quia et suscipit\nsuscipit recusandae consequuntur expedita et cum\nreprehenderit molestiae ut ut quas totam\nnostrum rerum est autem sunt rem eveniet architecto","restmessage":true,"type":"ewd-feder8" Master process has finished processing response from worker process 9538 which is back in available pool If we analyse what happened: - We've pointed the REST client at our instance of ewd-feder8 via the URL : the URL path /placeholder tells ewd-feder8 to forward the request to the server with the alias 'placeholder' (ie - anything following this in the path is appended on the URL for the 'placeholder' server, so in our example the forwarded URL is: The forwarding is carried out by the ewd-qoper8 worker process, and when it receives the response from the remote server, it is returned to the ewd-qoper8 master process which, in turn, returns it to our REST client. Simple Federated Access to a Group of Servers A more useful use of ewd-feder8 is to forward a copy of an incoming request to a group of HTTP/ REST servers and pool the responses into a single aggregated response object. In order to test this, you'd need to set up a couple of ewd-xpress servers which will handle a web service (you could actually use a group of any web service endpoints that return the same or similar responses). Setting up such an ewd-xpress server is described here: We can define a very simple web service using the following module: 2016 M/Gateway Developments Ltd ewd-feder8 Page 5

6 module.exports = { ; restmodule: true, handlers: { test1: function(messageobj, finished) { console.log('*** test messageobj: ' + JSON.stringify(messageObj)); finished({ test: 'finished ok', pid: process.pid ); If we save this module into the /ewd3/node_modules directory and name it mywebservice.js, then this web service can be invoked by a URL path of /mywebservice/test1 Looking back at the example startup file, it defined the following ewd-xpress servers: myewd1: { host: ' myewd2: { host: ' It also defined a group (named 'dest1') containing these servers: destination: { dest1: ['myewd1', 'myewd2'] If both of the ewd-xpress servers used our simple test web service, we can get our ewd-feder8 server to forward a request for it to both of the ewd-xpress servers by sending the following request: URL: Method: GET Content-type: application/json If your ewd-xpress servers have been correctly set up and running on the correct IP addresses defined in your ewd-feder8 startup file, you should get back the following response: { "myewd1": { "test": "finished ok", "pid": 7741, "myewd2": { "test": "finished ok", "pid": M/Gateway Developments Ltd ewd-feder8 Page 6

7 This is a composite response object - each server in the group is identified by its alias (myewd1, myewd2) with its respective response defined as a sub-object below the alias name. So, provided each of the federated servers need to perform the same task, ewd-feder8 makes it very simple and straightforward to create a federated proxy web service. Customising ewd-feder8 If your requirements are more complex, ewd-feder8 provides a simple but powerful mechanism for customising its behaviour: you can intercept: - the incoming request from the REST/HTTP client; and/or - responses returned from the federated servers Request Interception If you intercept the incoming request, you can do things such as augmenting or modifying its payload and/or headers, and having done so you can either: - let it be forwarded to each server in the group defined by the incoming request path - forward the request to a server or group of your choice - i.e. you can programmatically over-ride the federation group mapping Response Interception If you intercept responses returned by a server, you can programmatically: - modify its contents - allow it to be added to the aggregated response; or - forward a new request (probably based on the response contents) to another federated server or group So by using response interception, you could create a complex "dance" of responses and requests between your ewd-feder8 server and your federated endpoint servers, building up a complex response. Furthermore, by making use of the persistence provided by the database that you've configured in your ewd-qoper8 server (ie GT.M or Caché), you could cache intermediate results and build a complex composite response. Adding Interceptions In order to intercept requests and responses, you must create an ewd-feder8 extension module which contains your intercept logic. This is a standard Node.js module which you'd normally save in your server's node_modules directory (eg /ewd3/node_modules). You can name it whatever you like, but you instruct ewd-feder8 to load it within the startup file. So we could do this in our example by adding the line shown: 2016 M/Gateway Developments Ltd ewd-feder8 Page 7

8 var config = { managementpassword: 'keepthissecret!', servername: 'EWD Feder8 Platform', port: 8082, poolsize: 1, database: { type: 'cache' ; var feder8_params = { server: { placeholder: { host: ' myewd1: { host: ' myewd2: { host: ' destination: { dest1: ['myewd1', 'myewd2'], extensionmodule: 'myextensionmodule' ; var feder8 = require('ewd-feder8'); feder8.start(config, feder8_params); You can, in fact, load the extension module from any directory you wish by specifying a full path, e.g: extensionmodule: 'ewd-feder8/examples/myfeder8extension' An ewd-feder8 extension module should export a function that contains any or all of the following event handlers: - this.on('feder8-request', function(params) {..; and/or - this.on('feder8-response', function(params) {..; and/or - this.on('feder8-combined-response', function(params) {..; Handling the feder8-request event The feder8-request event is emitted whenever an HTTP request is sent to your ewd-feder8 server. Your handler's callback function has a single argument, params, with the properties: - messageobj: the incoming request message object. This is in the standard ewd-xpress REST format described in - destination: the server or group destination as defined in the first element of the request URL - forward.tohost: a function that allows you to forward a message to a specified server - forward.togroup: a function that allows you to forward a message to a specified group destination 2016 M/Gateway Developments Ltd ewd-feder8 Page 8

9 - finished: the standard ewd-qoper8 finished() function which you must pass as an argument to forward.tohost or forward.togroup Here's our example's messageobj contents: { "type": "ewd-qoper8-express", "path": "/dest1/mywebservice/test1", "method": "GET", "headers": { "host": " :8082", "content-type": "application/json", "params": { "0": "mywebservice/test1", "destination": "dest1", "query": {, "body": {, "application": "ewd-feder8/lib/worker", "expresstype": "ewd-feder8" Typically you'll be interested in the method, headers, query and/or body properties within this object. Here's a simple do-nothing request intercept example: module.exports = function () { ; this.on('feder8-request', function(params) { console.log('**** feder8-request event fired in extension module!'); console.log('*** message: ' + JSON.stringify(params.messageObj)); ); Here we'll return a response to the REST client: module.exports = function () { this.on('feder8-request', function(params) { params.messageobj.responsesent = true; params.finished({ foo: 'bar', response: 'from request intercept' ); ); ; Note the way we have to set the responsesent flag as a property of the params.messageobj object - this tells ewd-feder8 to stop any further processing of the request M/Gateway Developments Ltd ewd-feder8 Page 9

10 Here we'll forward the request to the server named EWD1: module.exports = function () { this.on('feder8-request', function(params) { params.forward.tohost('myewd1', params.messageobj, params.finished); ); ; forward.tohost() automatically instructs ewd-feder8 to stop any further processing of the request. The arguments of the forward.tohost() function are: - destination: the alias of the federated server to which you want to send the message - messageobj: the message object to send to the destination server - finished: the ewd-qoper8 finished() function for returning the server's response to the ewdfeder8 master process. forward.togroup() should be used if you want to forward your message to a group of federated servers. Its arguments are: - destination: the name of the server group to which you want to send the message - messageobj: the message object to send to the destination server - finished: the ewd-qoper8 finished() function for returning the server's response to the ewdfeder8 master process. - forward: you should add the params.forward object as this argument, to allow the message to be forwarded to each member of the group. Note: within your feder8-request event handler, you can get access to the database by using either: this.documentstore to use the Document Store APIs this.db to use the raw cache.node / NodeM APIs Handling the feder8-response event The feder8-response event is emitted whenever a response is received from a federated server. Your handler's callback function has a single argument, params, with the properties: - responseobj: the response JSON object received from the federated server; - messageobj: the original message that was sent to the server, to allow your event handler to understand the context for the event - destination: the server or group destination as defined in the first element of the request URL - error: if an error occurred, this object will be defined and contain the details. - body: the main body of the response received from the federated server - forward.tohost: a function that allows you to forward a message to a specified server - forward.togroup: a function that allows you to forward a message to a specified group destination 2016 M/Gateway Developments Ltd ewd-feder8 Page 10

11 - finished: the standard ewd-qoper8 finished() function which you must pass as an argument to forward.tohost or forward.togroup, or use to return your final response to the REST client. Here we've added a simple do-nothing response intercept event handler to our ewd-feder8 extension module: module.exports = function () { ; this.on('feder8-request', function(params) { console.log('**** feder8-request event fired in extension module!'); console.log('*** message: ' + JSON.stringify(params.messageObj)); ); this.on('feder8-response, function(params) { console.log('**** on response fired in extension module!'); console.log('*** response was ' + JSON.stringify(params.responseObj)); ); ewd-feder8 will continue handling the response in the above example. Here we'll return our own customised response to the REST client and instruct ewd-feder8 to stop further processing of the response: module.exports = function () { ; this.on('feder8-request', function(params) { console.log('**** feder8-request event fired in extension module!'); console.log('*** message: ' + JSON.stringify(params.messageObj)); ); this.on('feder8-response, function(params) { console.log('**** on response fired in extension module!'); console.log('*** response was ' + JSON.stringify(params.responseObj)); params.responseobj.responsesent = true; params.finished({hello: 'world!'); ); Alternatively, you can use the params.forward.tohost() and params.forward.togroup() functions to send a new message back out to one or more of your federated servers. The mechanism for using these is identical to that used in your feder8-request handler (see above). Using either of these functions automatically instructs ewd-feder8 to stop further processing of the response. Note that your feder8-response handler will fire on receipt of any and all received responses from federated servers. You'll probably mainly use the params.destination property to determine how to handle each incoming response - this property will tell you which server returned the response. Of course you may also need to inspect the response object's contents in order to determine how to handle each incoming response M/Gateway Developments Ltd ewd-feder8 Page 11

12 Your feder8-response handler function also has access to the database via this.db and this.documentstore, allowing you to create temporary cached data to build up a complex composite response resulting from multiple round-trips to federated servers. Handling the feder8-combined-response event The feder8-combined-response event is emitted when all the responses have been received from a destination group of servers, just prior to the combined response being returned to the client. You can therefore write a handler function for this event to further customise the combined response object, and/or perform other tasks within ewd-feder8. Your handler's callback function has a single argument, params, with the properties: - responseobj: the JSON object containing the composite/combined response from all servers in the destination group; - messageobj: the original message that was sent to the server, to allow your event handler to understand the context for the event - destination: the name of the group destination, as defined in the first element of the request URL - error: error string notifying you of the last undefined server in the group (if any were undefined) - forward.tohost: a function that allows you to forward a message to a specified server - forward.togroup: a function that allows you to forward a message to a specified group destination - finished: the standard ewd-qoper8 finished() function which you must pass as an argument to forward.tohost or forward.togroup, or use to return your final response to the REST client. Here we've added a simple do-nothing feder8-combined-response intercept event handler to our ewd-feder8 extension module: module.exports = function () { ; this.on('feder8-request', function(params) { console.log('**** feder8-request event fired in extension module!'); console.log('*** message: ' + JSON.stringify(params.messageObj)); ); this.on('feder8-response, function(params) { console.log('**** on response fired in extension module!'); console.log('*** response was ' + JSON.stringify(params.responseObj)); ); this.on('feder8-combined-response', function(params) { console.log('**** combined-response has been created'); console.log('*** params = ' + JSON.stringify(params)); ); 2016 M/Gateway Developments Ltd ewd-feder8 Page 12

13 2016 M/Gateway Developments Ltd ewd-feder8 Page 13