Market Data Platform Real Time. REAL TIME DATA Corporate Bond Market

Size: px

Start display at page:

Download "Market Data Platform Real Time. REAL TIME DATA Corporate Bond Market"

Jean Bailey
5 years ago
Views:

1 TECHNICAL DOCUMENT Market Data Platfrm Real Time REAL TIME DATA Crprate Bnd Market (STANDARD PRODUCT) Versin OCT 2015 DOTEX INTERNATIONAL LIMITED EXCHANGE PLAZA, PLOT NO. C/1, G BLOCK, BANDRA-KURLA COMPLEX, BANDRA (E), MUMBAI INDIA. Page 1 f 16

3 Revisin Histry Name Descriptin Date Versin 1.0 Final Specificatin Issued August 28, 2015 Versin 1.1 Trade structure revised Octber 12, 2015 Page 3 f 16

4 Cntents 1. Intrductin Cnnectin Details Structural diagram Online Requirements Platfrm ntes Prduct Overview Sessin Initializatin Lgin Request (Sent by client) Lgin Respnse (Sent by server) Heartbeat Message (Sent by server) End f Feed Message (Sent by server) Online Trade Infrmatin Steps fr decmpressing the data packets LZO Algrithm Details LZO Algrithm Details Decmpressin steps Checksum Calculatin Algrithm Cntact Infrmatin Page 4 f 16

5 CORPORATE BOND MARKET REAL TIME DATA (STANDARD PRODUCT) 1. Intrductin DtEx Internatinal Ltd. disseminates NSEIL s Real Time Bradcast data t varius infrmatin agencies. It prvides the 3 different types f data t Inf Vendrs, i.e. Real Time Data, Snapsht Data and End f Day Data. The Real Time Data is a packet bradcast available in TCP/IP packet frmat, whereas the Snapsht Data and End f day data are available in the frm f files. Certain prducts based n the Real Time Data are als made available thrugh files. The infrmatin agencies cnnect t the MDP-RT Server thrugh Leased Lines. These leased lines are terminated n MDP-RT Ruter and their data specific pneumnic calls are frwarded t MDP-RT server. The MDP-RT server accepts these pneumnic calls and creates a scket cnnectin. The TCP/IP data flws t the infrmatin agencies thrugh these scket cnnectins. 2. Cnnectin Details The Inf Vendrs cnnect t Real Time server ver Leased Lines using the prprietary prtcl specified in this dcument. In NSE premises tw Real Time Prductin Servers perate in an active-active cnfiguratin. Each server can be accessed using their IP addresses as shwn in the Structural Diagram. In case a server becmes inaccessible thrugh bth its IP addresses, the Inf Vendr sftware requires t fail-ver t the ther server. 2.1 Structural diagram NSE Trading System MTNL / Other Service Prviders Netwrk Leased Lines TCP/IP Feed Inf Vendrs IP Netwrk Channelized E1 Frm tw MTNL exchanges Read (UDP) TCP/IP Ruter Write (TCP/IP) MDP-RT Server 2 MDP-RT Server 1 Page 5 f 16

6 2.2 Online Requirements a) A Ruter / Switch r a card with TCP/IP capabilities t cnnect t 2 Mbps transmissin lines fr receiving NSEIL s Real time infrmatin. b) The Infrmatin agency shuld develp applicatins that initiate TCP/IP calls thrugh 2 Mbps Leased Line. c) Infrmatin agency can cnnect t the MDP-RT servers thrugh the internet als. Fr IP validatin at applicatin level, infrmatin agencies has t prvide the public static IP frm which they will cnnect t MDP-RT servers. Cnnectivity thrugh internet is available fr sme prducts nly. 2.3 Platfrm ntes 1. The service can be simultaneusly accessed thrugh IP addresses n each server. This is t enable Inf Vendrs t access the servers in case f failure. 2. There may be slight differences between the data disseminated by the tw servers because f differences in ruting f data, etc. 3. The feed cnsist f series f sequenced and un-sequenced variable length cmpressed messages. 4. The cmpressin algrithm used ver here is LZO Cmpressin. 5. Fllwing are sizes f Data Types referred thrughut this dcument: Data Type Size In Bytes CHAR 1 INT 4 LONG 4 DOUBLE 8 6. Byte rder (Endian-ness): All values are Little Endian. 7. Byte alignment (Structure Packing): All structures are aligned t 1 byte bundary. 3. Prduct Overview This dcument explains abut the NSE CBRICS Feed prduct. Thrugh this prduct, infrmatin NSE s CBRICS (Crprate Bnd Reprting Platfrm - infrmatin is disseminated n real time basis. 4. Sessin Initializatin The Feed is built n TCP/IP scket cnnectin. This feed cnsists f sequenced and un-sequenced messages. Un-sequenced messages prvides the lgin and cnnectin related messages such as lgin and heartbeats messages. Un-sequenced messages are nt part f the data. The sequenced data cntains the actual market data and are reliable and recverable. A sessin begins with client establishing a TCP cnnectin and sending the lgin request packet. Once the lgin request received the server authenticate it and send the lgin respnse. If the lgin is successful Page 6 f 16

7 server will begin t send the sequenced data, r reject the lgin and terminate the cnnectin. Data packet cnsist f sequence number as ne field. The first sequenced message f the day will send the sequence number as 1 and after that it will be incremented by 1 fr each sequenced message. 5. Packet Frmat The data is sent ut ver the TCP/IP stream in batches. Each batch received n the TCP stream is rganized as fllws: 1. Cmpressin Header: Als called as Cmpressin Header, this sectin f the packet describes whether the data packet is cmpressed, hw many messages it carries, etc. 2. Data Paylad: This cntains the actual data messages r recrds. The frmat is diagrammatically represented as fllws: Cmpressed Header 1. Cmpressed/ Uncmpressed = 0 then cmpressed/ 1 uncmpressed 2. Number f packets = Number f recrds in cmpressed data 3. Data Size = Cmpressed data size As the data packets are sent in cmpressed frmat there is a need t decmpress them. The cmpressin algrithm used is LZO. Page 7 f 16

8 All the packets received frm server cnsist f Cmpressin Header. Cmpressin Header indicates whether the data is cmpressed r nt. It als specifies the number f packets in the batch and the size f data paylad. In case the data is cmpressed, the client is required t decmpress the data paylad using LZO decmpressin algrithm. Server sends all the packets in fllwing frmat: typedef struct CHAR ccmpornt; SHORT ndatasize; SHORT inofpackets; }ST_COMP_BATCH_HEADER typedef struct SHORT icde; SHORT ilen; LONG lseqn; }ST_INFO_HEADER; typedef struct.. }ST_DATA_INFO; typedef struct SHORT ichecksum; CHAR ceot; }ST_INFO_TRAILER; typedef struct ST_INFO_HEADER stinfhdr; ST_DATA_INFO stdatainf; ST_INFO_TRAILER stinftrailer;. }ST_DATA_PACKET All the packets received frm server cnsist f cmpress batch header. Cmpress batch header gives the infrmatin abut the data packet cmpressed r nt, number f packets in the fllwing data packet and the ttal size f data packet. Client needs t decmpress the data packet using LZO decmpressin algrithm. After decmpressin each data packet cnsists f ST_INFO_HEADER, which has the icde field t identify the type f the packet. Using icde field, data inf packet is mapped t the respective data packet. Page 8 f 16

9 6. Sessin Messages Sessin messages are nt cnsidered as market data messages. These messages prvide the cnnectin and lgin related messages such as lgin, and heartbeat messages. 6.1 Lgin Request (Sent by client) Lgin request packet is sent by the client immediately after cnnecting t the server. This packet desn t cntain the cmpress batch header. If the client wants t change his default passwrd then he needs t send New Passwrd and Cnfirm Passwrd in the request therwise it shuld be kept blank. Passwrd is case sensitive. Field Name Data Type Value Remark INFO HEADER Cde SHORT CQ Length SHORT Numeric Size f (INFO HEADER + INFO DATA + INFO TRAILER) Sequence LONG Numeric 0 (Zer) Number INFO DATA User Id CHAR[10] Alphanumeric Exchange prvided user id (Null terminated) Passwrd CHAR[8] Alphanumeric Passwrd (Null terminated) New Passwrd CHAR[8] Alphanumeric New passwrd (Null terminated) Cnfirm Passwrd INFO TRAILER CHAR[8] Alphanumeric Cnfirm passwrd (Null terminated) Check Sum SHORT Numeric Refer Sectin n Checksum Calculatin Algrithm End Of Trailer CHAR \r Carriage Return 6.2 Lgin Respnse (Sent by server) Lgin respnse packet will be sent by server after receiving the lgin request packet. This packet des cntain the cmpress batch header. This packet cntains the errr cde frm which the client can identify the status f the lgin. Field Name Data Type Value Remark INFO HEADER Cde SHORT CR Page 9 f 16

10 Length SHORT Numeric Size f (INFO HEADER + INFO DATA + INFO TRAILER) Sequence Number LONG Numeric 0 (Zer) INFO DATA Errr Cde LONG Successful Lgin Passwrd Update Successfully Wrng UserId- Passwrd Cmbinatin Passwrd is nt valid in passwrd change request Lgin request is nt crrect. Errr cde ther than abve - Errr in receiving lgn respnse Errr Message CHAR[50] Descriptin abut the errr cde INFO TRAILER Check Sum SHORT Numeric Refer Sectin n Checksum Calculatin Algrithm End Of Trailer CHAR \r Carriage Return 6.3 Heartbeat Message (Sent by server) Heartbeat message will be sent if data is nt available. Field Name Data Type Value Remark INFO HEADER Cde SHORT CH Length SHORT Numeric Size f (INFO HEADER + INFO DATA + INFO TRAILER) Sequence Number INFO DATA Nt assciated with any data LONG Numeric 0 (Zer) INFO TRAILER Check Sum SHORT Numeric 0 (Zer) End Of Trailer CHAR \r Carriage Return Page 10 f 16

11 6.4 End f Feed Message (Sent by server) End f Feed message will be sent t indicate feed terminatin. Field Name Data Type Value Remark INFO HEADER Cde SHORT CE Length SHORT Numeric Size f (INFO HEADER + INFO DATA + INFO TRAILER) Sequence Number INFO DATA Nt assciated with any data INFO TRAILER LONG Numeric 0 (Zer) Check Sum SHORT Numeric 0 (Zer) End Of Trailer CHAR \r Carriage Return Page 11 f 16

12 7. Sequenced Data Messages (Sent by server) Sequenced data messages will be sent by server and will cntain the actual market data. These messages have a sequence number assigned fr each data message 7.1 Online Trade Infrmatin NSE-CBRICS Trade infrmatin is sent thrugh this message. Field Name Data Type Value Remark INFO HEADER Cde SHORT CX Length SHORT Numeric Size f (INFO HEADER + INFO DATA + INFO TRAILER) Sequence Number LONG Numeric Applicatin sequence number INFO DATA Time Stamp CHAR[11] Character N f secnds frm :00:00 (DD-MM-YYYY HH:MM:SS) Message Cde CHAR[1] Character L OTC listed Bnds U OTC unlisted Bnds ISIN CHAR[12] Character ISIN Cde f the Bnd Descriptr CHAR[128] Character Descriptr f the Bnd Weighted Average Price (Rupees) Weighted Average Yield (YTM %) CHAR[24] Character Weighted Average Price in Rupees CHAR[24] Character Weighted Average Yield (YTM) - Percent N. f Trades CHAR[24] Character N. f Trades Ttal Trade Value (Rupees in Lacs) Last Trade Price (Rupees) Last Trade Yield (YTM, Annualized, %) INFO TRAILER CHAR[24] Character Ttal Trade Value in Rupees in Lacs CHAR[24] Character Last Trade Price in Rupees CHAR[24] Character Last Trade Yield (YTM, Annualized) - Percent Check Sum SHORT Numeric Refer Sectin n Checksum Calculatin Algrithm End Of Trailer CHAR \r Carriage Return Page 12 f 16

13 8. Steps fr decmpressing the data packets 8.1 LZO Algrithm Details LZO is a data cmpressin library which is suitable fr data de- /cmpressin in real-time. This means it favrs speed ver cmpressin rati. LZO is written in ANSI C. Bth the surce cde and the cmpressed data frmat are designed t be prtable acrss platfrms. LZO implements a number f algrithms with the fllwing feature Decmpressin is simple and *very* fast. Requires n memry fr decmpressin. Requires 64 KB f memry fr cmpressin. Allws yu t dial up extra cmpressin at a speed cst in the cmpressr. The speed f the decmpressin is nt reduced. Includes cmpressin levels fr generating pre-cmpressed data which achieve a quite cmpetitive cmpressin rati. There is als a cmpressin level which needs nly 8 KB fr Cmpressin. Algrithm is thread safe. Algrithm is lssless. LZO supprts verlapping cmpressin and in-place decmpressin. 8.2 LZO Algrithm Details Include files, surce files (src) prvided by LZO LZO.lib LZO library versin used is Decmpressin steps Receive the packet in the temprary buffer i.e. array f characters. The first field is cmpressed r nt cmpresses? The secnd field is the number f packet in the fllwing data packet. The third field is data packet length. Use the fllwing functin f LZO t Decmpress. r = lz1z_decmpress ((lz_byte*)cinputbuf, iplength, (lz_byte*)coutputbuf, (lz_uint*)&plength, NULL); lz1z_decmpress: Functin which decmpresses the data packet received cinputbuf: Input buffer in which cmpressed data is received iplength: The length f the packet which applicatin has received using Receive (). coutputbuf: The uncmpressed utput data which is result f decmpressin. Page 13 f 16

14 plength: Length f uncmpressed data After decmpressin data will be available in Output Buffer. Each utput data packet cntains the INFO HEADER, after mapping the utput decmpressed buffer t INFO HEADER find ut the data packet and the accrding t it map the utput buffer t respective data packet. Algrithm: ST_INFO_HEADER *pstinfheader = NULL; fr (i=0; i < inofpackets; i++) // inofpackets received in // Cmpressed data header pstinfheader = (ST_INFO_HEADER*) coutputbuf; switch (pstinfheader->icde) case CX: //Indices Infrmatin ST_INDEX_DATA* stindexdata = (ST_INDEX_DATA*)cOutputBuf;.. coutputbuf = coutputbuf + sizef(st_index_data); break; } } } Page 14 f 16

15 9. Checksum Calculatin Algrithm The Checksum rutine fllwed fr Inf Vendr Feed is as fllws: // Fllwing are the defines fr checksum calculatin #define DC1 17 #define DC3 19 #define CR 13 #define LF 10 #define POLY 0x1021 // End f defines unsigned check_sum (cdata, ilength) char *cdata ; int ilength; unsigned uaccum = 0; unsigned udata; unsigned char ucchk[2]; int i,j; fr (i=0;i<ilength;i++) udata = *(cdata+i); udata <<= 8; fr(j=8; j>0 ;j--) if((udata^uaccum)&0x8000) uaccum=(uaccum<<1)^poly; /* SHIFT AND SUBTRACT POLY */ else uaccum<<=1; udata<<=1; } } ucchk[0] = uaccum>>8; if (ucchk[0] == DC1 ucchk[0] == DC3 ucchk[0] == CR ucchk[0] == LF ) ucchk[0] -= 1; ucchk[1] = uaccum&0xff; if (ucchk[1] == DC1 ucchk[1] == DC3 ucchk[1] == CR ucchk[1] == LF ) ucchk[1] -= 1; uaccum = ucchk[1]; uaccum = (uaccum<<8) + ucchk[0]; } return(uaccum); Page 15 f 16

16 10. Cntact Infrmatin Name Cntact Number DOTEX Business Technical Supprt Page 16 f 16

NSE - MARKET FEED (FO LEVEL - 1)

NSE - MARKET FEED (FO LEVEL - 1) Version: 1.1 Date: April 25, 2013 DOTEX INTERNATIONAL LIMITED EXCHANGE PLAZA, PLOT NO. C/1, G BLOCK, BANDRA-KURLA COMPLEX, BANDRA (E), MUMBAI 400 051. INDIA. 2009 National