YumaPro API Quick Start Guide

Transcription

1 YANG-Based Unified Modular Automation Tools Common API Guide Version

2 Table of Contents 1 Preface Legal Statements Additional Resources WEB Sites Mailing Lists Conventions Used in this Document.6 2 YumaPro API Overview API Overview Terminology9 3 YANG Data Examples Object Tree Ordering YANG Object Tree Handling Examples Find Top-Level Object Access Module Pointer Match Any Object Check For Child Nodes Find a Specific Child Node Find Choice or Case Child Node Get First Child Get Next Child Process All Child Nodes Process Ancestor Nodes YANG Data Tree Handling Examples Leaf Examples Leaf-list Examples Container Examples List Examples Database Access Common CLI Parameters Startup Options Database Deployment Variants Local internal database Local internal database + NV-store Hook Local internal database + no-nv-store Local internal database + External Edits DB-API Example32 5 Database Transaction Callbacks EDIT1 and EDIT2 Callbacks Error Handling for Edit Transactions EDIT1 Callback Initialization and Cleanup EDIT1 Callback Function Example EDIT2 Callback Initialization and Cleanup EDIT2 Callback Function Example EDIT2 Utility Functions GET1 Callback GET1 Callback Examples. Virtual data GET1 Callback Examples for Static Data GET2 Callback GET2 Callback Initialization and Cleanup..72 Version Page 2

3 5.3.2 GET2 Callback Function Examples GET2 Access Macros GET2 Access Functions.91 6 In-Transaction Callbacks Callback Invocation Order Set Hook Callback Set Hook Callback Initialization and Cleanup Set Hook Callback Function Examples Transaction Hook Callback Transaction Hook Callback Initialization and Cleanup Transaction Hook Callback Function Example Transaction Start Callback Transaction Start Callback Initialization and Cleanup Transaction Start Callback Function Examples Transaction Complete Callback Transaction Complete Callback Initialization and Cleanup Transaction Complete Callback Function Example Set Order Hook Callback Set Order Hook Callback Initialization and Cleanup Set Order Hook Callback Function Examples Add Edit API Get Data API Validate Complete Callback Validate Complete Callback Initialization and Cleanup Apply Complete Callback Apply Complete Callback Initialization and Cleanup Commit Complete Callback Commit Complete Callback Initialization and Cleanup Commit Completeness Callback Function Example System Callbacks Candidate Reload Callback Candidate Reload Callback Example Module Load Callback Module Load Callback Initialization and Cleanup Module Load Callback Function Example Module Unload Callback Module Unload Callback Initialization and Cleanup Module Unload Callback Function Example NV-Load Callback NV-Load Callback Initialization and Cleanup NV-Load Callback Function Example NV-Save Callback NV-Save Callback Initialization and Cleanup NV-Save Callback Function Example Config Replay Callback Periodic Timer Service Timed Config Replay Example RPC Operation Callbacks RPC Callback Initialization and Cleanup RPC Callback Function Examples165 Version Page 3

4 8.2.1 RPC Validate Callback Function RPC Invoke Callback Function.167 Version Page 4

5 1 Preface 1.1 Legal Statements Copyright , Andy Bierman, All Rights Reserved. Copyright , YumaWorks, Inc., All Rights Reserved. 1.2 Additional Resources This document assumes you have successfully set up the software as described in one or both of the printed documents: YumaPro Installation Guide Other documentation includes: YumaPro Quickstart Guide YumaPro User Manual YumaPro netconfd-pro Manual YumaPro yangcli-pro Manual YumaPro yangdiff-pro Manual YumaPro yangdump-pro Manual YumaPro Developer Manual YumaPro ypclient-pro Manual YumaPro yp-system API Guide YumaPro yp-show API Guide To obtain additional support you may contact YumaWorks technical support department: WEB Sites YumaWorks Offers support, training, and consulting for YumaPro. Netconf Central Yang Central Free information on NETCONF and YANG, tutorials, on-line YANG module validation and documentation database Version Page 5

6 Free information and tutorials on YANG, free YANG tools for download NETCONF Working Group Wiki Page Free information on NETCONF standardization activities and NETCONF implementations NETCONF WG Status Page IETF Internet draft status for NETCONF documents libsmi Home Page Free tools such as smidump, to convert SMIv2 to YANG Mailing Lists NETCONF Working Group Technical issues related to the NETCONF protocol are discussed on the NETCONF WG mailing list. Refer to the instructions on the WEB page for joining the mailing list. NETMOD Working Group Technical issues related to the YANG language and YANG data types are discussed on the NETMOD WG mailing list. Refer to the instructions on the WEB page for joining the mailing list. 1.3 Conventions Used in this Document The following formatting conventions are used throughout this document: Documentation Conventions Convention --foo <foo> foo $FOO $$foo some text some text Description CLI parameter foo XML parameter foo netconfd-pro command or parameter Environment variable FOO netconfd-pro global variable foo Example script Plain text Version Page 6

7 2 YumaPro API Overview YumaPro API Quick Start Guide This section gives a quick overview for those who want to get started with the netconfd-pro server and all the API that it provides without reading the entire manual first. It is assumed that the appropriate programs have been installed and configured correctly. 2.1 API Overview This section describes the basic API used in the netconfd-pro server. YumaPro tool provides numerous APIs that can be used to effectively perform desired tasks, improve efficiency, refine auto-generated code, and extend default functionality. Version Page 7

8 YumaPro netconfd-pro API map key: (1) Utility APIs YumaPro API Quick Start Guide Notification functions provide control over notifications. XPATH Handling/Validation functions can be used to manipulate XPATH expressions. Error Handling functions provide various alternative ways to record an error. Transaction Management functions provide control on the transaction and can be invoked within, before, and after the transaction. Timer Service functions allow some SIL code that may need to be called at periodic intervals to check system status, update counters, and/or send notifications. YANG Object Tree manipulation functions let you retrieve object properties without accessing them directly. Extension Access functions allow to manipulate with custom YANG extensions. YANG Data Tree manipulation functions provide access functions to the data nodes. (2) Server Instrumentation Library (SIL) Utility functions provide control on SIL libraries and allows them to be used more efficiently. (3) YControl and DB-API Interface functions provide control on subsystem and DB-API interfaces. (4) System Callback functions allow the creation and use of general system SIL libraries. (5) Database Access functions allow validation, manipulation, and management of your database. (6) Access Control functions allow configuration and enforcement of a vendor specific access control model (ACM). (7) SYSLOG and Log functions allow customized logging preferences. YumaPro provides the following APIs: Transaction Management functions that provide control on the transaction and can be invoked within, before, and after the transaction. YANG Object Tree manipulation functions, so that object properties do not have to be accessed directly. YANG Data Tree manipulation functions provide access functions to the data nodes. Error Handling functions provide various alternative ways to record an error. Notification functions provide control on notifications. XPATH Handling/Validation functions can be used to manipulate XPATH expressions. Extension Access functions allow to manipulate with custom YANG extensions. Timer Service functions allow some SIL code that may need to be called at periodic intervals to check system status, update counters, and/or perhaps send notifications. SIL Utility functions provide control on SIL libraries and allows to use them more efficiently. System Callback functions allow to creating and use the general system SIL library. Access Control functions allow to configure and enforce ACM. SYSLOG and Log functions allow to customize logging. Database Access functions allow to validate, manipulate, and manage database. YCONTROL Interface functions provide communication service between sub-agents and the main server. DB-API Interface provide database interface functions to edit the internal database from another internal process.. YUMAWORKS RESERVES THE RIGHT TO CHANGE INTERNAL DATA STRUCTURES AT ANY TIME WITHOUT NOTICE! API FUNCTIONS ARE STABLE, NOT DATA STRUCTURES! Version Page 8

9 2.2 Terminology Data Tree The Data Tree is a representation of some subset of all possible object instances that a server is maintaining within a configuration database or other structure. Each Data Tree starts with a Root container, and any child nodes represent top-level YANG module data nodes that exist within the server. Object Tree The Object Tree is a tree representation of all the YANG module RPC, data definition, and notification statements. It starts with a Root container. This is defined with a YANG container statement which has an ncx:root extension statement within it. The <config> parameter within the <edit-config> operation is an example of an object node which is treated as a Root container. Each configuration database maintained by the server (e.g., <candidate> and <running>) has a Root container value node as its top-level object. Root container The Root container is defined with a YANG container statement which has an ncx:root extension statement within it. The Root container does not have any child nodes defined in it within the YANG file. However, the YumaPro tools will treat this special container as if any top-level YANG data node is allowed to be a child node of the Root container type. Version Page 9

10 3 YANG Data Examples YumaPro API Quick Start Guide This section describes the basic design and demonstrates and exemplifies how to handle the YANG Object Tree and the corresponding Data Tree that represents instances of various object nodes that the client or the server can create. The Object Tree is a tree representation of all the YANG module RPC, data definition, and notification statements. It starts with a Root container. This is defined with a YANG container statement which has an ncx:root extension statement within it. The <config> parameter within the <edit-config> operation is an example of an object node which is treated as a Root container. Each configuration database maintained by the server (e.g., <candidate> and <running>) has a Root container value node as its top-level object. A Root container does not have any child nodes defined in it within the YANG file. However, the YumaPro tools will treat this special container as if any top-level YANG data node is allowed to be a child node of the Root container type. The Data Tree is a representation of some subset of all possible object instances that a server is maintaining within a configuration database or other structure. Each Data Tree starts with a Root container, and any child nodes represent top-level YANG module data nodes that exist within the server. Each configuration database maintains its own copy (and version) of the Data Tree. There is only one Object Tree, however, and all Data Trees use the same Object Tree for reference. Not all object types have a corresponding node within a Data Tree. Only 'real' data nodes are present. Object nodes that are used as meta-data to organize the Object Tree (e.g., choice, augment) are not present. 3.1 Object Tree Ordering Top-Level Data Nodes are derived from: data-def-stmt rpc-stmt notification-stmt The YANG object tree is maintained in schema order wherever that is defined. Top-Level Data Nodes: These nodes are sorted: Primary Key: local-name Secondary Key: module-name The top-level YANG objects from all loaded modules are present in the object tree (except if removed by status=obsolete or deviation=not-supported) Child Data Nodes, RPC and Notification Nodes Child nodes defined in the current module are in schema order (order they appear in the YANG module Augmenting child nodes are in no defined order. They will be after all the real child nodes and the order may change depending on how the augmenting modules are loaded. Version Page 10

11 DO NOT DEPEND ON THE ORDER OF AUGMENTING DATA NODES 3.2 YANG Object Tree Handling Examples In this section we will go through examples on how to utilize different type of the Object Trees, what API functions can be used for specific purpose, and when you should consider one API function to another. Let us go through the most prominent Object Tree handling examples, that are often used in SIL callbacks. There are various API functions that you can use to locate desired objects, they differ widely by available information that you have prior the search and by desired search precision Find Top-Level Object Assume you know the name of the target object and your goal is to obtain its object template. Assume the object is a top level object in the YANG module, in this case you cannot use obj_find_child or analogous API function that are using parent object to locate a child object. In this scenario you have two options, use module structure to locate desired objects or search through all the modules and try to match the object name string. If you know the module where the target object is located, the following retrieval function can be used: / get the top object node / obj_template_t topobj = obj_find_template_top(mod, module_name, object_name); if (!topobj) { res = ERR_NCX_DEF_NOT_FOUND; Access Module Pointer If you are developing code in the SIL code, the module usually is getting loaded to the server during the Initialization Phase 1 before the startup configuration is loaded into the running configuration database, and before the running configuration is loaded. The following code illustrates how the module pointer can be obtained during the module load procedure: / FUNCTION init_function Version Page 11

12 initialize the server instrumentation library. Initialization Phase 1 / static status_t init_function (const xmlchar modname, const xmlchar revision) { res = ncxmod_load_module(modname, revision, agt_get_savedevq(), mod); if (res!= NO_ERR) { return res; After we load the module and have the module pointer, we can utilize it to locate top level objects in the module. Alternatively, if the module already loaded and you want to locate the module in order to use it later to search for object templates you may use the following API function: ncx_module_t mod = ncx_find_module(modname, revision); if (ncmod == NULL) { res = ERR_NCX_DEF_NOT_FOUND; Match Any Object The second option to locate a top level object is to search through all the modules and try to match the object name string. The following API functions illustrate how to locate the top level object based on module name: obj_template_t topobj = ncx_match_any_object(object_name, NCX_MATCH_FIRST, alt_names, &res); Version Page 12

13 In the above example, NCX_MATCH_FIRST parameter dictates the API function to stop search on the first match. Alternatively, the parameter can be changed to the following options: Match mode NCX_MATCH_EXACT NCX_MATCH_EXACT_NOCASE NCX_MATCH_ONE NCX_MATCH_ONE_NOCASE NCX_MATCH_FIRST NCX_MATCH_FIRST_NOCASE description Try a case-sensitive exact-length match Try a case-insensitive exact-length match Try a case-sensitive partial-name match Try a case-insensitive partial-name match Try a case-sensitive first match Try a case-insensitive first match The alt_names parameter dictates if alternative names should be checked in addition to the YANG node names. If set to TRUE the check will be applied; if set to FALSE the check will be ignored. The res parameter may be set to ERR_NCX_MULTIPLE_MATCHES in case there are multiple matching objects available. In this case the function will not return any of the object templates. If you want to search with the highest precision for a specific object you should consider NCX_MATCH_EXACT value and set alt_names to FALSE. Alternatively, you can use the following extended version of the previous API function: obj_template_t topobj = ncx_match_any_object_ex(modname, object_name, dataonly, NCX_MATCH_EXACT, alt_names, &res); In the above example, the module name is optional; however it is recommended. If it is present the API function will search only within specified module and will not try to search through all the modules. The dataonly parameter specifies whether the search function should try to match only data nodes or not. Set to TRUE if it should. The res parameter may be set to ERR_NCX_MULTIPLE_MATCHES in case there are multiple matching objects available. In this case the function will not return any of the object templates. If you want to search with the highest precision for a specific object you should consider NCX_MATCH_EXACT value and set alt_names to FALSE Check For Child Nodes Now, since we know the top level object we can locate any children objects of the current top level object. However, only if the current object has and may have children. Version Page 13

14 In order to verify that the current object has any available children you may use the following API function: boolean has_children = obj_has_children(topobj); If there are any accessible nodes within the object the function returns TRUE Find a Specific Child Node The following example code illustrates how to locate a specific child object of the current top level object. The following API function checks for accessible names only. That means child nodes of choice, case will be present instead of the choice name or case name: / get the child obj template / obj_template_t child_obj = obj_find_child(topobj, modname, objname); Find Choice or Case Child Node In case you want to locate choice or case objects as well as other accessible names you may consider to use the following API function. This function checks not only for accessible names. That means the choice name or case name will be preferred instead of their child nodes: / get the child obj template / obj_template_t child_obj = obj_find_child_choice_case(topobj, modname, objname); Alternatively, you can use the following extended version of the previous API functions: Version Page 14

15 / get the child obj template / obj_template_t child_obj = obj_find_child_ex(topobj, modname, objname, NCX_MATCH_FIRST, alt_names, dataonly, &res); Get First Child In addition to the find_child family API functions, you may consider to use get_child API functions. The main difference is that find functions are trying to match desired object, but get functions merely get the first, next, last, etc objects. To exemplify, if you want to get the first child object of the current object, the following API function can be used. Note, that this function skips over augments and uses: obj_template_t first_obj = obj_first_child(topobj); Similarly, the following API function can be used to locate the first terminal object. Note, that this function also skips over augments and uses: obj_template_t first_obj = obj_first_terminal_child(topobj); Get Next Child In order to get the next, the last, parent or previous child object or the terminal child object the following API functions can be used: obj_template_t next_obj = obj_next_child(first_obj); Version Page 15

16 obj_template_t next_obj = obj_next_terminal_child(first_obj); obj_template_t prev_obj = obj_previous_child(next_obj); obj_template_t last_obj = obj_last_child(topobj); obj_template_t topobj = obj_get_parent(next_obj); Process All Child Nodes To summarize previous list of API functions, you may construct the following loop to apply custom actions for desired children objects: / check all the child nodes / obj_template_t chobj = obj_first_child(obj); for (; chobj; chobj = obj_next_child(chobj)) { const xmlchar modname = obj_get_mod_name(chobj); const xmlchar objname = obj_get_name(chobj); / process objects here / Process Ancestor Nodes In order to loop through ancestor entries and process them as required, the following code may be used: / go through the ancestor entries and process them as needed / obj_template_t testobj = obj_get_parent(chobj); while (testobj) { / process parent objects here / testobj = obj_get_parent(testobj); if (testobj && obj_is_root(testobj)) { testobj = NULL; Version Page 16

17 The obj_is_root API function signals when to stop. It is also possible to have a NULL pointer returned. That indicates you reached the top Root container. This container should NOT be modified, accessed or changed in any manner. Version Page 17

18 3.3 YANG Data Tree Handling Examples In this section we will go through examples on how to utilize different type of the Data Trees, what API functions can be used for specific purpose, and when you should consider one API function to another. Let us go through the most prominent Data Tree handling examples, that are often used in SIL callbacks. There are various API functions that you can use to manipulate with desired Data node, they differ widely the operation that you need to perform. Let us go through simple examples that will illustrate how to manage different YANG node types Leaf Examples The following examples illustrates how to construct a simple leaf data node. There are multiple ways to construct leaf node depending on what information you have before the construction and what data type is the node. The most common and generic API function to construct the data node that does not required to know the type of the node is demonstrated below. However, you have to find the object template prior the construction: obj_template_t leafobj = obj_find_child(parentobj, modname, objname); if (!leafobj) { / report an error or do not try to generate the leaf / return ERR_NCX_DEF_NOT_FOUND; / construct a leaf val_vale tree regardless of its type / val_value_t any_type_leaf = val_make_simval_obj(leafobj, (const xmlchar )"8", &res); The most beneficial part of this API function is that you do NOT have to investigate the type of the node before the construction. However, If you know the type of the node and want to construct the value based on the type, the following example demonstrates how to accomplish this goal. In this example we are constructing string type data node: / construct a sting type leaf / val_value_t string_leaf = agt_make_leaf(parentobj, leafname, (const xmlchar )"example value", Version Page 18

19 &res); In the example code above, if the PARENTOBJ is a leaf or leaf-list then it will be used directly instead of checking for a child node. In order to construct data node with integer types, the following API function can be used: / construct a uint32 type leaf / val_value_t string_leaf = agt_make_uint_leaf(parentobj, leafname, (uint32)32, &res); / construct a uint32 type leaf / val_value_t value = agt_make_int_leaf(parentobj, leafname, (int32)32, &res); / construct a uint64 type leaf / val_value_t value = agt_make_uint64_leaf(parentobj, leafname, (uint64)64, &res); / construct a int64 type leaf / val_value_t value = agt_make_int64_leaf(parentobj, leafname, (int64)64, &res); In order to construct data node with boolean type, the following API function can be used. The BOOLVAL value is getting set to TRUE is the value of this node should be TRUE: / construct a boolean type leaf / val_value_t string_leaf = agt_make_boolean_leaf(parentobj, modname, leafname, boolval, &res); Version Page 19

20 In order to construct data node with empty type, the following API function can be used. The BOOLVAL value is ignored by this API function: / construct a boolean type leaf / val_value_t string_leaf = agt_make_empty_leaf(parentobj, modname, leafname, boolval, &res); Leaf-list Examples The following examples illustrates how to construct a simple leaf-list data node. The leaf-list nodes can be constructed the same way as regular leaf nodes. However, there might be multiple instances, siblings of the same leaf-list node in the Data Tree. The following example code illustrates how to construct 3 leaf-list siblings using the same API functions as for leaf node: / construct 3 leaf-list entries / val_value_t leaflist_value1 = val_make_simval_obj(leaflist_obj, (const xmlchar )"53", &res); if (!leaflist_value) { return res; val_value_t leaflist_value2 = val_make_simval_obj(leaflist_obj, (const xmlchar )"30", &res); if (!leaflist_value) { return res; val_value_t leaflist_value3 = val_make_simval_obj(leaflist_obj, (const xmlchar )"80", &res); if (!leaflist_value) { return res; Alternatively, the following example can be used in order to construct multiple leaf-list entries of int32 type node and add them to the existent container parent: Version Page 20

21 status_t res = NO_ERR; int32 value = 0; for (value = 10; value <= 15 && res==no_err; value++) { val_value_t leaflist_value = agt_make_int_leaf(parentobj, leaflistname, value, &res); if (!leaflist_value) { return res; / add a new leaf-list entry into target container / res = val_child_add(leaflist_value, container_value); if (res!= NO_ERR) { val_free_value(leaflist_value); As a result, all the leaf-list entries will be added to the parent container. The next section will describe this construction in more details Container Examples The following examples illustrates how to construct a container data node. The container nodes can be constructed different ways depending whether it is a top level container or not. If the container is not a top level container, the following code can be used: / get the container obj template / obj_template_t cont_obj = obj_find_child(parentobj, modname, objname); if (!cont_obj) { return ERR_NCX_DEF_NOT_FOUND; / make a container value / val_value_t cont_value = val_new_value(); if (!cont_value) { return ERR_INTERNAL_MEM; val_init_from_template(cont_value, cont_obj); Version Page 21

22 On the other hand, if the container is a top level container, and you are constructing it in order to use as a static or virtual data within running datastore. If the container represents operational data, the following example can be used: cfg_template_t runningcfg = cfg_get_config_id(ncx_cfgid_running); if (!runningcfg!runningcfg->root) { return ERR_NCX_DEF_NOT_FOUND; / get the top object node / obj_template_t topobj = obj_find_template_top(mod, module_name, object_name); if (!topobj) { return ERR_NCX_DEF_NOT_FOUND; / construct top level container / val_value_t cont_value = val_new_value(); if (!cont_value) { return ERR_INTERNAL_MEM; val_init_from_template(cont_value, topobj); / handing off the malloced memory here / res = val_child_add(cont_value, runningcfg->root); if (res!= NO_ERR) { val_free_value(cont_value); return res; As a result of these two examples, we will have an empty container with no any children. In order to add a child to the container, the following example that creates multiple leaf-list entries and adds them into parent container can be used: status_t res = NO_ERR; int32 value = 0; for (value = 10; value <= 15 && res==no_err; value++) { val_value_t leaflist_value = agt_make_int_leaf(parentobj, leaflistname, value, &res); if (!leaflist_value) { return res; / add a new leaf-list entry to parent container / res = val_child_add(leaflist_value, cont_value); if (res!= NO_ERR) { val_free_value(leaflist_value); Version Page 22

23 return res; In order to construct nested containers, the same steps should be applied twice, as illustrated below: cfg_template_t runningcfg = cfg_get_config_id(ncx_cfgid_running); if (!runningcfg!runningcfg->root) { return ERR_NCX_DEF_NOT_FOUND; / get the top object node / obj_template_t topobj = obj_find_template_top(mod, module_name, object_name); if (!topobj) { return ERR_NCX_DEF_NOT_FOUND; / construct top level container / val_value_t cont_value = val_new_value(); if (!cont_value) { return ERR_INTERNAL_MEM; val_init_from_template(cont_value, topobj); / handing off the malloced memory here / status_t res = val_child_add(cont_value, runningcfg root); if (res!= NO_ERR) { val_free_value(cont_value); return res; / get the next container obj template / obj_template_t cont_obj = obj_find_child(topobj, modname, objname); if (!cont_obj) { val_free_value(cont_value); return ERR_NCX_DEF_NOT_FOUND; / make a next container value / val_value_t next_cont_value = val_new_value(); if (!next_cont_value) { val_free_value(cont_value); return ERR_INTERNAL_MEM; val_init_from_template(next_cont_value, cont_obj); res = val_child_add(next_cont_value, cont_value); if (res!= NO_ERR) { Version Page 23

24 val_free_value(next_cont_value); val_free_value(cont_value); return res; List Examples The following examples illustrates how to construct a list data node. The list nodes have several limitations and special handling nuances. For following list summarize the most important: NETCONF does not require entries to be sorted by index. It only requires the order to be maintained if ordered-by user is configured in the YANG module. The server will maintain the order or instances given in the <editconfig> or startup or other sources. It will maintain YANG schema order so the objects will be returned in the correct order according to the YANG module. val_gen_index_chain MUST be called after all the index leafs have been added to the list. This can be done before or after any additional child nodes are added to the list. The index (key) nodes should be added first, before any other nodes are added. There are multiple variants of val_child_add. Sometimes the value is only created so it can be output in XML or JSON, and not to be stored in the database. The API functions in val_child should be used instead of val.c. The val.c API functions still works, but the new functions should be used in new code. The new functions can fail with an error, so it is important to check the return value in server code. The yangcli-pro and compiler programs will use val_child_add_force by default, since they do not store the data nodes in a database. Function val_child_add val_child_add_force val_child_insert Description Replaces val_add_child and val_add_child_sorted Allows AVL tree insertion to be skipped for duplicates or incomplete list entries Replaces val_insert_child The following example illustrates how to construct the list data node with multiple children: / FUNCTION create_list_entry3 Make a list entry based on the key INPUTS: res = return status RETURNS: val_value_t of listval entry if no error Version Page 24

25 else NULL / static val_value_t create_list_entry(obj_template_t parentobj, int32 key, status_t res) { / get the obj template for the list obj / obj_template_t list_obj = obj_find_child(parentobj, (const xmlchar ) my-module, (const xmlchar ) my-list ); if (!list_obj) { res = ERR_NCX_DEF_NOT_FOUND; return NULL; / make the list node / val_value_t list_value = val_new_value(); if (!list_value) { res = ERR_INTERNAL_MEM; return NULL; val_init_from_template(list_value, list_obj); / make key leaf entry / val_value_t child = agt_make_int_leaf(list_obj, keyname, key, res); if (!child) { val_free_value(list_value); return NULL; res = val_child_add(child, list_value); if (res!= NO_ERR) { val_free_value(child); val_free_value(list_value); return NULL; / make an extra leaf entry / child = agt_make_uint_leaf(list_obj, (const xmlchar ) my-other-leaf, (uint32)32, res); if (!child) { val_free_value(list_value); return NULL; res = val_child_add(child, list_value); if (res!= NO_ERR) { val_free_value(child); val_free_value(list_value); return NULL; / generate the internal index Q chain / res = val_gen_index_chain(list_obj, list_value); if (res!= NO_ERR) { Version Page 25

26 val_free_value(list_value); return NULL; return list_value; / create_list_entry3 / YumaPro API Quick Start Guide This above function is an example function and can be changed according to the required goal. By using this function, we create one list entry that we can now add to the parent or repeat the same steps but with different key value in order to generate another list entry. The following example illustrates how to generate multiple list entries using the sample function above and add them to parent container value: / malloced and construct list values / status_t res = NO_ERR; int32 key = 0; for (key = 10; key <= 15; key++) { val_value_t list_value = create_list_entry(container_obj, key, &res); if (!list_value) { return res; / add a new list entry into target container / res = val_child_add(list_value, contvalue); if (res!= NO_ERR) { val_free_value(list_value); return res; As a result we generated container with 5 list entries. Version Page 26

27 4 Database Access YumaPro API Quick Start Guide This document describes the configuration and deployment options for the configuration database used by the netconfd-pro server. Information included: Database related CLI parameters Deployment Variants Use-Case description System integration description Related Callback APIs 4.1 Common CLI Parameters Startup Options There are 3 startup parameter variants, using the YANG 'start' choice statement. choice of 3 leafs: --startup --factory-startup --no-startup The --startup=filespec CLI parameter can be used to specify the NV-storage file. The default filespec is $HOME/.yumapro/startup-cfg.xml The parameter --factory-startup can be used to load an empty configuration instead of the stored configuration. If the default configuration is used (startup-cfg.xml) then it is reset to factory default values. The parameter --no-startup can be used to load an empty configuration instead of the stored configuration. The default configuration (startup-cfg.xml) is not affected by this parameter. Version Page 27

28 4.2 Database Deployment Variants There are many ways to configure and build the server for storing and retrieving datastore contents. Local internal database: configuration in memory and NV-storage in netconfd-owned XML file Local internal database + NV-store hook: configuration in memory and NV-storage via callback functions, and transferred to/from the server as an XML file Local internal database + no-nv-store: configuration in memory and NV-storage is managed by the yp-system and/or SIL code. The server will not attempt to load or store the configuration to non-volatile storage Local internal database + external edits: your external process communicates with the netconfd-pro process to initiate external edits (e.g., initiated from legacy CLI or internal database) Local internal database netconfd pro running config startup cfg.xml in memory This is the default database configuration and provides complete management of all database storage and management. No extra APIs or CLI parameters are required. Configuration properties: Tree-based data structures in memory for run-time transaction processing This is the canonical database There is no other database The non-volatile load and store is done by netconfd-pro using the startup filespec to store the configuration The file is encoded as an XML instance document containing 1 element named 'data' No YANG default leafs are stored in this file Version Page 28

29 4.2.2 Local internal database + NV-store Hook netconfd pro running config in memory startup cfg.xml NV store hook This database variant allows the vendor/system code to manage the non-volatile configuration. The system control is constrained to the lreplacement of the functions that load and store the configuration file. This variant requires that the NV-store APIs (described in the Database Transaction Callbacks section. Configuration properties: Tree-based data structures in memory for run-time transaction processing This is the canonical database There is no other database The non-volatile load and store is done by netconfd-pro via user callback functions. The NV-store callback functions manage the actual non-volatile representation and storage. The NV-Store callback transfers the configuration to/from netconfd-pro as an XML file YANG default leafs will be treated as explicitly set if contained in the transfer file Version Page 29

30 4.2.3 Local internal database + no-nv-store netconfd pro running config in memory This variant allows either a simple deployment that does not support non-volatile storage of configuration or a more flexible system-managed non-volatile storage design, using multiple server APIs. This variant requires that the CLI parameter --no-nvstore be set to 'false'. The --startup CLI parameter is ignored if --no-nvstore is used. Various initialization callback APIs and transaction APIs can be used to load and save the non-volatile storage Configuration properties: Tree-based data structures in memory for run-time transaction processing This is the canonical database There is no other database The non-volatile load and store is done by netconfd-pro via various user callback functions. The internal NV-store and NV-store callback functions are not used The yp-system library or individual SIL libraries can be used to initialize and fill the empty database with configuration and operational data nodes Transaction hooks can be used to save the running configuration at run-time if any client edit transactions are processed by the server Version Page 30

31 4.2.4 Local internal database + External Edits netconfd pro running config in memory startup cfg.xml DB API msgs your process This database variant allows an external process to initiate database edits, that are processed by the server as user or system edits. The DB-API functions described in the next section can be used by the subsystem to edit the server configuration database. This variant can be used with other variants for processing non-volatile storage. This variant is used to co-exist with a legacy or canonical database owned by the system. Configuration properties: Tree-based data structures in memory for run-time transaction processing This may or may not be the canonical database, depending on the non-volatile storage options. There may be another database in the system The server uses its copy in memory as the complete database The DB-API send_edit and send_edit_ex API functions can be used to transfer configuration from the system to tnetconfd-pro. Data is transferred in XML Error responses are sent by the server if an edit is not accepted. An error will be returned if a client is already in the process of editing the database Version Page 31

32 The db-api-app application in installed in /usr/share/yumapro/src/db-api-app by default, and includes some examples of initializing the DB-API service and sending edits to the server 4.3 DB-API Example The DB-API subsystem allows an external process on the same system as netconfd-pro to send database edits to the server. db_api_register_service db_api_service_ready db_api_send_edit db_api_send_edit_ex db_api_send_edit_full db_api_check_edit extern status_t db_api_register_service (void); extern boolean db_api_service_ready (void); extern status_t db_api_send_edit (const xmlchar edit_target, const xmlchar edit_operation, const xmlchar edit_xml_value); extern status_t db_api_send_edit_ex (const xmlchar edit_target, const xmlchar edit_operation, const xmlchar edit_xml_value, const xmlchar patch_id_str, boolean system_edit); / FUNCTION db_api_send_edit_full Create a YANG Patch edit request and send it to the DB-API service on the main server. The content should represent the intended target resource as specified in YANG-API (NOT RESTCONF) Only the data resource identifier is provided, not the API wrapper identifiers (so this can change when RESTCONF is supported) Example leaf: edit_target == /interfaces/interface/eth0/mtu edit_value == "<mtu>9000</mtu> Version Page 32

33 edit_operation == "merge" patch_id_str == "my-patch-x01' system_edit == true Example list: edit_operation == <operation string> "create" "delete" "insert" "merge" "move" "replace" "remove" edit_target == /interfaces/interface/eth0/ipv4 edit_value == "<ipv4> <enabled>true</enabled>< <forwarding>true</forwarding> <address> </address> <prefix-length>24</prefix-length> </ipv4>" INPUTS: edit_target == target resource (YANG-API path expression) edit_operation == edit operation (create merge replace delete remove) edit_xml_value == XML payload in string form, whitespace allowed MAY BE NULL if no value required (delete remove)) patch_id_str == string to use as the patch ID == NULL to use the default patch-id field system_edit == TRUE if this edit is from the system and should bypass access control enforcement == FALSE if this edit is from a user and should not bypass access control enforcement insert_point is a string like the target except a different instance of the same list of leaf-list; only for before, after insert_where == <insert enum string> "before" "after" "first" "last" RETURNS: status / extern status_t db_api_send_edit_full (const xmlchar edit_target, const xmlchar edit_operation, const xmlchar edit_xml_value, const xmlchar patch_id_str, boolean system_edit, const xmlchar insert_point, const xmlchar insert_where); / FUNCTION db_api_check_edit Check on the status of an edit in progress RETURNS: status: ERR_NCX_NOT_FOUND if final status and no message response is pending Version Page 33

34 ERR_NCX_SKIPPED if final status is not known yet NO_ERR if there is a last-completed operation that completed with an OK response <errcode> if there is a last-completed operation that completed with an ERROR response / extern status_t db_api_check_edit (void); Example db-api-app send_edit from ap-api-app.c / FUNCTION send_test_edit This is an example send edit function. The module ietf-interfaces needs to be loaded for this to work / static void send_test_edit (void) { / mef-cfm test / const xmlchar path_str = (const xmlchar )"/maintenance-domain/mdtest"; const xmlchar operation_str = (const xmlchar )"merge"; const xmlchar value_str = (const xmlchar ) "<cfm:maintenance-domain xmlns:cfm=' "ns/yang/mef-cfm'><cfm:id>mdtest</cfm:id><cfm:name-type>character" "-string</cfm:name-type><cfm:name>mdtestname</cfm:name><cfm:md-le" "vel>4</cfm:md-level><cfm:mhf-creation>none</cfm:mhf-creation>" "<cfm:id-permission>none</cfm:id-permission></cfm:maintenance" "-domain>"; const xmlchar patch_id_str = NULL; boolean system_edit = FALSE; status_t res = db_api_send_edit_ex(path_str, operation_str, value_str, patch_id_str, system_edit); if (res!= NO_ERR) { log_error("\nsend test edit failed %s %s = %s (%s)\n", operation_str, path_str, value_str, get_error_string(res)); else if (LOGDEBUG) { log_debug("\nsend test edit OK %s %s = %s\n", operation_str, path_str, value_str); / send_test_edit / / FUNCTION main This is an example main function. Version Page 34

35 RETURNS: 0 if NO_ERR status code if error connecting or logging into ncxserver / int main (int argc, char argv) { #ifdef MEMORY_DEBUG mtrace(); #endif / 1) setup yumapro messaging service profile / status_t res = ycontrol_init(argc, argv, (const xmlchar )"subsys1"); / 2) register services with the control layer / if (res == NO_ERR) { res = db_api_register_service(); / 3) do 2nd stage init of the control manager (connect to server) / if (res == NO_ERR) { res = ycontrol_init2(); useconds_t usleep_val = ; // 100K micro-sec == 1/10 sec boolean done = FALSE; / 4) call ycontrol_check_io periodically from the main program control loop / #ifdef DB_API_APP_DEBUG int id = 0; #endif // DB_API_APP_DEBUG boolean test_done = FALSE; while (!done && res == NO_ERR) { #ifdef DB_API_APP_DEBUG if (LOGDEBUG3) { log_debug3("\ndb-api-app: checking ycontrol IO %d", id++); #endif // DB_API_APP_DEBUG res = ycontrol_check_io(); if (ycontrol_shutdown_now()) { // YControl has received a <shutdown-event> // from the server subsystem is no longer active // could ignore or shut down YControl IO loop done = TRUE; // Using sleep to represent other program work; remove for real if (!done && res == NO_ERR) { (void)usleep(usleep_val); if (db_api_service_ready() &&!test_done) { send_test_edit(); test_done = TRUE; Version Page 35

36 / 5) cleanup the control layer before exit / ycontrol_cleanup(); #ifdef MEMORY_DEBUG muntrace(); #endif return (int)res; / main / Version Page 36

37 5 Database Transaction Callbacks YumaPro provides pre-transaction, in-transaction and post-transaction APIs to manage behavior of a database edit transaction, such as its functionality, order, etc. This section illustrates how to utilize transaction callbacks, highlighting some relevant use cases in detail. YANG can model state data, as well as configuration data, based on the "config" statement. When a node is tagged with "config false", its sub-hierarchy is flagged as state data, to be reported using NETCONF's <get> operation, not the <getconfig> operation. When a node has no tag or tagged with config true, its sub-hierarchy is flagged as configuration data. Consider this simplified, but functional, example. container interfaces { list interface { key "name"; leaf name { type string; leaf-list untagged-ports { type string; leaf speed { type enumeration { enum 10m; enum 100m; enum auto; leaf observed-speed { config false; type uint32; leaf version { config false; type string; In this example, three leafs are defined for each interface, a configured speed and untagged-ports and a nonconfigurable observed-speed. The speed and untagged-ports is configuration, so it can be returned with NETCONF <get-config> operations and they can be manipulated using <edit-config>. Note that the observed-speed is not configuration and it cannot be manipulated using <edit-config>. This differentiation is critical and based on it an appropriate callback should be used. For the configuration data there should be EDIT-like callbacks registered. However, for the state data, GET-like callbacks should be registered. Version Page 37

38 The following transaction management APIs are used to manage an object that is being edited or retrieved. They are used to provide a callback sub-mode for a specific named object. API Description EDIT-1 callback EDIT-2 callback GET-1 callback GET-2 callback agt_cb_fn_t: First generation callbacks are generated by yangdump-sdk using node-based APIs. An edit function is generated for every node, including terminal nodes (leaf, leaf-list, anyxml). The GET1 API is assumed for read-only data, so code to generate virtual read-only nodes (Make Read Only MRO) is included in an EDIT1 callback. This is the default in yangdump-sdk and make_sil_ scripts. agt_cb_fn_t: Second generation callbacks are generated by yangdump-sdk using container or list-based APIs. An edit function is generated for parent list and container nodes only. The terminal child nodes are handled by this callback. Each nested container or list has its own callback. This is generated in yangdump-sdk or make_sil_ scripts using the sil-edit2 parameter. The same callback function signature as EDIT1 is used, but the registration function and procedure is different. getcb_fn_t: Used for <get> and <get-config> operations. GET1 requires an actual data node in the data tree for each instance that exists. A data node with a GET1 callback function is called a 'virtual nodes'. It is usually created in SIL code for operational data. Only supports internal GET callback getcb_fn2_t: Used for the <get> operation. There is 1 GET2 callback for the object, not each instance in the data tree. GET, GETNEXT, and GETBULK operations are supported. Optional parameters allow subtree/xpath filtering to be passed to the GET2 callback for pruning. Version Page 38

39 5.1 EDIT1 and EDIT2 Callbacks The following sections illustrates how to utilize the EDIT1 and EDIT2 callbacks in examples. The server supports 2 modes of database editing callbacks. The original mode (EDIT1) is designed to invoke data node callbacks at the leaf level. This means that each altered leaf will cause a separate SIL callback. If no leaf callbacks are present, then the parent node will be invoked multiple times. The EDIT2 mode is list-based or container-based instead. The following key aspects define EDIT2 callbacks: In this mode there are no SIL callback functions expected for terminal nodes (leaf, leaf-list, anyxml). The SIL callback for a container or list will be invoked, even if only child terminal nodes are being changed. The parent SIL callback will be invoked only once (per phase) if multiple child nodes are being altered at once The parent node for such an edit is flagged in the undo record as a special edit2_merge. The edit operation will be OP_EDITOP_MERGE, but the parent node is not being changed The special edit2_merge type of edit will have a queue of child_undo records containing info on the child edits. For example, 1 leaf could be created, another leaf modified, and a third leaf deleted, all in the same edit request. The child_undo records provide the edit operation and values being changed. The following function template definition is used for EDIT1 and EDIT2 callback functions: / Typedef of the EDIT1/2 callback functions / typedef status_t (agt_cb_fn_t) (ses_cb_t scb, rpc_msg_t msg, agt_cbtyp_t cbtyp, op_editop_t editop, val_value_t newval, val_value_t curval); Callback Template Type: User Callback Max Callbacks: 1 per object File: agt_cb.h Template: agt_cb_fn_t Inputs: scb == session control block making the request msg == incoming rpc_msg_t in progress cbtyp == reason for the callback editop == edit operation enumeration for the node being edited newval == object holding the proposed changes to apply to the current config, depending on the editop value. Version Page 39