Software Development Toolkit for Desire2Learn Web Services

Transcription

1 Software Development Toolkit for Desire2Learn Web Services Preview of the Developer Guide by Zdenek Nejedly June 1, 2010 Disclaimer: You are using this document or any provided code samples at your own risk. This is an experimental product for education purposes only. There are no warranties whatsoever. It is not endorsed by Desire2Learn or the University of Guelph. Desire2Learn is a trademark of Desire2Learn Incorporated, Kitchener, ON. Abbreviation D2L used throughout this Guide refers to Desire2Learn. Zdenek Nejedly 2010 All rights reserved

2 Frequently Asked Questions Why Software Development Toolkit? Application design with Web Services often requires significant overhead you need to set the request/response variables, manage WS-Security, and handle errors. Additionally, you may wish to add object caching to enhance performance or quality assurance layer to account for communication errors. This Toolkit strives to simplify the application development, avoid code duplication, and allow the developer to focus on the business domain instead. What does the Toolkit provide? The Toolkit provides the following benefits: a) Management of the generic java Web Services overhead b) D2L Web Services authentication and secure token management (expiration, re-issue) c) Streamlining of the D2L Web Service calls into simple function calls d) Automatic management of the database (for example for object mapping) e) Providing an interface which almost hides the D2L specific implementation and decreases the coupling of the business logic Any examples of the Toolkit use? The following code connects to the D2L web service at address serviceurl and retrieves the user object associated with the central account bsmith12 in just three lines of code: TokenManager.initialize(serviceUrl, login, passwd, 30000L, 200L); // initialize authentication ToolBox toolbox = new ToolBox(serviceUrl); // initialize the toolkit UserInfo user = toolbox.getusertools().getuserbyusername( bsmith12 ); // retrieve the user The same code block with the database cache enabled differs only in the name of the toolbox module: TokenManager.initialize(serviceUrl, login, passwd, 30000L, 200L); // initialize authentication ToolBox toolbox = new ToolBoxCached(serviceUrl); // initialize the toolkit UserInfo user = toolbox.getusertools().getuserbyusername( bsmith12 ); // retrieve the user What is the Database Cache?

3 The Desire2Learn WebServices API sometimes operates with references (pointers) to values instead of the values themselves. For example, let s say Jane Smith has an account with the University under username jane_smith. If we search for an account in D2L based on the user s real name then the D2L Web Services will return an internal D2L userid, let s say, If you wish to receive Jane s actual username (jane_smith) you will need to issue another Web Service call with the internal userid (123456) and then you will receive her username. The Toolkit combines both Web Service calls in just one high level function call and it even takes care of the Web Service authentication (generating and managing the WS-Security token). Additionally, if you use the cached version of the Toolkit object then the mapping between the internal D2L userids and the account usernames will be cached and the real number of Web Service calls issued in the background will be reduced. The database cache stores the object mapping in a relational database. The schema part applicable for example to user mapping is illustrated in Figure 1. Connectivity between the application and the database is accomplished via the object-relational mapping framework Hibernate version 3 [3]. Currently we have it implemented in MySQL server but there is no specific dependency on MySQL. Moving to a different relational database is a matter of changing the Hibernate configuration as long the particular database type is supported by Hibernate [3]. grades PK,I1 UserId PK,I2 CourseId PK,I3 GradeId Value PK,I1 I1,U1 users UserId UserName Figure 1: ER Diagram of the database cache schema Are all WebService calls from the D2L API included in this Toolkit? Currently, the Toolkit provides function equivalents to setting and reading of the grade values, user and course lookups and closely related functions. It contains the essential layer for authentication and handling of the WS request/response logic so it is very easy to expand and include other D2L WS functionality as needed.

4 2. Developer s Guide 2.1 Overview The toolkit consists of two main categories of components (Figure 2): a) Authentication components TokenManager SecurityToken b) Specific services, e.g., for managing grades, users, or courses +createtoken() : SecurityToken 1 1 +tostring() : String DescriptorTools AbstractServiceTools UserTools UserToolsCached For more details please see the javadoc pages and the java class files within the project deliverables. OrgUnitTools GradesTools GradesToolsCached ToolBox ToolBoxCached Figure 2: Simplified Toolkit class diagram +getdescriptortools() +getusertools() +getorgunittools() +getgradestools() 2.2 Authentication components Each D2L Web Service request requires an authentication token which is valid for specific number of uses and a defined time interval. Requesting the token for each web service call would double the number of interactions between the application and D2L and is very resource-intensive. Therefore, a custom service TokenManager was created. TokenManager requests a security token from D2L, manages the token s expiry and usage count, recreates it as needed, and issues the token to toolkit services, e.g., the grade service. The application business code typically does not interact with the TokenManager directly except for one-time initialization (per application execution). In terms of architecture, TokenManager is a singleton. 2.3 Toolkit services

5 The toolkit contains fours specific services that provide simple interface for corresponding D2L Web Services: a) DescriptorTools b) GradesTools c) OrgUnitTools d) UserTools Each toolkit service implements and extends abstract AbstractServiceTools, which provides common methods like WS request setup via TokenManager, response validation, and error handling. Additionally, the UserTools and GradesTools have corresponding cached versions (UserToolsCached and GradesToolsCached) that utilize the database cache for user mapping (UofG-D2L) and upload of grade values. 2.4 Test framework Toolkit services have integrated test classes based on the junit test framework [5]. To facilitate testing on two different instances (development and production), the test configuration data are stored in the properties files and loaded in the setupclass() methods. Please note that the role mapping needs to be set manually. 2.5 Developing applications with the toolkit The toolkit provides interface to those D2L WS calls that are required by the UofG Physics Quiz Room systems (note the Toolkit was created at UofG for the purpose of transferring grades between the UofG Quizroom and D2L). Other D2L Web Services operations can be readily implemented by cloning the methods in the toolkit as a template. 2.6 Case study download of final grades Application design with the toolkit is illustrated via a specific business case: download of final grades. Full source code of the functional prototype is provided in Appendix 1. The essentials steps are described below. 1) Process the command line parameters and set up the application variables, e.g., courseid, gradename, etc. 2) Initialize the toolkit

6 TokenManager.initialize(serviceUrl, login, passwd, 30000L, 200L); ToolBox toolbox = null; if (enablecache) toolbox = new ToolBoxCached(serviceUrl); else toolbox = new ToolBox(serviceUrl); 3) Retrieve the toolkit services GradesTools gradestools = toolbox.getgradestools(); UserTools usertools = toolbox.getusertools(); OrgUnitTools orgtools = toolbox.getorgunittools(); 4) Find the given course course = orgtools.getcourseoffering(courseid); 5) Find the grade object within the given course grade = gradestools.getgradebyorgunitandgradename(courseid, gradename); 6) Retrieve the grade values String strgradeid = grade.getgradeobjectid().getid(); List<GradeValueInfo> values = gradestools.getgradevaluesbygradeobject(courseid,strgradeid);

7 7) Store the grade values in a text file for (GradeValueInfo value : values) { if (value!=null && (pointsnumerator = value.getpointsnumerator())!=null) out.println(usertools.getusernamebyuserid(value.getuserid().getid()) + "," + pointsnumerator.floatvalue()); An example of a command that exports a given grade is below (to export the final grades substitute the appropriate column name for Quiz 1 ): java -Djava.util.logging.config.file=logging.properties -classpath webfm.jar ca.uoguelph.identity.d2l.webfm.exportgrade -l wsusername -w pwd3 -a -c x "Quiz 1" -e -o out.txt The command generates the following output: [quizroom@main quizroom]$ java -Djava.util.logging.config.file=logging.properties -classpath webfm.jar ca.uoguelph.identity.d2l.webfm.exportgrade -l wsusername -w pwd3 -a -c x "Quiz 1" -e -o out.txt About to export Grade Book data for course "-DEV- PHYS1070 F09 Intro Physics for Life Sciences" dev_phys_1070_f09 Looking up the grade objects in course "12102" Found grade "Quiz 1" Retrieving grade values for "Quiz 1" with gradeid: Grade values for "Quiz 1" received (total count = 847) Redirecting the exported grades to file out.txt 2.7 Related technologies Knowledge of several technologies is recommended when developing applications with the toolkit (Table 2). Table 1: Related technologies Technology Java Standard Edition version 6 (JSE6) see When used This is the essential requirement JSE is the application s design platform

8 Java Enterprise Edition (JEE 5 and up), see Hibernate, see MySQL database design Desire2Learn Web Services NetBeans IDE (version 6.7 and up) see Web Services, e.g., when generating the WS config files, database access Implementation of the database cache relational persistence When implementing the database cache on MySQL (any relational database supported by Hibernate can be used instead) When extending the toolkit (services in ToolBox and ToolBoxCached) Integrated Development Environment (IDE) used to design and compile the application

9 3. References [1] Desire2Learn Inc.: Desire2Learn Web Services Developer and API Guide, July 2, 2008 [2] Desire2Learn Inc.: Desire2Learn Web Services Code Samples User and reference guide, July 3, 2008 [3] Hibernate: Relational Persistence for Java and.net [4] Java downloads at [5] JUnit test framework (

10 Appendix 1: Source code for the case study (download of specific grades) package ca.uoguelph.identity.d2l.webfm; import java.io.; import java.util.list; import java.math.bigdecimal; // toolkit imports import ca.uoguelph.identity.d2l.webfm.auth.tokenmanager; import ca.uoguelph.identity.d2l.webfm.service.gradestools; import ca.uoguelph.identity.d2l.webfm.service.usertools; import ca.uoguelph.identity.d2l.webfm.service.orgunittools; import ca.uoguelph.identity.d2l.webfm.service.toolbox; import ca.uoguelph.identity.d2l.webfm.service.toolboxcached; // Desire2Learn Web Services imports //.. import statements for D2L code removed... znejedly@uoguelph.ca Downloading a specific grade column from D2L - Demo on how to use the toolkit / public class ExportGrade { public ExportGrade() { / Prints the usage - command line parameters / public static void printusage() { System.out.println( "Usage: java -jar webfm.jar -l username (-p password -w pwdfile) -c courseid -a url\n" + " (-u filename -o filename) [-d] [-s] [-x (full quiz pretest)]\n"+ " [-i] [-z] [-e] [-n]\n" + "\t-l username.. sets the D2L Web Service account (required)\n" + "\t-p password.. sets the password (required, see -w)\n" + "\t-w pwdfile.. reads the password from a file (required, see -p)\n" + "\t-c courseid.. sets the course identifier (required)\n" + "\t-a url.. sets the service address incl protocol and port (required)\n" + "\t-e.. enable database cache (Hibernate config is in the jar file\n" + "\t-o filename.. downloads the grades to a text file called \"filename\" " + " (recommended for download)\n" + " the default behaviour is to download a file to stdout\n"); filename.. name of the file where the password is stored (clear pasword.. in clear WebFMException.. mostly applicable on I/O errors (file does not exist, etc) / public static String getpwdfromfile(string filename) throws WebFMException {

11 String password = null; try { BufferedReader is = new BufferedReader(new FileReader(fileName)); if ((password = is.readline()) == null) throw new WebFMException("The file " + filename + " is empty"); catch (java.io.filenotfoundexception e) { throw new WebFMException("The file " + filename + " not found.", e); catch (java.io.ioexception e) { throw new WebFMException("The file " + filename + " could not be read.", e); return password; / Processes the command line options, initiates the toolkit ({@link ToolBox {@link ToolBoxCached) and passes control to {@link ExportGrade#exportColumn <p/>usage example:<br/> java -Djava.util.logging.config.file=log_hib.properties -jar webfm.jar ExportGrade -l ws.physquiz3 -w pwd_prod_3.txt -a -c x "Quiz 1" -e -o args.. command line arguments - execute with no arguments to get the full listing / public static void main(string[] args) { String passwd = null; boolean enablecache = false; String serviceurl = ""; String login = ""; String pwdfilename = ""; String courseid = ""; String downloadfilename = ""; String gradename = null; int i = 0; try { // process all command line arguments while (i<args.length) { String par = args[i]; if (par.length()>=2 && par.charat(0)=='-') { switch (Character.toUpperCase(par.charAt(1))) { case 'A' : i++; serviceurl = args[i]; break; case 'C' : i++; courseid = args[i]; break; case 'E' : enablecache=true; break; case 'L' : i++; login = args[i]; break; case 'O' :

12 i++; downloadfilename = args[i]; break; case 'W' : i++; pwdfilename = args[i]; break; case 'X' : i++; if (args.length>i){ gradename = args[i]; else { System.out.println("ERROR: Incorrect export option"); printusage(); return; break; i++; catch (Exception e) { System.out.println("ERROR: Incorrect command line parameters"); printusage(); return; if (login==null login.length()<1 (pwdfilename==null pwdfilename.trim().length()<1) serviceurl==null serviceurl.trim().length()<1) { System.out.println("ERROR: login, password (or password file name), " + " and serviceaddress required"); printusage(); return; if (courseid==null courseid.length()==0) { System.out.println("ERROR: course identifier is required "); printusage(); return; // initialize the toolkit try { if (pwdfilename!=null && pwdfilename.trim().length()>0) passwd = getpwdfromfile(pwdfilename); TokenManager.initialize(serviceUrl, login, passwd, 30000L, 200L); ToolBox toolbox = null; if (enablecache) { toolbox = new ToolBoxCached(serviceUrl); else { toolbox = new ToolBox(serviceUrl); // perform the export exportcolumn(toolbox, courseid, gradename, downloadfilename); catch (WebFMException e) { System.out.println("ERROR: " + e.getmessage());

13 System.out.print("Stack Trace: "); e.printstacktrace(system.out); finally { // place any cleanup code here / Exports the values of the given grade column in the given course as CSV file along with the corresponding usernames (already converted to UofG central login account toolbox.. toolbox (either cached or not courseid.. internal D2L identifier for the given gradename.. name of the grade object (column lable in the downloadfilename.. where the grade values should be WebFMException / static public void exportcolumn(toolbox toolbox, String courseid, String gradename, String downloadfilename) throws WebFMException { // retrive the toolkit services GradesTools gradestools = toolbox.getgradestools(); UserTools usertools = toolbox.getusertools(); OrgUnitTools orgtools = toolbox.getorgunittools(); GradeObjectInfo grade = null; CourseOfferingInfo course = null; // find the course object try { course = orgtools.getcourseoffering(courseid); catch (Exception e) { throw new WebFMException("The web services user account does not have access " + "to the specified course " + courseid,e); if (course==null) throw new WebFMException("The web services user account does not have access " + "to the specified course " + courseid); System.out.println("About to export Grade Book data for course " + courseid + " \"" + course.getname() + "\" " + course.getcode()); // find the grade object within the given course try { System.out.println("Looking up the grade objects in course \"" + courseid + "\""); grade = gradestools.getgradebyorgunitandgradename(courseid, gradename); if (grade==null) throw new WebFMException("No grades (columns) with the specified name " + gradename + " were found in the course " + courseid); else System.out.println("Found grade \"" + grade.getname()+ "\""); catch (UnsupportedOperationException e) { throw new WebFMException("Failed to lookup the grades. May not have "+ " access to the course.", e); catch (Exception e) {

14 throw new WebFMException("Failed to lookup the grades. May not have " + " access to the course.", e); // retrieve the grade values and store them in the text file try { String strgradeid = grade.getgradeobjectid().getid(); System.out.println("Retrieving grade values for \"" + grade.getname() + "\" with gradeid: " + strgradeid); List<GradeValueInfo> values = gradestools.getgradevaluesbygradeobject(courseid,strgradeid); if (values!= null) { System.out.println("Grade values for \"" + grade.getname() + "\" received (total count = " + values.size() + ")"); System.out.println("Redirecting the exported grades to file " + downloadfilename); PrintWriter out = new PrintWriter(new FileOutputStream( downloadfilename)); out.println("username," + grade.getname()); BigDecimal pointsnumerator; for (GradeValueInfo value : values) { if (value!=null && (pointsnumerator = value.getpointsnumerator())!=null) { out.println(usertools.getusernamebyuserid(value.getuserid().getid()) + "," + pointsnumerator.floatvalue()); //"," + Math.round(pointsNumerator.floatValue())); // use the line above to print rounded values out.close(); else System.out.println("The WS call for grade values for courseid " + courseid + " and gradeid " + strgradeid + " returned null"); catch (IOException e) { throw new WebFMException("Could not save the grade values to file " + downloadfilename, e) ; catch (Exception e) { throw new WebFMException("Could not retrieve grade values.", e) ;