A1- Overview of Previous Project

Size: px

Start display at page:

Download "A1- Overview of Previous Project"

Beatrice Price
5 years ago
Views:

1 Appendix to the Project Solar Flare: Solar Plant Data Acquisition & Visualization Samuel Caguana Instructor: Dr. Janusz Zalewski CEN 4935 Senior Software Engineering Project Florida Gulf Coast University April 25, 2014 A1- Overview of Previous Project The project taken over is the SolarFlare application maintenance as well as updating several features in the solar.fgcu.edu website. The architecture of the previous project is not changed, as shown in Figure A1. Figure A1: System Architecture The SolarFlare application is located on the Rock server and as of reception of the project, was not operational. The application should run on the server and continuously collect data from solarems.net as shown in Figure A2. Page 1

2 Figure A2: SolarFlare Flowchart. [1] The user shall start the application on the server and it shall automatically connect to solarems.net. The website shall then check the application credentials; the application shall then parse the html data and update the database hosted on the database location on the server. The solar.fgcu.edu website hosts all the data gathered by the SolarFlare application for it to be public. Solar.fgcu.edu connects to the database located on the Rock server, gathers the data, generates JSON files and from those files, generates the charts that are displayed on the website. This process is shown in Figure A3. Page 2

3 Figure A3: solar.fgcu.edu Flowchart. [1] The original application has been developed fully, as described above, but several issues are preventing it from running. SolarFlare could not successfully access the solarems.net website to collect data due to not having correct certificates for solarems.net. Another issue was some of the data collected were not correct. Several values such as the Ambient Temperature and the Energy Total were not read correctly as it was in the solarems.net website. Additionally, the solar.fgcu.edu website had an issue where it would take a several minutes to load the graph pages. The website page would be blocked while it refreshed and loaded the data to be displayed. Page 3

A2- New Objectives This maintenance of the SolarFlare application and the solar.fgcu.edu website has to include the repair of the connection between SolarFlare and solarems.

4 A2- New Objectives This maintenance of the SolarFlare application and the solar.fgcu.edu website has to include the repair of the connection between SolarFlare and solarems.net, correct the data acquisition of the invalid values from solarems.net as well as increase efficiency of solar.fgcu.edu. New features shall be attempted to be added to the current project during the maintenance without hindering the quality and functionality of the current state of the project. New objectives include making SolarFlare operational and maintaining it functional, adding and fixing linked content on solar.fgcu.edu and also making the website more efficient. The SolarFlare application shall be running continuously on the server as it was originally intended to. Several links to new publications about the FGCU s solar field need to be added as well as a feature to allow users who have accounts on solarems.net to be able to login via solar.fgcu.edu. More user control over displayed data shall be a new feature added to the website. This feature shall also improve on the efficiency of the website, as shown in Figure A4, by making it load faster due to information pulling from the database being at a minimum. Figure A4: Website- solar.fgcu.edu Page 4

A3- Implementation The SolarFlare applications class diagram is shown in Figure A5. This shows the connection of each class and the operation, as a whole, of the application.

5 A3- Implementation The SolarFlare applications class diagram is shown in Figure A5. This shows the connection of each class and the operation, as a whole, of the application. The application is a web crawler, HTML parser as well as a database manager. As shown in the figure below, the HTMLNavigator class is the web crawler section of the application. This section is responsible for connecting to solarems.net, navigating the site and retrieving data. The HTMLParser class is in charge of parsing through the data retrieved and converting it to text. The DBManager class is in charge of taking the text parsed and updating the database with the relevant information. Figure A5: SolarFlare Class Diagram. [1] In maintaining the SolarFlare application, the issues discovered were that the application did not connect to solarems.net and some of the data being gathered were not correct. The solarems website login screen is shown in Figure A6. Page 5

Figure A6: Website- solarems.net The application was giving the errors that it could not access solarems.net. These errors were due to SolarFlare not having the correct secure sockets layer (SSL) certificates for the website.

6 Figure A6: Website- solarems.net The application was giving the errors that it could not access solarems.net. These errors were due to SolarFlare not having the correct secure sockets layer (SSL) certificates for the website. The correct certificates were retrieved by an open source project InstallCert [2] developed by Sun Microsystems and the application could successfully access solarems.net. InstallCert is an open source java project to retrieve SSL certificates securely from a website an application accesses and stores them locally in the Java JDK security folders for the application to access when running. The InstallCert.java program is shown in Section A7. Page 6

7 Figure A7: Incorrect Data Gathered Figure A7 shows the issue of incorrect data being gathered. The Ambient column shows the ambient temperature of the area around the solar panels in degrees Celsius, but as shown, these temperatures are reaching 600 degrees Celsius, which is highly incorrect data. Also, the AC Energy column values were also off compared to the represented values in solarems.net. These issues were caused by parsing errors in the SolarFlare application code. The code section where the errors were created is shown in Figure A8. Page 7

8 public Double getvalue(string attributename) { Scanner byname = scan.usedelimiter("name\">"); int counter = 0; while (byname.hasnext()) { String currentline = byname.next(); if (currentline.contains(attributename)) { Scanner byvalues = new Scanner(currentLine); byvalues = byvalues.usedelimiter("value\">"); byvalues.next(); if (byvalues.hasnext()) { String valuestring = byvalues.next(); char[] c = valuestring.tochararray(); String strval = ""; for (int i = 0; i < c.length; i++) { if (c[i] == '0' c[i] == '1' c[i] == '2' c[i] == '3' c[i] == '4' c[i] == '5' c[i] == '6' c[i] == '7' c[i] == '8' c[i] == '9' c[i] == '.' c[i] == ',' c[i] == '-') { if (c[i] == ',') { continue; strval = strval + c[i]; else { break; try { double value = Double.parseDouble(strVal); return value; catch (Exception e) { return null; Figure A8: Parsing Error The line of code if (currentline.contains(attributename)).. parses the first line it comes across where it finds the attribute name it is searching. The issue with this is solarems.net has repeating attributes contained in other strings as shown in Figures A9 and A10. In Figure A9, the attribute AC Energy Total is being searched, but the SolarFlare application will stop at the first red outlined attribute and obtain the 5,661,057.0 when in actuality the data is Page 8

9 from the Aggregate AC Energy Total. The correct AC Energy Total is the second red outlined attribute in Figure A9. Figure A9: AC Energy Total Data Acquisition In Figure A10, the Ambient Temperature attribute is being searched by the SolarFlare application. Instead of the application returning 21.13, it will return 0 due to the parsing stopping at the first red outlined line and returning 0 because it does not retrieve any data. Page 9

Figure A10: Ambient Temperature Data Acquisition These issues of the incorrect data being retrieved were corrected by applying the section of code right after the if (currentline.

10 Figure A10: Ambient Temperature Data Acquisition These issues of the incorrect data being retrieved were corrected by applying the section of code right after the if (currentline.contains(attributename)).. line. This is shown in Figure A10 where after reaching the first line, SolarFlare will skip it and return the data for the actual attribute being retrieved. The correct results being stored in the database are shown in Figure A11. if(currentline.contains("ambient Temperature")){ currentline = byname.next(); if(currentline.contains("ac Energy Total") && counter == 0){ counter++; continue; Figure A11: Error Correction Code Several links to news publications were fixed in the toolbar on solar.fgcu.edu. Also, the feature of a user accessing and logging into solarems.net was added by adding a link to the website as shown on the top bar in Figure A3. Another issue that was resolved was that the loading times of pages with graphs were reduced. Previously these pages took several minutes to load. This was due to the website accessing and loading all the data points currently on the database. This was resolved by limiting Page 10

11 the query that resulted in loading selected data points. This allowed the website to retrieve that limited data points faster and more efficiently without causing the user to wait for the page to load. Figure A12 shows the PHP function that requests data from the database. It is passed attributes that are the selected date ranges of which records and how many records to retrieve. This makes the pages with the graphs load faster than it previously was due to these limiting factors of limiting the date range and the amount of records for displaying onto the graphs. Figure A12: Access Database Function In continuation of making the website more efficient, it was decided to let the user have more control over data displayed to them. Figure A11 shows the added section that provides the user with control over data. They are given the option of selection a certain time, day, month and year to monitor. They input the selected date and time and also select how many records from the inputted date to display. There is a section beside the input boxes that shows the guidelines for inputting the date the user would like to see displayed. Another option, also shown at the bottom of Figure A13, is the option for the user to select predetermined data ranges from the specific date the user entered. They have the option to load twelve hours of data, twenty four hours, one week, one month or six months worth of Page 11

records by selecting the corresponding option. The user also has the option to load all the records that are currently in the database to display on the graphs.

12 records by selecting the corresponding option. The user also has the option to load all the records that are currently in the database to display on the graphs. Figure A13: User Data Selection The code for the user input section is shown in Section A8. After the submit button has been selected, this JavaScript function fetches the data from the textboxes inserted from the user and checks whether they are null or not. If one textbox is left blank, the site shall revert back to the first data point held on the database and display data using that starting point. The site shall load the graphs starting at the user specified start date if the user entered the correct data inputs. Page 12

A4- Testing As shown in Figure A14, the SolarFlare application is continuously running on the server, as of now, and updating the database stored on the server with correct results.

13 A4- Testing As shown in Figure A14, the SolarFlare application is continuously running on the server, as of now, and updating the database stored on the server with correct results. A user can access solar.fgcu.edu from any browser in any location. Figure A14: Corrected Database Results Further results of testing resulted in correct results from the user data selection section of the website as shown in Figure A14. The user could enter a date to access and data records would be displayed on the charts. This testing is shown in Figure A15. The user enters the time, day, month, year and the total number of records to display from the entered start date and the graphs shall be accordingly updated. The testing example shows the input of July 25, 2011 at 5:30 pm and shows 3,000 records after this date. Page 13

14 Figure A15: User Data Selection Testing Further testing was implemented to test functionality for the user to select specific days and time to appear on the hash tables. The test cases included in this project are described below. Page 14

15 Test Case No 1. Objective: Successfully entry of Hour. Test Description: A correct input of Hour should lead to successful query of data. Inputs: Values from range of 0 to 23. Expected Results: Graphs should load with specified Hour and other related inputs. Result: Passed Test Case No 2. Objective: Successfully entry of Minute. Test Description: A correct input of Minute should lead to successful query of data. Inputs: Value 00, 15, 30 or 45. Expected Results: Graphs should load with specified Minute and other related inputs. Result: Passed Test Case No 3. Objective: Successfully entry of Day. Test Description: A correct input of Day should lead to successful query of data. Inputs: Integer value of a day in month. Expected Results: Graphs should load with specified Day and other related inputs. Result: Passed Page 15

16 Test Case No 4. Objective: Successfully entry of Month. Test Description: A correct input of Month should lead to successful query of data. Inputs: Values in range of 1 to 12 Expected Results: Graphs should load with specified Month and other related inputs. Result: Passed Test Case No 5. Objective: Successfully entry of Year. Test Description: A correct input of Year should lead to successful query of data. Inputs: Values in range of 2010 to current year. Expected Results: Graphs should load with specified Year and other related inputs. Result: Passed Test Case No 6. Objective: Successfully entry of Records to Display. Test Description: Correct input of Records to Display should lead to successful query of data. Inputs: Positive values Expected Results: Graphs should load with specified Records and other related inputs. Result: Passed Page 16

17 For the test cases where the user enters the values correctly, the graphs will show the data the user requested as shown in Figure A15. Test Case No 7. Objective: Unsuccessful entry of Hour, shown in Figure A16. Test Description: An incorrect input of Hour should lead to error message display Inputs: Values from outside the range of 0 to 23. Expected Results: Error message displays and revert back to default settings. Result: Passed Figure A16: Unsuccessful Hour Entry Page 17

18 Test Case No 8. Objective: Unsuccessful entry of Minute, shown in Figure A17. Test Description: An incorrect input of Minute should lead to error message display Inputs: Values not 00, 15, 30 or 45. Expected Results: Error message displays and revert back to default settings. Result: Passed Figure A17: Unsuccessful Minute Entry Page 18

Test Case No 9. Objective: Unsuccessful entry of Day, shown in Figure A18. Test Description: An incorrect input of Day should lead to error message display Inputs: Integer value of a day in month.

19 Test Case No 9. Objective: Unsuccessful entry of Day, shown in Figure A18. Test Description: An incorrect input of Day should lead to error message display Inputs: Integer value of a day in month. Expected Results: Error message displays and revert back to default settings. Result: Failed: Value check for date range of each month and user s entry not implemented. Figure A18: Unsuccessful Day Entry Page 19

20 Test Case No 10. Objective: Unsuccessful entry of Month, shown in Figure A19. Test Description: An incorrect input of Month should lead to error message display Inputs: Values outside range of 1 to 12 Expected Results: Error message displays and revert back to default settings. Result: Passed Figure A19: Unsuccessful Month Entry Page 20

21 Test Case No 11. Objective: Unsuccessful entry of Year, shown in Figure A20. Test Description: An incorrect input of Year should lead to error message display Inputs: Values outside range of 2010 to current year. Expected Results: Error message displays and revert back to default settings. Result: Passed Figure A20: Unsuccessful Year Entry Page 21

22 Test Case No 12. Objective: Unsuccessful entry of Records to Display, shown in Figure A21. Test Description: An incorrect input of Records should lead to error message display Inputs: Values less than 1. Expected Results: Error message displays and revert back to default settings. Result: Passed Figure A21: Unsuccessful Record Entry Page 22

23 A5- Conclusion The accomplishments made during the maintenance were that SolarFlare is now able to connect to solarems.net and is currently gathering the correct data, fixed broken links and added links to solarems.net website, made solar.fgcu.edu more efficient by not having to wait several minutes for pages to load and also added more user control of the data displayed. Several areas of improvement are to change the format of the text describing the variables for it to be more appealing to the user. Also, the gathering of data from solarems.net is correct, but an improvement can be made to display the newly gathered points to solar.fgcu.edu. Page 23

24 A7- InstallCert.java [1] V. Giannone, F. Velosa, Solar Flare: Solar Plant Data Acquisition & Visualization, FGCU, Fall [2] InstallCert.java Source, Page 24

25 A8- Increased User Control <script language="javascript"> function checktextboxes() { var userhour = document.getelementbyid('userhour').value; var userminute = document.getelementbyid('userminute').value; var userday = document.getelementbyid('userday').value; var usermonth = document.getelementbyid('usermonth').value; var useryear = document.getelementbyid('useryear').value; var userrecord = document.getelementbyid('userrecord').value; if(!userhour.match(/\s/)!userminute.match(/\s/)!userday.match(/\s/)!usermonth.match(/\s/)!useryear.match(/\s/)!userrecord.match(/\s/) isnan(userhour) isnan(userminute) isnan(userday) isnan(usermonth) isnan(useryear) isnan(userrecord)) { alert ("Value's left blank or incorrect input.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&userday=1&usermonth=4&useryea r=2010&userrecord=24#"; else { if(userminute == "00" userminute == "15" userminute == "30" userminute == "45"){ if(useryear >= 2010){ if(usermonth >=1 && usermonth <=12){ if(userhour >= 0 && userhour <=23){ if(userday >= 1 && userday <= 31){ if(userrecord >= 1){ document.location.href = "environmental.php?usertime=" + userhour + ":" + userminute + "&userday=" + userday + "&usermonth=" + usermonth + "&useryear=" + useryear + "&userrecord=" + userrecord; else { alert ("Make correct day input for your current month.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3 A45&userday=1&usermonth=4&userye ar=2010&userrecord=24#"; else{ Page 25

26 </script> alert ("Make correct day input for your current month.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&us erday=1&usermonth=4&useryear=2010&user record=24#"; else{ alert ("Hour needs to be between 0 and 23 inclusive.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&userday =1&usermonth=4&useryear=2010&userrecord=24# "; else{ alert ("Month needs to be between 1 and 12 inclusive.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&userday=1&use rmonth=4&useryear=2010&userrecord=24#"; else{ alert ("Year needs to be 2010 or after.\nreverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&userday=1&usermont h=4&useryear=2010&userrecord=24#"; else{ alert ("Minute needs to either be 00, 15, 30 or 45.\nReverting to default start date."); document.location.href = "environmental.php?usertime=18%3a45&userday=1&usermonth=4&u seryear=2010&userrecord=24#"; Page 26

27 A9- InstallCert.java /* * Copyright 2006 Sun Microsystems, Inc. All Rights Reserved. * */ /** * Use: * java InstallCert hostname */ import javax.net.ssl.*; import java.io.*; import java.security.keystore; import java.security.messagedigest; import java.security.cert.certificateexception; import java.security.cert.x509certificate; /** * Class used to add the server's certificate to the KeyStore * with your trusted certificates. */ public class InstallCert { public static void main(string[] args) throws Exception { String host; int port; char[] passphrase; if ((args.length == 1) (args.length == 2)) { String[] c = args[0].split(":"); host = c[0]; port = (c.length == 1)? 443 : Integer.parseInt(c[1]); String p = (args.length == 1)? "changeit" : args[1]; passphrase = p.tochararray(); Page 27

28 else { System.out.println("Usage: java InstallCert <host>[:port] [passphrase]"); return; File file = new File("jssecacerts"); if (file.isfile() == false) { char SEP = File.separatorChar; File dir = new File(System.getProperty("java.home") + SEP + "lib" + SEP + "security"); file = new File(dir, "jssecacerts"); if (file.isfile() == false) { file = new File(dir, "cacerts"); System.out.println("Loading KeyStore " + file + "..."); InputStream in = new FileInputStream(file); KeyStore ks = KeyStore.getInstance(KeyStore.getDefaultType()); ks.load(in, passphrase); in.close(); SSLContext context = SSLContext.getInstance("TLS"); TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorith m()); tmf.init(ks); X509TrustManager defaulttrustmanager = (X509TrustManager) tmf.gettrustmanagers()[0]; SavingTrustManager tm = new SavingTrustManager(defaultTrustManager); context.init(null, new TrustManager[]{tm, null); SSLSocketFactory factory = context.getsocketfactory(); System.out.println("Opening connection to " + host + ":" + port + "..."); SSLSocket socket = (SSLSocket) factory.createsocket(host, port); socket.setsotimeout(10000); try { System.out.println("Starting SSL handshake..."); socket.starthandshake(); socket.close(); System.out.println(); System.out.println("No errors, certificate is already trusted"); catch (SSLException e) { System.out.println(); e.printstacktrace(system.out); Page 28

29 X509Certificate[] chain = tm.chain; if (chain == null) { System.out.println("Could not obtain server certificate chain"); return; BufferedReader reader = new BufferedReader(new InputStreamReader(System.in)); System.out.println(); System.out.println("Server sent " + chain.length + " certificate(s):"); System.out.println(); MessageDigest sha1 = MessageDigest.getInstance("SHA1"); MessageDigest md5 = MessageDigest.getInstance("MD5"); for (int i = 0; i < chain.length; i++) { X509Certificate cert = chain[i]; System.out.println (" " + (i + 1) + " Subject " + cert.getsubjectdn()); System.out.println(" Issuer " + cert.getissuerdn()); sha1.update(cert.getencoded()); System.out.println(" sha1 " + tohexstring(sha1.digest())); md5.update(cert.getencoded()); System.out.println(" md5 " + tohexstring(md5.digest())); System.out.println(); System.out.println("Enter certificate to add to trusted keystore or 'q' to quit: [1]"); String line = reader.readline().trim(); int k; try { k = (line.length() == 0)? 0 : Integer.parseInt(line) - 1; catch (NumberFormatException e) { System.out.println("KeyStore not changed"); return; X509Certificate cert = chain[k]; String alias = host + "-" + (k + 1); ks.setcertificateentry(alias, cert); OutputStream out = new FileOutputStream("jssecacerts"); ks.store(out, passphrase); Page 29

30 out.close(); System.out.println(); System.out.println(cert); System.out.println(); System.out.println ("Added certificate to keystore 'jssecacerts' using alias '" + alias + "'"); private static final char[] HEXDIGITS = " abcdef".toCharArray(); private static String tohexstring(byte[] bytes) { StringBuilder sb = new StringBuilder(bytes.length * 3); for (int b : bytes) { b &= 0xff; sb.append(hexdigits[b >> 4]); sb.append(hexdigits[b & 15]); sb.append(' '); return sb.tostring(); private static class SavingTrustManager implements X509TrustManager { private final X509TrustManager tm; private X509Certificate[] chain; SavingTrustManager(X509TrustManager tm) { this.tm = tm; public X509Certificate[] getacceptedissuers() { throw new UnsupportedOperationException(); public void checkclienttrusted(x509certificate[] chain, String authtype) throws CertificateException { throw new UnsupportedOperationException(); public void checkservertrusted(x509certificate[] chain, String authtype) throws CertificateException { this.chain = chain; tm.checkservertrusted(chain, authtype); Page 30

31 Page 31

Spark 源码编译遇到的问题解决 Spark 大数据博客 -

Spark 源码编译遇到的问题解决 Spark 大数据博客 - 1 内存不够 [ERROR] PermGen space -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors,re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging.