National Optical Astronomy Observatory Science Portal. Software Requirements Document for Release 1.0 Version 1.1 (02/08/06)

Transcription

1 National Optical Astronomy Observatory Science Portal Software Requirements Document for Release 1.0 Version 1.1 (02/08/06)

2 Table of Contents...2 Executive...3 High Level Design...3 Science Requirements...4 The and Subtasks:...10 Functional Requirements Prototype DEMO Clean-up Discovery: GESTALT Discovery Explorer: Rebuild Calendar Monitor VO Services Discovery: View-now GIF/PNG Previews from Archives Access: Rebuild Staging Global: Minimize/Reduce Javascript Global: Rebuild Persistency Global: More browser support Understanding Discovery Discovery: NOAO-Sky Discovery: NOAO-Time Discovery:NOAO-VOSIAP Discovery: NOAO-VOADQL Access Access: HTTP Links to Data Access: RSYNC Service Access: Staging via CURL/HTTP Analysis Analysis: WESIX Analysis: SWarp Analysis: IRAF Webservices Analysis: Aladin Analysis: Group Image Statistics and Plotting Documentation Documentation Users Users: Portal User Registration Users: Portal User Login Certificate Service Call Secure HTTPS SIAP/ADQL Calls to NSA... Executive A''portal'' or entrance-way to astronomical data, tools, services, and documentation is just as it sounds: the front door. It's design should enable the astronomer to discover, access

3 understand, and analyze astronomical data. In this context, the NOAO-VO portal is designed to seamlessly link the the NOAO E2E system with the Virtual Observatory (VO). There exist dozens of astronomical archive "portals" today. They vary in their appearance, analysis tools, data access mechanisms, etc. But there are many similarities: (1) the 10 major astronomical image archives examined use web browsers (as opposed to GUI clients); (2) Few provide analysis services on their data (and those that do are often simple links to Aladin and Oasis); 3) Few provide access, analysis, and knowledge of data outside their archive; 4) None are portals to time domain data. Current astronomical portals are evolving quickly. DataScope, a VO-specific tool, has recently come online which allows users to search literally hundreds of online datasets (both catalogs and images). The (IRSA) has formed a data-access coordination with the Multimission A (MAST). IRSA has also "incorporated" non-infrared data in its archive (via the Sloan Digital Sky Survey --SDSS). The SDSS has begun to link its web service analysis tools directly to their data (via the Spectrum Service Archive protocol). The ESO archive, which was previously a European/UK proprietary archive, has gone public to the world. The large-glass telescopes are building their own archives (KOA at Keck and the GSA at Gemini). The future NSA R3 data release in the context of the E2E system, alongside a newly designed NOAO-VO portal, can place the NOAO amongst the worlds most utilized astronomical archives (e.g., MAST, IRSA, SDSS). The NOAO-VO Portal Software Design is made with the following goals in mind: (1) To be innovative by providing tools and services which are needed and do not yet exist in the currently available archives; (2) To use solutions which exist today when appropriate (no re-inventions) and choose those new innovations in the context of limited resources; (3) To be completed in a relatively short time-scale (but allowed to evolve). High Level Design The portal is broken up into four pieces: (1) Discovery, (2) Access, (3) Understanding, (4) Analysis. These are independent aspects of scientific analysis (more may exist--e.g. Data Processing). The Portal will provide easy navigation through these four pieces, as well as information to apply them to ones everyday research (tutorials, examples, etc). The Portal should provide one-stop shopping for ground based astronomical research. In Figure 1, we highlight lower-level design requirements under each of these four major headings. Notice how each Portal requirement is designed to lead to a scientific product (e.g., Discovery should a user to some real data while Analysis will lead to some real science).

4

5 Figure 1: The High-Level Functional Design of the Portal Science Requirements The science requirements for the E2E system are the starting point for building the detailed functional requirements of the Portal. In the following table, we show these requirements and discuss briefly whether they have a direct relation to the Portal. In the cases where no obvious relationship to the portal is present, the Science Requirement wording has been shortened.

6 Requirements for E2E V1: Release for full operation July 1, 2006 Version: December 9, The E2E system will ingest... No obvious relationship to the Portal 2. The E2E system will provide access to the survey data products that currently reside in NSA R2 and will provide access to similar NOAO survey data products as acquired in the future. Access here is via the Portal through the NSA. The Portal will treat the R2 and R3 survey data similarly. For the E2E V1 system, the Portal may consider R2 as a separate NOAO Archive (separate from R3) It is noted that the Portal Prototype already provides access to the NSA R2, which has a functioning and publicly available DB querying service and SIAP service. Access to both R2 and R3 survey data goes beyond the Portal and includes standard VO services. 3. The E2E system will be delivered with the raw data in DCI storage... No obvious relationship to the Portal. 4. The E2E system will provide raw data and metadata to the NOAO pipelines... No obvious relationship to the Portal. 5. The E2E system will be able to identify data objects within its holdings that match queries delivered to the archive. The availability of such metadata will be subject to approved policies. The Portal will be a major supplier of those queries to the E2E system from external users. The Portal will query standardized services exposed by the VO Interface (VOI) of the E2E system. The VOI component of the E2E system will return metadata information in a standardized format (VOTable). 6. The E2E system will allow for users to download, via the internet, selected data identified through such queries. The availability of such data will be subject to approved policies. Included in the VOTable returned by queries will be a link to a data file or a link to a data collection. The Portal will supply these links to the user. The Portal will stage multiple data files for easier access. Access to proprietary data will be handled through the E2E Security Component, in which only a authenticated users have access to proprietary data. 7. External queries and external data transport will be accomplished through VO-approved standard protocols, where they exist. This is closely (too closely?) related to Sci Req The E2E system will support queries of and return values for the following parameters: 1. PI Name 2. Proposal ID

7 3. Telescope & Observatory 4. Instrument/Detector 5. Filter or other essential configuration information 6. RA (specified RA falls in image area) 7. DEC (specified DEC falls in image area) 8. Integration Time, Epoch, Date-Obs, UT 9. Reduction level (e.g. Level 1 4) 10. Data type (a 'classification' of the data product, e.g. Observation/Flat/Bias, etc.) 11. Data quality defined by some combination of image quality and depth (reduced data only) 12. Unique identifier The Portal will allow for queries against these parameters. In SIAP form, the precise WHERE clause could be less general than the ADQL form. This information will be displayed to the user after a Portal query is made. 9. In addition to the items above, the E2E system will store and return values for the following parameters and information, if they exist: 1. WCS 2. Focal plane image (CCD or MOSAIC) corners 3. PSF 4. Zeropoint 5. Background Noise Level (or Photometric Depth) 6. Background Level 7. Estimate of Uncertainty in the Photometry 8. Ancillary and associated data product unique Ids 9. Coverage in wavelength These will not be queryable by the Portal, but will (can) be returned by the ADQL and SIAP queries. 10. The E2E system will allow queries for, and downloads of, proprietary and nonproprietary data and metadata according to approved policies. This is closely related (too close) to Sci Req 6). 11. The E2E system will hold a database of registered users, accessed through a registration service. This DB will store a minimum of name, , and organization. The E2E system will determine authorizations for these users and supply the portal with the necessary security certificates. The Portal will access the Registration service and supply it with the necessary information to newly register a user with the NSA. The Portal will call an Authorization service to obtain the proper certificate which will be used by the Portal when queries to the E2E system are made. 12. Raw and processed data from a scheduled observing run shall be available to author... No relationship to the Portal.

8 13. Data ingest must keep up with data product generation. Data should be ingested... No relationship to the Portal. 14. The E2E system will keep statistics and display or print reports on the quantity of data ingested per day, month, and year, and the time delay between a data request and the delivery of data to satisfy the request. The Portal will supply the registered user with an access link to their data. Is this link (once active) considered delivery? Or is successful data-transfer considered delivery? If the latter, the Portal will need to report the start time of the data transfer. While the NSA will need to report the end time (as determined by the NSA) of the data transfer. (Or is there some other solution?) 15. Excluding network latencies, user queries of the database should produce displayable results within 10 seconds 90% of the time. Goal: 3 seconds Full sky queries are not likely to meet this demand. The Portal Prototype caches VO data for the quickest possible response times. Thus, the Portal could use a cached version of the NSA archive for the quickest response. However, SIAP/ADQL access could meet most of the timing requirements (although this is not the case for current SIAP servers). 16. The E2E system will support bulk data downloads. Three simultaneous users of the NSA will be able to achieve download rates of 500 MB/hour average. Goal: 1 GB/hour. The Portal will support data staging. 17. Data objects must be protected (from the time they leave the instrument... No relationship to the Portal. 18. A pipeline-compatible observing protocol will be documented and distributed... No relationship to the Portal. 19. The MOSAIC Pipeline will process data taken in the pipeline-compatible... No relationship to the Portal. 20. MOSAIC pipeline processing will include removal of instrumental and... No relationship to the Portal. 21. In addition to individual calibrated science frames, the pipeline will produce... No relationship to the Portal. Questions: How will the E2E system serve these data? What is the Data Model for these data? 22. For processed images, the pipeline will provide identification information...

9 No relationship to the Portal. 23. The pipeline will be overseen by an operator, who will make decisions... No relationship to the Portal. 24. The E2E system will include pipeline operator interface tools and monitoring... No relationship to the Portal. 25. The E2E system will include tools that monitor and record pipeline processing... No relationship to the Portal. 26. The MOSAIC pipeline will process raw data to completion within 22 hours of the raw data being made available to the pipeline. No relationship to the Portal. 27. The pipeline will process the accumulated DCI holdings that come from the... No relationship to the Portal. 28. The portal will allow queries submitted through SIAP, SQL and/or web-form interfaces. This is closely (too close) related to Sci Req 5. The SQL queries will actually be in the form of the VO standard ADQL. The E2E NSA VO Interface will translate this ADQL to SQL. The queryable parameters are defined in Sci Req 8. The return parameters can be specified by the user (ADQL) or set to some default (the combination of Sci Req 8,9). 29. The portal will provide interactive graphical (position-based and time-based), and text (sortable-list-based) visualization of metadata for the results of queries. The Portal will provide a sky-view of the E2E holdings along with available VO data. Users will be able to graphically chose an image or query a region of the sky. Users will be able to search through time via a monthly calendar and timeline. Users will see available NSA R2/R3 data by month and day and be able subselect on Survey Name (if a survey), telescope name, instrument name, or PI name. Users will be able to specify a region in the sky for the time-based queries. Users will be see image metadata for selected objects. Users will be able to Search and Sort the metadata for a subselect on previously selected objects. 30. Datasets supported by the portal include all NSA holdings plus all registered VO-compliant data that meet a minimum of requirements for items 27 and 28 (i.e., return appropriate metadata to be visualized by the portal tools). The Portal will monitor VO compliant SIAP servers for data that can be plugged into the time and sky visualization. SIAP servers that cannot be utilized in the Time and/or Skyviews, will be accessible through a smart

10 client SIAP which is tailored to each SIAP server and enables easy access to these external services. The Portal will monitor external SIAP servers daily for their status and report that status back to the User. 31. Subsets of query results can be selected for data display and identifiers for these selected results can be moved to a temporary cache. The Portal will store for each user a user-defined shopping cart which may be used for data access and analysis. If the user is registered with the Portal and logged into the Portal, their shopping cart can persist beyond a single access session. Non-registered and non-logged-in users will lose their cache at the end of a session. The Portal will utilize cookies for non-registered and non-loggedin users. These cookies will disappear when the user shuts down their browser. 32. Data identified in the temporary cache may be submitted to any of the supported web-service operations, including cutout generation, WESIX, SWARP, Gim2D, imcombine, and an image statistics tool. The Portal will utilize external and internal (to the NOAO) services which operate on an image. The Portal will allow users to download textual results via the external webservice where appropriate. The Portal will download and stage and newly created image data for a user in the Portal staging area. The Portal will monitor these services for their status and report this status to the User. 33. Data identified in the temporary cache can be displayed using Aladin or OASIS. The Portal will utilize an internal version of Aladin to mitigate external server downtimes. 34. Any data identified in the cache, including the results of the operations listed above, may be downloaded to the users desktop. The Portal Staging Area will collect external image data for easy access by the User. Textual output from can also be stored here. 35. The portal will allow for NSA User registration and logins. The Portal registration and login will be be utilized for persisting the Portal shopping cart beyond a browser session and to retrieve an authorization certificate from the E2E Security Service. 36. Searchable on-line documentation for the E2E system will include users guides, tutorials, and FAQs. The Portal will provide these to the user. 37. A help-desk will provide a mechanism to answer questions not addressed by the static documentation. Queries to the help-desk will be acknowledged within 5 minutes of their receipt. The Portal will utilize the E2E Help Desk.

11 38. An internal issue tracking system will be used to report problems and their resolutions. The Portal will utilize this internal bug tracking system. 39. Additional documentation provided with the system includes operators manuals and software maintenance instructions. The Portal will provide installation and operations manuals

12 The and Subtasks: In the following sections, we outline and specify the tasks and subtasks needed to meet the science requirements with a direct relationship with the NOAO NVO Portal. There are, in effect, seven areas of effort: 0. remaining for the NOAO NVO Prototype Release in January These include bug fixes, improvements, and and unfinished work that was supposed to be done for the January release. 1. Understanding This effort includes tutorials, handouts, newsletter articles, announcements, etc. These written material are designed to encourage and assist the astronomical community to use the NOAO NVO Portal and related NOAO NVO tools. 2. Data Discovery These include additional actions which allow users to discover NOAO and NVO data. 3. Data Access These include additional actions that allow users to access NOAO and NVO data. 4. Data Analysis These include connecting the NOAO NVO Portal to NVO analysis tools. 5. Documentation These are the need documentation efforts for this software project, including architecture design documents, installation and operations documentation, etc. 6. Users These are the needed infrastructure to support known and registered astronomical users to the NOAO NVO Portal via the NSA. In the next sections, we go through the Function Requirements and the tasks and sub-tasks that are required effort to meet the Science Requirements for the NOAO NVO Portal. Each Functional Requirement is assigned a PRIORITY which governs whether it will be assigned during the E2E V1 project. All PRIORITY = Functional Requirements will have tasks and subtasks assigned to achieve the necessary goal. Those requirements with PRIOITY = IUM can be assigned, but not all will. There is a STABILITY measure, defining how stable the requirement definition is. Additionally, there is a level of DIFFICULTY and RISK assigned to each Functional Requirement. Each Functional Requirement has a short SUMMARY explaining concisely exactly what the final product will allow for in the NOAO NVO Portal. When possible, each Functional Requirement is then mapped backwards to specific Science Requirements. Following this is a list of TASKS which will be assigned to software developers working on the Portal. DEPENDENCIES to requirements within the Portal development are noted and PRECONDITIONS to functions that exist outside of the Portal development are noted. The next table goes into a bit more detail for each of these definitons:

13 Functional Requirement Status Science Reqs Dependency Precondition Represents the general action described in the use case. Extremely important to the client and must be built. Important to the client but the system can function and meet the basic needs of the users without it. Not important to the client but would be a nice feature to have if resources are available. Requirements are close to exactly describing the needs of the client. Requirements are somewhat close but need further information or explanation from the client. Requirements are vague and need more detail from the client. Extremely difficult to implement. Somewhat difficult to implement. Not difficult to implement. Extremely risky to the project if implemen. fails. Somewhat risky to the project if implementa fails. Not risky to the project if implementation fails. Either a percentage of work completed or: Proposed Initial requirements status. Approved Validated by the client as accurate. Built Functionality is built by the developer/architect. Tested -- Functionality has been tested internally. Deployed Fully working. Removed Functionality was removed from the scope of the project for any reason. The summary is a one or two sentence description of the use case. A list of Science Requirements (from the official list of approved Science Requirements) for trace-ability. A list of tasks (detailed) to meet the Functional Requirement. A list of NOAO NVO Portal Dependencies that must be completed before this task can be assigned. Pre-conditions describe things that must be in place prior to executing the basic course of events. The basic course of events is the process description of a user interacting with the system in order to accomplish the goals of the use case. The alternative path is any process that deviates from the basic course of events to either accomplish the goal of the use case or a related goal. criteria are relevant, measurable, and specific tests that can be executed to ensure that the goals of the use case are met.

14 Functional Requirements 0 Prototype Demo Clean-up 1 Understanding 1.1 NOAO Registry Stuffing 1.2 ADS/VO Data Product Registration 1.3 VO Information Gathering 2 Discovery [maps to Sci Req #s (5), 7, 8, 9, 15, 28, 29, 30, 31] 2.1 Dynamic Services 2.2 Graphical Wandering NOAO-Sky NOAO-Time 2.3 Focused Queries NOAO-VOSIAP NOAO-VOADQL 2.4 Registry Queries 3 Access [maps to Sci Req #s 2, 6, (10), 15, 16, 30, 34] 3.1 HTTP links to data 3.2 RSYNC the NSA 3.3 Staging via WGET/CURL 3.4 SOAP-based Image Retrieval 4 Analysis [maps to Sci Req #s 30, 32, 33] 4.1 WESIX 4.2 Swarp (or IRAF equivalent) 4.3 IRAF Web-services Cut-out Service and Quicklook IRAF GIM2D Service IRAF WREG Service IRAF IMCOMBINE Services 4.4 Aladin 4.5 Group image statistics and plotting VOPlot VOStat 5 Documentation 5.1 Online Documentation [maps to Sci Req #s 36,37,38,39] NOAO-Sky NOAO-Time NOAO-Access NOAO-Analysis 5.2 Online Tutorials (per above) 5.3 Help Desk Access 6 Users [maps to Sci Req #s 11, 35] 6.1 Portal User Registration 6.2 Portal User Login 6.3 Certificate Service Call 6.4 Secure HTTPS SIAP Calls 6.5 Secure ADQL Calls

15 0. Prototype DEMO Clean-up 0.1 Discovery: GESTALT Status 50.00% The GESTALT service allows arbitrary footprints (circles, squares, image WCS, etc) to be drawn and for images lying within that shape to queried and their metadata returned. This service runs at Pitt and will be connected to the Portal. Sci Reqs Pop-up a GESTALT window for user to specify footprint Read GESTALT returned VOTable with image Ids Grid-view of GESTALT image list Dependency Precondition GESTALT is fully functional 1. User access portal website and clicks the GESTALT 'archive' link. 2. A GESTALT window pops up, asking the user to specify a footprint (e.g., circles, squares, WCS's, etc). 3. User enters footprint information. Window closes automatically. 4. Check-box next to GESTALT Wire-frame 'archive' turns 'on' and the GESTALT footprint wire-frames appear on the NOAO Sky. 5. Check-box next to GESTALT 'archive' SEARCH turns 'on'. User clicks 'New Search' 6. Data products that exists within the GESTALT foorprint are listed. (GESTALT returned a VOTable of these Portal image ids).

16 0.1 Discovery: GESTALT 0.2 Discovery Explorer: Rebuild Calendar Status 10.00% The Calendar code has become a bit unwieldy. Rebuild while maintaining integrity of the current Portal infrastructure. Sci Reqs Dependency Precondition 0.3 Monitor VO Services

17 0.3 Monitor VO Services Status 0.00% The Portal uses a finite number of external services and archives. These external IPs should be 'pinged' for their status as well as 'tested' (e.g., via a test image download) regularly. The Portal should display the status of these services. This service should be very general and may run as its own system external to the Portal. Sci Reqs Dependency Precondition 1. User clicks 'View Monitor' 2. A list of services and archives is seen with a red/green/yellow status color. Red means the link is down. Yellow means the link is up, but the service is not working. Greem means the link and service/archive is working. 0.4 Discovery: View-now GIF/PNG Previews from Archives Status 0.00% Many archives (SDSS, NSA) provide pre-made GIFs/PNGs which allow for immediate viewing of the image (as opposed to Aladin).

18 0.4 Discovery: View-now GIF/PNG Previews from Archives Sci Reqs Make a new column in the Grid-view to pop-up the PNG/GIF Find out where the GIF/PNG exists at the service (SIAP?) Dependency Precondition PNGs/Gifs/Jpgs exist 1. User clicks 'View Now' 2. A browser window pops up with the preview. 0.5 Access: Rebuild Staging Status 20.00% Sci Reqs Dependency

19 0.5 Access: Rebuild Staging Precondition 0.6 Global: Minimize/Reduce Javascript Status 0.00% Some Javascript is needed (for visuals). Others (such as the display of the Title and Header on each page) are simply not necessary and can be done more elegantly using Ruby on Rails. Sci Reqs Remove JS for Header displays Dependency Precondition

20 0.6 Global: Minimize/Reduce Javascript 0.7 Global: Rebuild Persistency Status 10.00% The sky coordinates (and time) are stored in an object that is passed around the Portal (e.g., when switching from time to sky). It would be better to store such information in a DB with a time-stamp. Users can then view their 'history' and/or go back-in time to a specific search result. Sci Reqs Build the DB Map DB session id and time to Actions/Controllers Dependency Precondition

21 0.8 Global: More browser support Status 0.00% Must get IE working. Safari cleaned up. Sci Reqs Dependency Precondition

22 1. Understanding

23 2. Discovery Discovery: NOAO-Sky / (some below are medium priority) Status 0.00% Must be able to visualize image wire-frames on the sky of current NSA data, proprietary NSA data, and additional VO archive data. Sci Reqs 9 (Image/CCD Corners, PropID), 11, 15, : Add NSA R3 as a new Portal DB : Automate NSA R3 wire-frame caching (uses image corners) Make module (RMagick? Javascript? What else?) to draw images Contingency: Use GESTALT locally : Generate R3 Survey Data wire-frames (uses image corners) : Automated wire-frame generation of wire-frames for proprietary data from ADQL service call after user login Connect module to Portal via an 'Archive' button Call Module made in or GESTALT : Add additional VO Data as wire-frames and to DB Search the VO for possible candidates Edit SIAP config file Edit which code files? Make wire-frames : Allow user to upload a list of Positions and Times to display on the NOAO-Sky (i.e., an asteroid track) Allow user to 'specify' marker. 'Google-maps' Flag? Ruby upload module Portal must highlight images which overlap in position and time on the NOAO-Sky with a user list SQL query and image flag placement Dependency (User Logins) and (Secure ADQL Queries)

24 2.3.1 Discovery: NOAO-Sky Precondition NSA R3 and NSA R3 Survey Data Cache provided via E2E SIAP For 2.3.1: 1. User access portal (but does not log on). 2. User sees NSA R3 wire frames (plus NSA R2?) on the NOAO -Sky. These are the publicly available wire-frames for NSA R3. 3. User logs on as a PI. 4. User clicks 'PI' wire-frame archive and sees his/her own archive data on the NOAO-Sky Discovery: NOAO-Time Status 0.00% Must be able to visualize image data in the time domain. Proprietary data must show up in the calendar automatically. Sci Reqs 8 (Image date-obs),11, 15, Enable R3 Survey data into the NOAO-Time Convert whatever R3 time into Portal time (decimal year) : Automate NSA R3 proprietary display in the Calendar and Timeline via ADQL service call after user login Add 'Archive' button to Calendar for proprietary data Enable additional VO Data which have date information

25 2.3.2 Discovery: NOAO-Time Search the VO for possible candidates Allow users to upload a list of positions and times for display in the Calendar and Timeline Clarify file format and length limits Dependency and Precondition NSA R3 SIAP Service Exists For 2.3.1: 1. User access portal (but does not log on). 2. User sees NSA R3 wire frames (plus NSA R2?) on the NOAO -Sky. These are the publicly available wire-frames for NSA R3. 3. User switches to NOAO-Time view and checks the NSA R3 archive. User sees the NSA R3 data in time (for those in the Ra/Dec range of the previous viewing window on NOAO-Sky). 3. User logs on as a PI. 4. User clicks 'PI' wire-frame archive and sees his/her own archive data on the NOAO-Time Discovery:NOAO-VOSIAP Status 80.00% Must be able to query NSA and VO SIAP Servers (via a browser). INPUT

26 2.4.1 Discovery:NOAO-VOSIAP parameters ti SIAP services should be tailored to each archive. Output table should contain active links to the data products. Sci Reqs (5), 7, 8, 9, 11, : Finish Portal SIAP Form Generator and Call Fix or drop rico.js Create Archive-tailored configuration file Fix VOTable parser Error checking : Add NSA R3 SIAP Service call Use 'ICD' to NSA SIAP and edit config file : Add VO SIAP Servers to match NOAO-Sky and Time Edit config file Dependency Precondition NSA R3 SIAP Service Exists, 'ICD' to R3 SIAP exists 1. User goes to NOAO NVO Portal and chooses the Form (then SIA) page. 2. User highlights NSA R3. 3. User specifies any allowable input requirements (these follow the new NSA R3 SIAP data model). 4. User clicks 'submit' 5. User gets returned a list of data products. The links to the images are active Discovery: NOAO-VOADQL

27 2.4.2 Discovery: NOAO-VOADQL Status 0.00% Must be able to build and send ADQL queries to NSA. Sci Reqs (5), 7, 8, 9, 11, : Create ADQL form generator and service call page : Create or use ADQL/s to ADQL/x translator : Add NSA R3 ADQL Service call Configuration file Dependency Precondition NSA R3 ADQL Service Exists, 'ICD' to ADQL call exists 1. User goes to NOAO NVO Portal and chooses the Form (then ADQL) page. 2. User highlights NSA R3. 3. User utilizes a graphical SQL builder (similar to that of Open SkyQuery) to fill in a SQL query. User can also type query by hand. The User must be able to see and understand the ADQL data model. 4. User clicks 'submit' 5. User gets returned a list of data products. The links to the images are active.

28 3. Access 3.1 Access: HTTP Links to Data Status % Portal provides active links of image data to the user. Portal should have access to the FITs images as well as Preview PNGs (or GIFs or whatever). Sci Reqs 2, 6, (10), 15, 30, 34 Dependency Precondition Links to NSA R3 Data exist via SIAP; NSA R3 SIAP exists 1. User searches for data at the NOAO NVO Portal. 2. User sees a list of data products with active links. 3.2 Access: RSYNC Service Status 0.00%

29 3.2 Access: RSYNC Service Registered Portal Users can set up RSYNC daemons for time-based queries and downloads. For example, a user specifies that he/she wants to actively update MOSAIC data as it becomes publicly available. Sci Reqs 15, Set-up an RSYNC Server on the Portal Connect Portal User DB to RSYNC DB Allow users to Stage data to the RSYNC Servers Allow users to specify time-based queries which automate data staging for the user into the RSYNC area. Dependency 6.1 and 6.2 Precondition 3.3 Access: Staging via CURL/HTTP Status 0.00% Portal needs to access NSA R3 data via active links Sci Reqs 2,6, (10), 30, Rebuild Staging

30 3.3 Access: Staging via CURL/HTTP Dependency Precondition See also Analysis 4.1 Analysis: WESIX Status 90.00% Connecting the NSA R3 to WESIX allows for instant catalog creation Sci Reqs 32, Link NSA R3 to WESIX Install WESIX at the Portal to Mitigate image transfer times Dependency Precondition WESIX exists. NSA R3 SIAP exists. Pipeline processed data allows for fixed aperture photometry. 1. User adds an image to his/her cart. 2. User switches to NOAO-Analyze

31 4.1 Analysis: WESIX 3. User chooses WESIX (via a 'click' in the proper column). 4. User is redirected to the WESIX homepage, with the URL already filled in with the correct data product. 5. User runs WESIX. 4.2 Analysis: SWarp Status 0.00% Swarp allows multiple images to be registered to the same pixelization. Swarp returns a new image of a user-specified sub-region or of the total combined images. Swarp functionallity is also available via multiple calls to IRAF services and the same functionality might be available via 4.3 (below). Sci Reqs 32, Portal must indicate to user overlap regions Portal must allow the user to sub-select on PSF Portal must allow user to specify which images to Swarp and to specify what region to center on and which image to base the Swarp on Portal must Stage resultant images Portal must have error reporting back from the Swarp service Portal must return overlap region as a cutout data-cube and stage as well as show image metadata in appropriate queries.

32 4.2 Analysis: SWarp Portal must allow user to add new image to a cart for analysis and access. Dependency 4.3 (cut-out service) Precondition Swarp web-service exists. Cut-out web service exists. 4.3 Analysis: IRAF Webservices Status 0.00% User can specify regions of an image and retrieve a cutout (fits) as well as a quick-look (jpg/png/gif). Allow users to run morphology services on NVO data (e.g., GIM2D). Allow users to re-register pixels in images and/or combine images that have been registered. There are interdependencies in these services. For example, the cutout service will need image registration and imcombine to be finished for greater functionality. Sci Reqs 32,34 The Portal access to these IRAF webservice tasks will be designed early on in the project so that the individual services can be built in a timeline fashion. We expect integration between these services and the NOAO NVO portal to be difficult Porta l must allow user to specify what region (type and size) for the

33 4.3 Analysis: IRAF Webservices cutout on a user specified image Portal must stage and display image metadata for the cutout Portal must allow image to be placed in cart by user for access and analysis Portal must be able to display Cut-out quick-look (png/jpg/png) in a pop-up browser window Portal must allow GIM2D for data with VO Catalogs through WESIX Portal must stage cutouts and models of the GIM2D results Portal must supply user with GIM2D textual output Portal must indicate to user overlap regions Portal must allow the user to sub-select on PSF Portal must allow user to specify which images to Swarp and to specify what region to center on and which image to base the Swarp on Portal must Stage resultant images Portal must have error reporting back from the Swarp service Portal must return overlap region as a cutout data-cube and stage as well as show image metadata in appropriate queries Portal must allow user to add new image to a cart for analysis and access Portal must allow only Swarped or WREG images to be used for IMCOMBINE (i.e., any data in a data-cube). Dependency Precondition Cut-out webservice exists. GIM2D webservice exists. WREG webservice exists. IMCOMBINE webservice exists.

34 4.3 Analysis: IRAF Webservices 4.4 Analysis: Aladin IUM Status 40.00% User should be able to use the Aladin JAVA Applet tool to view FITs images. Sci Reqs Allow Portal to upload multiple images to Aladin (images within the same region) Install Aladin JSP at Portal Make Portal choose the NOAO Aladin JSP to show images Allow users to 'choose' via a checkbox in the Aladin column Users can still one-click Aladin Make another Aladin link for multiple images Dependency Precondition 1. User selects images from the cart via check-box in the Aladin column. 2. User could also choose to view an individual image via Aladin. 3. Once multiple images are checked, user selects 'View Selected in Aladin'

35 4.5 Analysis: Group Image Statistics and Plotting Status 0.00% Allow user to view statistics and summary statistical plots of image subsets (for example, plot the seeing distribution for a subset of images). Sci Reqs 32, Make Ruby module to write a VOTable from a DB metadata list Install local copy of VOPlot Point VOPlot to the local VOTable Dependency Precondition

36 5. Documentation 5. Documentation Status 50.00% The code-base should be documented in such a way that a new team could re-build the Portal from scratch. However, the documentation should not be too obtuse and/or detailed. There will be an architechture document which describes the code from high-to-mid level. There will be an installation and operational document. Additionally, there will be any number of examples, tutorials, FAQs, etc., supplied to the users. We expect to write a short manuscript for distribution to a major astronomical pre-print service or journal. We expect to write shorter newsletter-style documents for local and/or internal distributions. We expect to produce posters and oral talks for appropriate meetings and conferences. Sci Reqs 36 and 39

37 6. Users 6.1 Users: Portal User Registration Status 50.00% A page where users can specify their information for future log in Sci Reqs 11, User Registration Page Dependency User DB ( ) Precondition 'ICD' to NSA exists 6.2. Users: Portal User Login Status 90.00% A previously registered user can log in to the Portal

38 6.2. Users: Portal User Login Sci Reqs 11, User Login Page Dependency User Registration Precondition 'ICD' to NSA Usewr Registration Exists 1. User chooses the 'Login Here' link. 2. User enters his/her user name and password. 3. A 'logged in' status is displayed on all VIEWS 6.3 Certificate Service Call Status 50.00% Call to the NSA service to get back a certificate Sci Reqs 11, Make call via Ruby System call to script sending user name and password. Dependency 6.2.1, Precondition 'ICD' to service exists

39 6.3 Certificate Service Call 6.4 Secure HTTPS SIAP/ADQL Calls to NSA Status 90.00% Ruby call to HTTPS SIAP and ADQL Service at NSA Sci Reqs 11, Build Ruby module Dependency Precondition 'ICD' to service exists 1. All SIAP and ADQL calls (HTTP POSTs) are sent with a certificate using Ruby::Net and https.rb