System and Software Architecture Description (SSAD)

Transcription

1 1

2 System and Software Architecture Description (SSAD) Soccer Data Web Crawler Team 02 First Name Last Name Role Trupti Sardesai Project Manager Wenchen Tu Prototyper Subessware Selvameena Karunamoorthy System/Software Architect Pranshu Kumar Requirements Engineer Zhitao Zhou Feasibility Analyst Yan Zhang Operational Concept Engineer Qing Hu Life Cycle Planner Amir ali Tahmasebi Shaper 2

3 3

4 Version History Date Author Version Changes made Rationale 2014/10/09 YZ, PK SSAD for Soccer Data Webcrawler Draft Initial draft of Soccer Data Webcrawler 4

5 Table of Contents System and Software Architecture Description (SSAD) Version History Table of Contents Table of Tables Table of Figures Introduction Purpose of the SSAD Status of the SSAD System Analysis System Analysis Overview System Context Artifacts & Information Behavior Capability 5

6 6

7 Table of Tables Table 1: Actors Summary Table 2: Artifacts and Information Summary 4 Table 3: Process Description 6 Table 4: Typical Course of Action 7 Table 5: Alternate Course of Action 7 Table 6: Exceptional Course of Action 8 7

8 Table of Figures Figure 1: System Context Diagram Figure 2: Artifacts and Information Diagram 4 Figure 3: Process Diagram 6 8

9 1. 9

10 Introduction 1.A.1.1. Purpose of the SSAD The purpose of this SSAD document is to document the object oriented analysis and design of the soccer data WebCrawler project. The SSAD is used by Team 02 as reference to the system and software architecture of the Soccer Data WebCrawler. The development of Soccer Data WebCrawler should be faithful to the architecture specified in the SSAD. In addition, this SSAD document is used by the maintainer of the soccer data WebCrawler and SporTech B.I. LLC to help understand the structure of the software after it is delivered. 1.A.1.2. Status of the SSAD This document is currently version 1.0, it is the first draft version. The current version of the SSAD is at the Valuation phase. At this point, only the Technology Independent Model is described because not all the products and implementation technologies that will be used have been finalized as yet. 10

11 2. 11

12 System Analysis 1.A.1.1. System Analysis Overview The primary purpose of the soccer data webcrawler is to developed a web crawler based web application that can crawl sport websites for key statistical soccer data and upload the data into the company s database, so that the web crawler can be integrated with other SporTech applications to help soccer organizations improve their performance. The web crawler can crawl public websites for structured data, check for data conflicts and ingest the data into the company s PostgreSQL database. The web crawler can also crawl social media websites (including Facebook, Twitter, Instagram and Youtube) for unstructured data (including feeds, images and videos), check for data conflicts and ingest the data into the company s data store including Hadoop/Hive/Pig. The web crawler also has an user interface that can allow users to drag and drop files of different formats including Excel and csv so that the web crawler can identify, order and reorganize the data and upload the data into the company s Postgre database. The webcrawler allows developers to revise/update the player data as the season progresses. The web crawler also allows developers to add/delete/update the specific website visited, fields to capture from the website and frequency of crawler refreshes for each website. 1.A System Context Figure 1: System Context Diagram 12

13 Actor Description Responsibilities 1. User The user can interact with the crawler toinput the correct requirement to make the get the desired data crawler retrieve correct data. 2. Player Permanent member of a team or a club that is registered. 3. Coach The member of executive /managerial staff of any team. 4. Manager The member of executive /managerial staff of any team. Their details are stored in the database that th system returns to get the user when asked for. Coaches the players and has a GUI to refine the data he wants to make correct decisions. Manages the players, staff and coaches. Keep a track of the profits of the club/team and manage other financial accounts. 5. Team Staff Other members on the team like medica They work as a support for the better staff, team PR, administration, Sponsorsmanagement of the team. They perform variou and ground staff. other important functionalities. 6.Developer 7. Database Storage System 8. GUI System for interaction The person who is responsible to develop the complete system and make work They design, implement and maintain the system. They have an option to add or remove any functionality. A separate system built to work with thethis system stores and keeps a track of all the web crawler. data a crawler retrieves from various website the correct format. It is updated periodically. This system is an interactive interface fothis is an interface for users and coaches to users and coaches. modify the result they want and even update t stored result to analyze the data. Table 1: Actors Summary 13

14 1.A Artifacts & Information Figure 2: Artifacts and Information Diagram Artifact ATF1: sport websites ATF2: csv/excel files ATF3: Youtube ATF4: Facebook Description a list of sport websites to crawl data from csv/excel files dragged and dropped by users onto the web user interface so that the crawler can identify, sort and reorganize the data in thos files and upload the data into SporTech s database a social media website to crawl unstructured dat (video) from a social media website to crawl unstructured dat (feeds and images) from 14

15 ATF5: Twitter ATF6: data record fields ATF7: frequency of crawler refreshes ATF8: structured data ATF9: unstructured data ATF10: specific player/page ATF11: corrected/clean/complete data ATF12: PostgreSQL database ATF13: data store (Hadoop/Hive/Pig) ATF14: updated soccer data a social media website to crawl unstructured dat (feeds and images) from a list of attributes of the data records that users can add/delete to specify the format of data records crawled from the web a parameter given by a user to specify how often the web crawler should crawl a sport website structured data crawled from sport websites, prepared to be corrected unstructured data crawled from social media websites (Facebook/Twitter/Youtube), prepared to be corrected a parameter given by user to specify which player/club the crawler should crawl informatio about corrected/clean/complete structured and unstructured data, prepared to be uploaded into SporTech s database or data store. SporTech s PostgreSQL database containing all the clean/complete structured data SporTech s Hadoop/Hive/Pig data store containing all the clean/complete unstructured data clean/complete data in the company s database or data store that are updated/revised as soccer seasons progress Table 2: Artifacts and Information 1.A Behavior Fig 3: Process Diagram 15

16 1.A A Identifier Purpose Requirements Development Risks Pre conditions Post conditions Capability Process Retrieval of information using the crawler To get the required information that a user wants from the web and store it in the database. Web crawler system and a database system None System database is well connected to the crawler and is compatible. All the information retrieved is accurate and stored properly. Table 3: Process Description Seq# Actor s Action System s Response 1 User enters the information it want [user] 16

17 2 Initiate the search command [user] The requirement is sent to the crawler to retrieve the data from the websites [web crawler] 3 The system sends back the retrieved data that it crawled back to the database [crawler database] 4 The database stores the information and produces it to the user[database] 5 The user get the desired results [user] Table 4: Typical Course of Action Seq# Actor s Action System s Response 1 User enters the information it want [user] 2 Initiate the search command [user] The requirement is sent to the crawler to retrieve the data [web crawler] 3 Checks if data is there in database already [database] 4 If so then the crawler returns the data directly from there and no need to crawl [crawler] 5 The user get the desired results [user] Table 5: Alternate Course of Action Seq# Actor s Action System s Response 1 From typical course of action 2 Displays an error message showing no result or match found [database 3 Next command / click on OK 17

18 4 Redirects the user to the search option again. Table 6: Exceptional Course of Action 18