Project III for A.L.L. COVENTRY UNIVERSITY A.L.L. Project 3 Database Project Jordan Bignell, Robert Bowry, Muhammed Abba Ismail, Emmanuel Idowu 7 th March 2015 Contribution to the A.L.L. Project 3 Report Page 0 By Robert, Ismail, Jordan and Emmanuel
Contents Introduction P2 Project Plan P3 Entity Relationship Diagram P4 Class Diagram P5 Use Case Diagram P6 Research P7 to P9 Selected Service Catalogue P10 ITIL Report P11 to P12 Normalization to 3 rd Normal Form P13 SQL+ Database P14 to P21 Copy Of Python Code P22 to P27 References P28 1
Introduction As a business Intelligence Company, we need to use CSI (Continual Service Improvement) which is one of the packages contained within the 3 rd version of ITIL, to help improve our service by highlighting the current problem will face and then find solution for said problem. The solution to this problem will be for us to create a database using an ORM (Object Relational Mapper) such as SQL+ that will allow us to build said database. Once we have the database created, we need to create a class for each table and them implement them into Python and create a piece of test code that will; A, enable us to connected to the database and B, enable us to pull data from SQL and display it in Python. In addition to the code, we will also be creating an ERD (Entity Relationship Diagram) as well as a class diagram to help us plan out our database and python test code. Along with this we will also be doing research into SQL, NOSQL and ORM. Contributions: Introduction All GANTT chart, ER Diagram, Class Diagram, Two Python Classes, Python Quire, ITIL Report Ismail Research Jordan (+1 Python Class and Use Case Diagram) and Emmanuel (+1 Python Class) Service Catalogue All GANNT chat, Use Case Diagram, ITIL Report, SQL+ Database, Two Python Classes, Normalization to 3 rd normal form Report edited by Jordan, finalized by Robert and Ismail. 2
Project Plan In addition to the project plan we created, we also used WhatsApp for communication and Podio to store our document; helping us to keep organized throughout the project. 3
Entity Relationship Diagram PK ID Client First name Last name Address Contact number PK PK Project Project ID Client ID FK Name Start date/time End date/time PK PK PK PK Work packages Work packages ID Document Id FK Employee ID FK Project ID FK Start time End time Projected Cost Actual Cost PK Documentation Documentation ID Document name Document Type Document Status Employees PK Employees ID First Name Last Name Address Contact Number Employee Role PK PK Employee ID FK Role ID Role name Charging Rate 4
Class Diagram 5
Use Case Diagram 6
Normalizing Tables Database normalization organizes the attributes and tables within the relational database to minimize the complexity and data redundancy. One example of normalization could be removing data that has been stored duplicate times in one table. This cuts down the size of the tables. In addition to this, you typically have 0NF, 1NF, 2NF and then 3NF (NF standing for Normal Form). As you go along each form, you are creating a more suitable and streamlined database No SQL The no SQL attempts to look and solve the scalability and big data performance issues that the relational databases were not designed to address. No SQL is mainly useful for large packs of unstructured data that is stored on multiple servers. Types of NoSQL Databases: Key Value Store This is one of the least complexes NoSQL. Key value store data in a schema less way (which means it doesn t have a fixed representation and you copy and paste into it) Column Store storing data tables as columns of data instead of rows of data. This allows high performance and highly scalable architecture Document Database is an expansion of key value store in which each document I s given a unique key which can be used to retrieve the data at a later time Graph Database this is for data that is presented as a graph but must have elements which are connected 7
Data model Performance Scalabilit y Flexibility Complexity Functionality Key-value store High High High None Variable (None) Column Store High High Moderate Low Minimal Document Store High Variable (High) High Low Variable (Low) Graph Database Variable Variable High High Graph Theory Reasons businesses use NoSQL database Most businesses tend to embrace the NoSQL database over SQL for various reasons as it provides them with competitive advantage in countless industries one being continuous data availability when a company experiences downtime it could ruin their reputation as well as result in loss of customers. The setup of NoSQL Other reasons to use a NoSQL Database The ability to store and retrieve great quantities of data Storing relationships between the elements is not important Dealing with growing lists of elements Twitter posts, internet server logs, Blogs The data is not structured or the structure changes with time Prototypes or fast applications need to be developed Constraints and Validations logic is not required to be implemented in database 8
SQL SQL is used as a standard language for a relational database management system. Also SQL is known as Structured Query Language. It communicates with the database. SQL is mainly designed for Managing, sorting and organizing the data stored within the relational database management system (RDSMS). Programs for SQL are implemented by businesses and other companies as an alternative way to access and manipulate the information and data stored in their databases. Small businesses will find a cheap way to manage their data such as Microsoft excel, however as the business populates they will eventually have to change the database to cope with the capacity of data incoming. Therefore, businesses will have to use Database management systems to organize the data. ORM Object Relational Mapping is abbreviated to ORM. It is a technique of entering a relational database from an OO (Object Oriented) language. An example of an OO language the ORM would access could be Java. ORM is used for sitting on top of a database abstraction layer (DBAL). This is an alternative to SQL as it is flexible as is does not require the duplication of code. 9
Selected Service Catalogue 10
ITIL: Continual Service Improvement Model Our Business: We are a business intelligence company that provides a wide range and variety of financial data to our clients. Using an API that is connected to the data sources we have created, it would enable our clients to access all of the information that they required and although our API works well, there is always room for improvement which is why we are always looking CSI (continual service improvement) so that we are always looking at ways in which we can improve our service as well as the experience for our clients. What is the vision? Our vision as a business intelligence company is to use a data-analysis tool which we will provide that will allow our clients to access the relevant and accurate information within a chosen financial sector; the information will be gathered by our organization. Where are we now? Currently, we have collected relevant and accurate financial data from a specific chosen sector; this sector being the automotive industry. Once we had created our data sources using the information we had previously gathered, we then moved on to create our dataanalysis tool (an API) that would allow our clients to view specific data. For example, using the API tool they would be able to access the following; DR Return On Capital Employed Opening Stock Purchases Closing Stock Gross Profit CR Gross Profit Margin Sales Net Profit Net Profit Margin Bad Debt Where do we want to be? Now that we have created the API and are now available to our clients, we now want to ensure that when our clients are using our data-analysis tool that our API is fast and efficient. In addition to this, we also want to be able to store our clients data to ensure that 11
if they lose their client data and want to use our service again, we shall have a record of their data stored; enabling us to negate past going through the entire cycle again. How do we get there? For our business to be able to achieve the goals/targets we have set ourselves, we will need to create an API that allows our clients to quickly gather and view specific information from our chosen financial sector. To make sure that our API is able to store all of our client s information as well as being able to calculate different financial ratios such as the ones mentioned before. In addition to this, to ensure that our data is both relevant and accurate we will employ a team of data-analyst that will go through each record to ensure that it is suitable for our client. Did we get there? Unfortunately, we were not able to achieve the targets/goals because of several reasons. The first reason being is that the API we created was very slow in both gathering and displaying the data to our clients; whiling not being able to store our clients information as well. Moreover, the second reason as to why we were not able to get achieve our goals/targets was because the initial employee we hired to create and maintain our API has departed. In addition to this, because of the high amount of data we had to handle for our clients, some of this data was lost and overall impacted the quality of our service that we were providing to our clients. Solution In order to fix these problems that we have highlighted, we have now employed a team of people who have had experience with using databases. Once employed, we are now currently training the group of employees on how to create a database as well as how to maintain one and how to manage it. The reason why we have done this is because by creating a database, we will be able to save all of our client s data as well as being able to efficiently and quickly access all of the data stored within it; also allowing our data-analysts to easily manage large volumes of data In addition to this, as a business we can also look at the seven step improvement process that will help as measure whether the goals we want to achieve are long-term, short-term or day-to-day. 12
Normalizing to 3 rd Normal Form 13
SQL+ SQL Database Since I have already created my tables and inserted my data, below I will be showing print screens of how I have created by tables as well as how I have inserted my table. 14
In order to add our foreign keys into our table, we had to use the following statement: 15
The print screen above shows the six tables we have created within SQL for our Database. Below we will be evidencing our database outputting different information regarding to the quires we have been given in our brief. 16
List of All Records in Each Table: Displayed above is all of the records within both the Client and Project table. 17
18
Displayed above is all of the records within the Work-Packages and Employees Table. 19
Displayed above is all of the records for both the Documentation and Employee-Roles table Total cost of Actual Cost for a Work Package in a given month: Displayed above shows both the work package ID and the actual costs for the work packages within a month. The work package ID represents the work package name and highlighted next to it is the actual cost for that work package within a given month. For example, for Work package 5, the actual cost was 3300 for the month. The print screen to the left of this text, displays the second quire from the brief. First of all, we can see the Project ID and the Projected Costs for that project. Since the projected costs are on a monthly basis, we then have to multiply whatever figure we have by 12 which will give us the answer to the following quire: Total Projected Cost for a Project in a year 20
List of Employees and their Assigned Project Name: These two print screens represent the Project Name and who has been assigned to it. To explain this, we look at the Project ID and the Employee ID. The project ID corresponds to the employee ID. This was intended when creating the database since (for example) if Warren Peace as their employee ID of 2, which relates to project ID 2, we can see that project ID 2 has an assigned name of Project, 2. Therefore, my viewing the ID s of the projects and employees, we can see who exactly has been assigned to which project. 21
Copy of the Python Code: ############ Connection to the DataBase ############ from sqlalchemy import Table, MetaData, Column, Integer, String, ForeignKey from sqlalchemy.orm import mapper from sqlalchemy import create_engine engine = create_engine("oracle://bowryr:grey.wolf101@dbacad.services.coventry.ac.uk:1521/eclive.coventry.ac.uk") metadata = MetaData() ############ Client Table ############ Robert Client_Table('client', metadata, Column('clientId', Integer, primary_key=true), Column('lastName', String(38)), Column('firstName', String(38)), Column('address', String(38)), Column('contactNum', Integer) ) clientid = select clientid from Client class Client(object): def init (self, clientid, lastname, firstname, address, contactnum): self.clientid = clientid self.lastname = lastname self.firstname = firstname self.address = address self.contactnum = contactnum def get(self): return client. init () 22
mapper(client, client) ############ Project Table ############ Robert Project_Table('project', metadata, Column('client_clientId', Integer, ForeignKey('Client.clientId')), Column('projectId', Integer, primary_key=true), Column('projectName', String(38)), Column('startDate', Integer), Column('endDate', Integer), ) projectid = select projectid from Project class Project(object): def init (self, clientid, projectid, projectname, startdate, enddate): self.clientid = clientid self.projectid = projecid self.projectname = projectname self.startdate = startdate self.enddate = enddate def get(self): return project. init () mapper(project, project) ############ WorkPackages Table ############ Ismail Workpackages_Table('workpackages', metadata, Column('workpackageId', Integer, primary_key=true), Column('Employees_employeeId', Integer, ForeignKey('Employees.employeeId')), 23
Column('Project_projectId', Integer, ForeignKey('Project.projectId')), Column('Documentation_documentationId', Integer, ForeignKey('Documentation.documentationId')), ) Column('time', Integer), Column('projectedCosts', Integer), Column('actualCosts', Integer) class workpackages(object): def init (self,workpackagesid,employeeid, projectid, documentid, time, projectedcost, actualcost): self.workpackageid = workpackagesid self.employeeid = employeeid self.projectid = projectid self.documentationid = documentationid self.time = time self.projectedcosts = projectedcosts self.actualcosts = actualcosts def get(self): return workpackages. init () mapper(workpackages, workpackages) ############ Employees Table ############ Ismail Employees_Table('employees', metadata, Column('employeeId', Integer, primary_key=true), Column('employeeRoles_roleId', Interger, ForeignKey('EmployeeRoles.roleId')), Column('lastName', String(38)), Column('firstName', String(38)), 24
Column('address', String(38)) Column('contactNumber', Integer) ) employeeid = select employeeid from Employees class employees(object): def init (self,employeeid,lastname,firstname,address,contactnumber): self.employeeid = employeesid self.lastname = lastname self.firstname = firstname self.address = address self.contactnumber = contactnumber def get(self): return employees. init () mapper(employees, employees) ############ EmployeeRoles Table ############ Jordan Employeeroles = Table('employeeRoles', metadata, Column ('employeelastname', String(38)), Column ('employeefirstname', String(38)), Column ('roleid', Integer(38), PrimaryKey,('roleId')), Column ('nameofrole', String(38)), COlumn ('chargingrate', Integer) ) class employeeroles (object): def init (self,employeeroles,employeelastname,employeefirstname,roleid,nameofrole,chargingrate): 25
self.employeelastname = employeelastname self.employeefirstname = employeefirstname self.roleid = roleid self.nameofrole = nameofrole slef.chargingrate = chargingrate def get(self): return employeeroles. init () mapper(employeeroles, employeeroles) ############ Documentation Table ############ Emmanuel Documentation = Table('documentation', metadata, Column ('documentationid', Integer, PrimaryKey,('documentationId')), Column ('documentname', String(38)), Column ('typeofdocument', String(38), Column ('statusofdocument', String(38)), ) class documentation (object): def init (self,doucmentationid, documentname, typeofdocument, statusofdocument): self.documentationid = documentationid self.documentname = documentname self.typeofdocument = typeofdocument self.statusofdocument = statusofdocument def get(self): return documentation. init () mapper(documentation, documentation) 26
# conn controls the span of connection, while the transaction holds the select statement which prints all the rows from the tables in the database. conn = engine.connect() trans = conn.begin() conn.execute("select * from Client") conn.execute("select * from Project") conn.execute("select * from Work Packages") conn.execute("select * from Employee") conn.execute("select * from Employee Roles") conn.execute("select * from Documentation") conn.fetchall() trans.commit() conn.close() 27
References Margaret Rouse. (2014). NoSQL (Not Only SQL database) definitionavailable:http://searchdatamanagement.techtarget.com/definition/nosql-not- Only-SQL. Last accessed 07/03/2015. Carry Janssen. (N/A). Object-Relational Mapping (ORM). Available: http://www.techopedia.com/definition/24200/object-relational-mapping--orm. Last accessed 07/03/2015 N/A. (N/A). NOSQL DATABASES EXPLAINED. Available: http://www.mongodb.com/nosqlexplained. Last accessed 07/03/2015 N/A. (N/A). SQL. Available: http://techterms.com/definition/sql. Last accessed 07/03/2015. Margaret Rouse. (N/A). SQL (Structured Query Language) definition. Available: http://searchsqlserver.techtarget.com/definition/sql. Last accessed 07/03/2015. Steve Pailthorpe. (2014). Practical Advice from the Experts in Business Technology. Available: http://www.ncs-london.com/blog/should-you-consider-using-a-nosql-database. Last accessed 07/03/2015. N/A. (2015). ITIL - Introducing Continual Service Improvement (Online). Available: https://www.ucisa.ac.uk/~/media/files/members/activities/itil/continual_service_improv/i TIL_Introducing%20Continual%20Service%20Improv%20pdf/. Last accessed 07/03/2015 28