F3 FACETs Technical Report 05/19/2010
PROJECT OVERVIEW... 4 BASIC REQUIREMENTS... 4 CONSTRAINTS... 5 DEVELOPMENT PROCESS... 5 PLANNED/ACTUAL SCHEDULE... 6 SYSTEM DESIGN... 6 PRODUCT AND PROCESS METRICS... 8 METRIC DESCRIPTION... 8 METRIC RESULTS... 8 Results Per Release... 8 Defect Density... 9 Requirements Volatility... 9 PRODUCT STATE AT TIME OF DELIVERY... 9 REFLECTION... 10 2 P a g e
Revision History Name Date Reason For Changes Version Mike Phillips 05/19/10 Initial Creation 1.0 3 P a g e
Project Overview The goal of this project was to create and enhance web-based tools that will assist engineers in designing quality products. The tools were part of a larger system that offers an open-source, real-time collaborative environment that allow engineers to seamlessly share and develop projects using a shared web interface. The first part was the existing Engineering Design Guide and Environment (EDGE) system. EDGE is a webbased project management system that handles user authentication, user permissions, and hosting of projects. Each project is then provided with its own project tools such as a SVN document repository and website. Most of the work of this project was on FACETs, a set of tools provided to each EDGE project. The FACETs tools are based off the FACETs concept which supports that all engineering processes can be broken up into twelve different phases of engineering or facets. Our work focused on three of the tools which addressed two of the early facets. We made enhancements to the objective tree tool which is part of the needs assessment facet and created the function tree and house of quality tools which belong to the concept development facet. An important feature of the system is the ability to track data integrity as data is shared and manipulated throughout the system. The house of quality tool is the first tool to handle this aspect and will likely serve as precedent for future tools. Basic Requirements Since the system already exists, the requirements for this project only deal with the portions of the system being developed by us. There are four areas from which these requirements are taken. These include adding several features to the objective tree tool, implementing the function tree tool, implementing the house of quality tool, and performing improvements to the system itself. The requirements for the objective tree mostly included improvements to the existing work. One requirement here was the ability for the moderator to delete desired nodes from the tree by clicking a button. The nodes also had to be rendered in such a way to display the name of the user who created it. The function tree tool did not exist before, so it had to be implemented. The function tree tool was similar to the objective tree. It had to be able to organize values entered by the user into a hierarchical tree. The function tree had to be drawn vertically, with the root node at the top. The moderator had to be able to delete nodes in this tool as in the objective tree. The House of Quality tool was the third area for our requirements. Only the correlation matrix room of the house had to be implemented. This tool had to present the user with a grid, where he could enter the correlation values that would go in a house of quality diagram. The column headers were taken from level 2 and 3 of a function tree session (where level 1 is the root). The user was allowed to select this session from a list of existing sessions. The row headers were done the same way as the column headers - with a selectable session, and using the 2nd and 3rd levels - but from an objective tree session instead of a function tree. 4 P a g e
Constraints The major constraining force behind this project was the fact that the system already existed. All work done by our team had to fit into the existing architecture. The programming languages were constrained somewhat by the fact that this was a web project, and certain languages are the standard way of doing things. For example, there wouldn't have been a choice about using JavaScript. The frameworks that were being used (such as jquery) were already integrated into the system too deeply to change. The database tables were also affected by these pre-determined constraints. With the way the PHP code for the database layer was written, every table was forced to have the two columns lastedit and lasteditor. The previous team's design of the data model for storing the objective tree nodes affected how we implemented the function tree and the house of quality. We opted to use the exact same table for the nodes in the function tree. When implementing the house of quality, this database structure made it difficult to put the headings into the correlation matrix. A special algorithm had to be written to parse this data into the proper form. Another constraint was simply that there were only four people on the team. There was a big learning curve to be able to work on the system. Each team member had to learn the project domain in order to understand how the tools will be used, and certain concepts such as house of quality. We also had to learn how the system worked in order to be able to develop new features for it. Development Process A custom process developed from aspects from Agile, Rapid Application Development (RAD), and Joint Application Design (JAD) methodologies rounded our hybrid methodology. We pulled points from these three models to fit the team's and project's needs and development style. Attributes of our process include: test-driven (acceptance) development incremental and flexible releases, dynamic requirements, and the possibility of prototyping. Following the agile development model, the team's focal point is incremental releases allowing for constant adaptation and inspection of requirements. RAD promotes quick, flexible, and content-filled releases without unnecessary time-consuming documentation. Most of the documentation derived from the acceptance testing including use cases and test cases. 5 P a g e
Figure 1 - RAD process compared to traditional development Both the project sponsor and advisor approved the process knowing that the iteration schedules and requirements were flexible. Since the sponsor attended at least one of the meetings each week, it proved very easy to communicate the status of our projects. The process' short release cycle also enforced communication to the sponsor and faculty advisor to report the status. There were no defined roles for the process except a metric gatherer for after each release. The cycle of our software was very short so the metrics gathered could not conclude anything. Planned/Actual Schedule Our planned project schedule consisted of 4 or 5 releases spanning 2 to 3 weeks in time. The first release was slated to begin January 31st, 2010, followed by our second release beginning March 8th, 2010. Because of the flexible nature of our schedule, the end date of the last release was unknown; however our absolute-end deadline was scheduled for May 22nd, 2010. Our actual schedule ended up as 4 releases each being between 2 and 4 weeks long. Our last release ended May 9th, 2010. Release 1 began and ended on time, although release 2 began a week late on March 15, 2010. Slippage in the schedule was mainly a result of the team underestimating implementation times. The only release that was greatly affected by this problem was release 3, which had to be extended one week past the limit of three weeks due to the implementation of the House of Quality. Also, the fifth release was deemed unfeasible due to end-of-quarter presentations and artifacts. System Design 6 P a g e
Since we were building on an existing system, many of the architectural decisions had already been made, but we still evaluated the architecture to ensure that it will meet future needs. There were several key aspects to the architecture. First, the database subsystem uses the PHP Pear package MDB2 which allows the underlying database to be abstracted away. So even though the production environment is using MySQL, other databases can easily be swapped in. There is also a custom set of classes used to perform the basic SQL commands like select, insert, update, and delete. We felt that the use of the MDB2 package improve the portability and modifiability of the system but the custom SQL command classes had several constraints and limitations that may affect the system in the future. The biggest limitation is the lack of a way to perform more complicated SQL commands such as joins. Though it had little effect on us, future teams may have to reevaluate the use of these classes or implement additional capabilities. Next, the EDGE PHP files are constructed in a very modular manner, making the system highly modifiable. This will be a great advantage as the system grows and additional features are implemented. This modularity helped us to reuse code when creating the function tree tool and made it easy for us to add the house of quality tool. Another design decision was the use of Smarty templating. Even though it is used in a minimalist way, it helps to separate the visual rendering from the underlying logic, which increases the system s modularity and improves maintainability. An additional aspect of the system is the flow of data throughout the tools. This will be a key aspect of the finished system, but is currently only being handled by the house of quality tool. To implement data integrity, a timestamp is attached to the data which helps to determine if the data has been modified by any tools. It is likely that in the future a more sophisticated approach will be required as the number and interdependencies of the tools increase. The last important aspect of the system is the synchronization of the tools. When the tools are being used, AJAX calls are used to synchronize changes between the clients and server. Currently, there is an AJAX poll every five seconds. The length of five seconds was to allow sufficient time for the server to process multiple requests as the number of users could grow quickly. The downside to the five seconds is the delay that users will experience even if there are only a few users. We discussed the possibility of 7 P a g e
implementing a variable length poll that would increase as the number of users on the system increase, but due to time and resource limits we chose to accept the current static five seconds. Product and Process Metrics Metric Description Two metrics were chosen: one for process and another for product. We focused on the process metric, requirement volatility, due to the nature of the product and the foreseen modifications of the system because of technology limitations. The other, product metric, emphasized defects contained in the product through the analysis of defect density. The first, requirement volatility, measured the number of functional requirements changes in given iterations of development. This measurement evaluated our progress in assessing the requirements defined for each iteration. Along with progress, it measured our ability to define and successfully elicit system requirements after each iteration for future releases. Throughout the development process, new requirements added to the list of previous requirements were not considered changes. After being accepted, changes to these new requirements were counted towards the process metric for that iteration. For the product metric, defect density was chosen as the most appropriate metric for our project. Defect density measured the number of reported defects versus lines of code written (excluding comments and whitespace). The metric measured the overall productivity of our team, but more importantly the quality of the product, itself. Since we were working from an existing system, there was an array of defects from the previous and current team's development. Metric Results Results Per Release Release 1: Defect Density: 1.36% Requirement Volatility: 40% Release 2: Defect Density: 1.06% Total Defect Density: 0.61% Requirement Volatility: 0% Total Requirement Volatility: 11% Release 3: Defect Density: 0.51% Total Defect Density: 0.93% Volatility: 0% Total Volatility: 7.7% 8 P a g e
Release 4: Defect Density: 0.83% Total Defect Density: 0.87% Volatility: 0 Total Volatility: 5.4% Defect Density The defect density was relatively low throughout the project. Each developer performed sufficient tests on his code before the end of the release, that most potential defects were prevented before the testing phase had begun. The defects that were found during testing and not resolved before the end of the release were usually eradicated early enough that they weren't included in the running total. Requirements Volatility The metric for requirement volatility was not used in a direct fashion. The team never looked at the volatility and took a specific action because of it. This is partly because we only had four releases, so there were not very many opportunities to react and change. Also, a lot of our requirements gathering was done up-front, so the majority of the requirements changes were caught early on. This led to a high volatility at the beginning and low volatility thereafter. The team did use this data indirectly. During every requirements gathering session with the sponsor, each team member kept in mind that volatility was being measured. As a result, the team tried very hard to capture the requirements right the first time. Product State at Time of Delivery We have finished the implementation of the Function Tree and the basic House Of Quality tool as planned. Most of the features that we planned have been added, although we were not able to implement a new visual system for drawing the Objective and Function Trees as we had hoped. We had originally planned on fixing some existing issues with the Affinity Diagram tool. We later decided that this was out of the scope of our work as we wanted to concentrate on the tools that we were creating. Server load testing had also been discussed at the beginning of the project. While our test server was unofficially tested during user testing, no formal stress testing was performed. In fulfilling our requirements we were able to fix multiple defects that existed from the previous teams' work. Some of those defects, for example, involved problems with creating and editing sessions. Overall, we feel we have finished roughly 85% of the original requirements we listed, as well as multiple extra endeavors. 9 P a g e
Reflection This was a very successful project overall so a good amount went right, but there are always obstacles and attributes that would like to have been changed. Communication was an aspect of the project that went well. We were able to keep in constant contact with our sponsor and each other. The process methodology was easy to follow and allowed for revision without causing any observable setback on the project. The metrics we set to track were facilitated by the process. Schedule slippage and documentation that added no value were two facets of the project that we would have re-evaluated if done over. Our schedule slipped twice, but we still ended up completing 85% of the requirements set in the beginning of the project. Some of the documentation we chose to complete, we did not start because it (collaboration diagram) would have added no value to our project. Also, some of the documentation, though completed for each release (use cases), was ultimately ignored. If re-doing this project or continuing, we would make a better schedule and adhere to it more strictly. We would choose better documentation that we utilized constantly throughout the project for future teams. Since this was an already existing project, we had to continue the work and change code according to our design of new tools. We would have chosen a slightly different technology base that would have been more suitable for this project. Overall, we learned team cooperation on a longer term project compared to the usual project within our 10-week courses. We learned how to take the requirements from our sponsor and transform them into a working product. Communication with the sponsor was probably not like in industry because our sponsor was almost always accessible and constantly providing feedback. 10 P a g e