Compression in Bankware This work was done on the sk-bankware\skbankware instance of the sk-bankware server aimed for the ART applications in Skopje RO. The sk-bankware server characterizes with the following HW&SW info: - CPU: 2 x 12 = 24 cores - RAM: 32 GB (28 GB dedicated for SQL Server) - Storage: one disk of 2.5 TB - SQL Server: Microsoft SQL Server Developer Edition (64-bit) - OS: Windows server 28 R2 Standard edition Short introduction to compression Data compression in SQL Server can be two types: PAGE and ROW. The PAGE type is superset of the ROW type. PAGE compression is more common for OLAP-like systems and ROW type is more common for OLTPlike systems. As PAGE is stronger than ROW it requires the CPU to do more compressing and uncompressing of data, and it s one reason why PAGE is usually used at systems where there is no much data update (update, delete, insert), or it is less frequently done. Data compression is recommended by Microsoft for both OLTP and OLAP systems. In the following white paper they show the benefits of the compression as an advanced feature on an OLTP system. SAP by default use ROW type of compression for all their applications (blog white paper 1, white paper 2). Another satisfying story of this feature is the following of NetApp, and so on. Conclusion of the Microsoft s white paper regarding compression is that it s safe and useful feature for all systems in general. Applying compression on Bankware Advantages and disadvantages In order to check and ensure that the Microsoft s recommendations for applying compression on systems is working well for Bankware, some tests were done on it. There are advantages and disadvantages of applying compression. If the advantages are high and disadvantages are low or negligible then the tradeoff is warrantable and the feature is useful for the system where it s applied. If the disadvantages impact is significant on the system, then the feature should not be applied on the system. Advantages: - Significantly reduced storage for the data - Significantly reduced number of data pages in the memory buffer for SQL Server - Significantly reduced I/O transfer - Faster queries Disadvantages: - CHECKDB is expected to last more - REBUILD indexes is expected to last more - Some other maintenance operations (update stats, reorganize, ) are expected to last more
Size (GB) More explanation and discussion on them will be provided next in this document. Results and tradeoff of compression PAGE type compression was applied on Bankware. Sometimes the PAGE and ROW type compression do not differ much in size, or are almost equal. However PAGE is more recommended for OLAP-like systems. Besides that, the decision for PAGE was based additionally and on the following document Bankware columns info.xlsx which overviews the columns definitions in Bankware. The overview showed that the char-type columns with their definitions dominate in terms of storage required for storing the data. Also the numeric-type columns definitions are quite present regarding the storage they consume for storing data. Data storage, I/O performance counters and Queries execution The data storage gain of the applied compression is graphically shown with Figure 1. Data space for Bankware 35 3 25 2 15 1 5 292 112 Compressed vs. Figure 1. Data storage for compressed and uncompressed Bankware The storage gain by applying compression usually flows between 3-9%. It depends on the system kind. An OLAP system can be compressed up to 9%, whereas the OLTP systems are usually compressed within the range of 3-5%. A compression rate of 6% for Bankware is very satisfying. The I/O benefit is significant. It is mostly due to the big storage gain of 6%. A number of 21 I/O performance counters were monitored in a simulating multi-session environment during a 1 minute timeframe interval. Their values are showed in document IO performance counters for Bankware.xlsx. Results are improving. The graphical comparisons of the counters behavior and their values as well can be seen in Figures of folders c_ipcbanker and uc_ipcbanker for the compressed and uncompressed Bankware database respectively. Regarding the Queries speed and execution, the queries that execute behind the following reports were run on a compressed and uncompressed version of the Bankware database:
Duration time (sec) - Reporting->Reports->Current Internal Balance - Reporting->Reports->Statements - Reporting->Entry analysis->ledger entries - Reporting->Entry analysis->subledger entries - Reporting->Reports->Generate report The number of queries that took part in the test was around 17. What queries were exactly used for this test can be seen in file Test queries.txt. Runs were done under equal overloading conditions on the server. Results for the two runs are shown with the next Figure 2. Queries on compressed and uncompressed BW 5 456 4 348 3 2 1 Queries runs Figure 2. Execution on queries on a compressed and uncompressed Bankware database The gain for the queries execution speed is 23%. This is a good improvement. CHECKDB This is a maintenance command that is used at Bankware (and in all systems). It s healthy for the database and as a kind of check for the database is strongly recommended. The results were expectantly to be lower for the compressed database because of the data is compressed and in that case more operations are done in the CPU, i.e. decompression, check and compression operations are done, against when the CHECKDB is done on uncompressed database where only the check operation is done by the CPU. Results are shown in the next Figure 3.
Duration (seconds) Execution (seconds) CHECKDB for Bankware 3 25 2 15 1 2476 1995 5 Compressed vs. Figure 3. Results for the CHECKDB on compressed and uncompressed BW database Whilst the CHECKDB for uncompressed database finished in 1995 seconds (33 min and 15 sec), the same operation for the compressed database lasted for 2476 seconds (41 min and 16 sec). The expected downtime for this operation is about 2%. REBUILD REBUILD-ing of an index means coping the data of the currently being rebuilt index into new index structure, sorting it and eliminating the fragmentation. The test for the REBUILD of the indexes is at the same time test for the inserting operations as well as for the page splits. Less page splits will occur in the new index structure on the compressed database but more CPU time will be spend for the compression and decompression during the update-type operations. Tradeoff regarding these is almost fifty-fifty. Results are shown in the next Figure 4 6 5 REBUILD indexes in Bankware 5676 5364 4 3 2 1 Compressed vs. Figure 4. Results for the REBUILD operation for compressed and uncompressed BW database.
The REBUILD time for the indexes does not differ much, i.e. the difference is about 3 seconds or 5 minutes which compared to 1 hour and 3 minutes (approximately) is practically not a difference. Conclusions A conclusion is that compression as an advanced feature in SQL Server is fitting very well on the Bankware database and therefore it should be applied. The tradeoff between the advantages and disadvantages is in very positive for the advantages. Disadvantages are such small that they are almost not to be noticed. Another reason for the appliance of the compression as a feature on a system like Bankware is that it s a system which is not a very busy during the working hours. It s used by a small number of users (max 2), there are not many and intensive update operations during the day and the maintenance operations are done during the night hours which eliminates the little-as-shown disadvantages in case to be applied during the day hours, and most importantly the benefit comes exactly on the queries speed which are used during the working day hours. Best, Igor