Further GroupBy & Extend Operations

Transcription

1 Slide 1 Further GroupBy & Extend Operations Objectives of the Lecture : To consider whole relation Grouping; To consider the SQL Grouping option Having; To consider the Extend operator & its implementation in SQL.

2 Slide 2 Example : Whole Relation as a Group Get the total salary bill. Total 238,500 2 No attribute EMPLOYEE ENo EName M-S E1 3 Robson 7 6D E2 Atkins M 1 E3 1 Smith 5 2S E4 Fenwick S E5 2 Mitchell 6 4M E6 4 Blake 8 8M E7 Watson D E8 Jones D Sal 32, ,000 18, ,000 24, , ,000 18,000 EMPLOYEE GroupBy[ ] With[ Total Bag[ Sal ] Sum ] There is no reason why a whole relation should not form a single group. The Groupby operation will then produce a single tuple as a result. In the example above, the algebra operation expresses the fact that : groups or collections are formed using no attribute; in the single group created, the Sal attribute values are summed, without getting rid of any duplicate Sal values, to yield a Total for that group. The procedure is the same as normal, except that an empty set of attributes is used for grouping.

3 Slide 3 Whole-Relation GroupBy Procedure 1. Remove irrelevant attribute(s). AS USUAL. 2. Add an empty set of attributes to the operand as the grouping attributes. It does not alter the operand, since it is empty! The empty attribute has the same value (i.e. the empty set) in every tuple. Group on this attribute; this yields only one group. 3. Create one result tuple / group consisting of the grouping attribute. Thus there is one tuple initially with no attributes, i.e. an attribute containing the empty set. 4. Carry out aggregations. AS USUAL. This will add aggregation result attribute(s) to the one tuple. Adding an empty set of attributes to a relation is like adding zero to a number : x + 0 = x Mathematically an empty set can be considered as a scalar value, even though there is nothing in it. The result relation initially consists of no attributes (i.e. an empty set of attributes) until the aggregation result attribute(s) is/are added to it in the usual way.

4 Slide 4 SQL : Whole-Relation Group The SQL syntax for a whole-relation GroupBy is : Select aggregation As Result-Name From RELATION_NAME ; The Group By phrase is omitted - no grouping attributes to put in it. No grouping attributes in the Select phrase. Repeat this for each aggregation. The syntax of aggregation remains unchanged. Example : Get the total salary bill. (Previous example). Select Sum( All Sal ) As Total From EMPLOYEE ; As there are no grouping attribute(s), they are simply omitted from a standard SQL Group By statement. This means that : they are omitted from the Select phrase; they are omitted from the Group By phrase - but as in SQL keywords followed by nothing are prohibited, the entire Group By phrase is omitted. The aggregation options are unaltered, but now apply to the whole relation as one group of tuples.

5 Slide 5 After a GroupBy? Sometimes it is useful to carry out another operation on the result of a GroupBy. There is no need to do a Projection on the result since it can be incorporated into the GroupBy. To do a Restriction on the result, SQL provides a Having phrase. This is because SQL s built-in sequence of operations has already executed all Restrictions. There is no way to do another without introducing a special phrase, i.e. the Having phrase. The Having restriction condition is limited to using : a grouping attribute, an aggregation results. Because the result of a GroupBy contains nothing but these. The Having phrase operation is the only one that SQL permits to be executed in its fixed sequence after a Group By phrase operation, apart from executing the Select phrase (which is really part of the grouping), and the Order By phrase (which does not affect what is retrieved, only how it is sorted). No other kinds of operations - e.g. Joins, further GroupBys - may be carried out on the result of a GroupBy in the same statement. To overcome this shortcoming, a View can be used in SQL; see later in the course.

6 Slide 6 Executing an SQL Select Statement From The phrases are executed in the following order. Where Restrictions done here. Joins / Cartesian Products done here. Group By Having Grouping done here. Sequencing done here. Another Restriction here! Order By Select Projections done here. SQL s fixed built-in sequence of operations contrasts with relational algebra s flexibility that allows any sequence of operations, so that any required expression can be written explicitly. SQL can overcome this limitation by means of Views and sub-queries, the latter being beyond the scope of this course, but remains more cumbersome.

7 Slide 7 Having Phrase The SQL syntax for a Group By with a Having phrase is : Select GroupingAttribute(s), aggregation As Result_Name RELATION_NAME From Group By GroupingAttribute(s) Having condition ; The Having phrase must immediately follow a Group By phrase, and not be anywhere else. condition can contain only grouping attribute(s) and aggregate function applications in a Boolean combination of comparisons. The Having condition does not need to use the same aggregate function(s) as those that appear in the Select phrase, and often doesn t. Where an aggregation has been given a name using As in the Select phrase, it would be sensible to use that name instead of repeating the aggregation in the Having phrase. However this is not possible, and the aggregation must be repeated. When SQL is used to create a single group out of a whole relation, the Having phrase cannot be used in the SQL statement. The rationale is that syntactically the Having phrase can only appear after a Group By phrase, and there is no Group By phrase in this case. If a Having phrase were allowed (as is the case in algebra), it might or might not remove the single tuple resulting from the Group By phrase.

8 Slide 8 Example of the Having Phrase (1) Get the total salary paid to each marital-status group, for groups where the total exceeds 50,000. EMPLOYEE GroupBy[ M-S ] With[ Total Bag[ Sal ] Sum ] A GroupBy followed by a Restriction. Select M-S, Sum( All Sal ) As Total From EMPLOYEE Group By M-S Having Sum( Sal ) > ; Restrict[ Total > ] Cannot use the name Total. Slide 9 Example of the Having Phrase (2) How many different shipment sizes are there per supplier, for suppliers who ship more than 2 shipment sizes? SHIP GroupBy[ SNo ] With[ Sizes Project[ Qty ] Count ] Restrict[ Sizes > 2 ] Select SNo, Count( Distinct Qty ) As Sizes From SHIP Group By SNo Having Count( Distinct Qty ) > 2 ;

9 Slide 10 Before and After Restrictions It is possible to write algebraic expressions like : ( expression ) Restrict[ ] GroupBy[.. ] With[ ] Restrict[ ] Before After The equivalent in SQL is : Select From Where Group By Having ; Before After A Before Restriction that removes tuples can make the groups smaller or even nonexistent, which can effect the result of the GroupBy. An After Restriction that removes tuples effectively removes groups of tuples from the original operand of the Groupby.

10 Slide 11 Example of Extend A/C 10/23A Amt O/D 100 Euro MaxB BANK A/C Amt 10/23A O/D /29C /29C /14X /14X /18G /18G BANK Extend[ Euro Amt/0.65 ; MaxB Amt + O/D ] We have seen the usefulness of using the GroupBy operation to do aggregate calculations vertically down columns. It is equally useful to be able to do calculations horizontally along rows. The Extend operation allows just such calculations to be specified; its result relation contains the result(s) of the calculation(s) put in a new attribute(s) that are added to the operand. In contrast to GroupBy, all the Extend calculations are scalar; i.e. individual values from one or more attributes of a single tuple are manipulated to produce another single value that is put into a new attribute in the same tuple. The above example demonstrates this. The result relation has two new attributes, Euro and MaxB, in addition to those of the operand. The values in the Euro attribute are calculated using the formula Amt / 0.65 and those in the MaxB attribute by using the formula Amt + O/D where Amt and O/D are the names of attributes in the operand. Optionally, by animating the PowerPoint file ExtendAnim.ppt (obtainable from the Blackboard system), a demonstration of the Extend operator can be seen.

11 Slide 12 Definition of Extend Creates a new relation formed from its operand by adding one or more named attributes to it. Each new attribute(s) contain values specified in a scalar expression. The expression s type is the type of the new attribute. An expression can contain one or more of : a constant; an attribute name : this yields that attribute s value in the same tuple; scalar functions & operators (of the relevant data type); parentheses to control the execution sequence. Note that Extend does not demolish its operand in order to produce its result, but creates the result by copying operand data into the result. For each new attribute to be added, both a name and a defining scalar expression must be provided. Each new attribute value in each tuple is derived in isolation from every other new attribute value in every other tuple; i.e. logically every new attribute value is determined in parallel, not sequentially, with every other new attribute value. Extend is the opposite of Project, in that the former adds attributes and the latter removes them. (In fact some algebraic notations combine both Extend and Project into one generic operator).

12 Slide 13 SQL : Extend The SQL equivalent of an Extension has the syntax :- Select RELATION_NAME.*, calculation As Result_Name From RELATION_NAME ; An * on its own is only allowed if there is nothing else in the Select phrase. This yields all the relation s existing attributes. Repeat for each new attribute. The same considerations that applied to the use of As for a GroupBy operation also apply to As here for an Extend operation : As may optionally be omitted, but it is recommended that it is inserted to make the SQL easier to read. If As and the new attribute s name are both omitted, then the expression is used as the new attribute s name.

13 Slide 14 SQL Examples The previous example is written in SQL as follows : Select BANK.*, Amt/0.65 As Euro, Amt + O/D As MaxB From BANK ; In addition to the details of all bank accounts, show how much the amount in each account would be above an overdraft limit that was 5 times greater than the current overdraft limit. Select BANK.*, Amt + (5 * O/D) As Possible From BANK ; In writing scalar calculations for Extend operations, note that it is possible to create a new column that contains the same data as an existing column, e.g. Select BANK.*, Amt As Loan From BANK ; or create a new column all of whose rows contain the same value, e.g. Select BANK.*, 100 As Loan From BANK ; Neither of these outcomes is usually what is desired.

14 Slide 15 Combining Algebra Operations SQL treats an Extension as the reverse of a Projection, by putting both in the Select phrase. As the Select phrase is the last to be executed by SQL, then an Extension is always the last operation to be executed, in parallel with a Projection. This applies even when there is a Group By (with / without Having). In this case, since Extend applies scalar expressions and an aggregation can be embedded in a scalar expression, the Extension comes after the GroupBy. In algebraic terms, this corresponds to writing : ( some expression ) Extend[ calculation ; ] Project[. ] The Project is designed to be after the Extend in algebra (instead of in parallel) in case the Extension uses any attributes that are then removed by the Projection.