Arena SQL: Query Attendance History Addendum

Transcription

1 Arena SQL: Query Attendance History Addendum (Course #A252) Presented by Tim Wilson Arena Software Developer 2017 Shelby Systems, Inc. Other brand and product names are trademarks or registered trademarks of the respective holders.

2 Terminology Cluster term for any branch of a group tree above the level of an individual group. Group either (a) the bottom level of a group tree into which attendees are enrolled, or (b) a term synonymous with attendance type category, a named collection of occurrence types; every occurrence type is a member of one and only one group. Occurrence Type synonymous with attendance type, a set of preferences and parameters that determine who checks in under the specific type. Occurrence an individual meeting of an occurrence type. Profile synonymous with tag. Template synonymous with frequency, a named set of preferences that controls how often an associated occurrence type occurs.

3 Tables core_occurrence_type_group one row per occurrence type Column Name Data Type Notes group_id Int primary key; uniquely identifies one collection of occurrence types organization_id Int foreign key; joins to orgn_organization group_name varchar(100) name of the collection of occurrence types report_start report_end min_age max_age min_grade max_grade datetime datetime Int Int Int Int date_created datetime date and time the collection is created created_by varchar(50) user name logged in when the collection is created date_modified datetime date and time of the last change to the collection information modified_by varchar(50) user name logged in when the collection is last changed 3

4 core_occurrence_type one row per occurrence type Column Name Data Type Notes occurrence_type_id int primary key; uniquely identifies one set of preferences and parameters for checkin organization_id int foreign key: joins to orgn_organization.organization_id group_id int foreign key: joins to core_occurrence_type_group.group_id sync_with_profile int foreign key: joins to core_profile.profile_id sync_with_cluster int foreign key: joins to smgp_group_cluster.group_cluster_id_ sync_with_group int foreign key: joins to smgp_group.group_id profile_source_luid int foreign key: joins to core_lookup.lookup_id profile_status_luid int foreign key: joins to core_lookup.lookup_id theme_id int foreign key: joins to comp_theme.theme_id type_order int sorting position in on screen list of occurrence types type_name varchar(50) name of the occurrence type active bit 1 = active, 0 = inactive min_age decimal(10,3) minimum age, to three decimal places; = no age restriction max_age decimal(10,3) maximum age, to three decimal places; = no age restriction age_range_text varchar(50) user supplied description of age range min_grade int 1 for no restriction, 0 for kindergarten, or 1 through 12 max_grade int 1 for no restriction, 0 for kindergarten, or 1 through 12 min_birth_date datetime minimum date of birth; 1/1/1900 = no restriction max_birth_date datetime maximum date of birth; 1/1/1900 = no restriction location_specific_occurrences bit 1 = yes, 0 = no is_service bit 1 = weekend service, 0 = not weekend service 4

5 core_occurrence_type merge_days int 1 = do not merge, or 0 (Sunday) through 6 (Saturday) to merge all occurrences on that day membership_required bit 1 = yes, 0 = no secured bit 1 = checkin allowed only at non centralized kiosks, 0 = checkin allowed at any kiosk gender_preference int 0 = male, 1 = female, 2 = no preference pager_required bit 1= yes, 0 = no use_room_ratios bit 1 = yes, 0 = no people_per_leader int number of non leader attendees who can check into a particular location per each leader who has checked in to that same location min_leaders int minimum number of leaders required to check into a location before any non leaders can check into that same location date_created datetime date and time the collection is created created_by varchar(50) user name logged in when the collection is created date_modified datetime date and time of the last change to the collection information modified_by varchar(50) user name logged in when the collection is last changed schema_file_path varchar(1000) unused photo_confirmation bit unused use_altid_for_family bit unused checkout_required bit unused allow_unknown_age bit unused core_occurrence_type_leader one row per tag/attendance type relationship (many to many table) Column Name Data Type Notes occurrence_type_id int foreign key; joins to core_occurrence_type.occurrence_type_id profile_id int foreign key: joins to core_profile.profile_id 5

6 core_occurrence_type_template one row per frequency plus one row per frequency/occurrence type relationship Column Name Data Type Notes occurrence_type_template_id int primary key; uniquely identifies one row group_id int foreign key: joins to core_occurrence_type_group.group_id or is null for individual occurrence type rows occurrence_type_id int foreign key; joins to core_occurrence_type.occurrence_type_id or is null for parent template rows parent_template_id int self reference key: joins to occurrence_type_template_id for parent of current row or is null for the top parent row inherit bit 0 for parent rows, 0 for frequency assignments that are overridden, 1 for shared schedule_name varchar(50) frequency name occurrence_freq_type int 0 for daily, 1 for weekly, 2 for monthly, 3 for one time occurrence freq_qualifier varchar(50) When occurrence_freq_type = 1, then a value 0 (Sunday) through 6 (Saturday) for the day of the week; when occurrence_freq_type = 2, then a value 1 to 31 for the day of the month; when occurrence_freq_type = 3, then the date of the occurrence in m/d/yyyy format. start_time datetime time the occurrence begins (date is 1/1/1900) end_time datetime time the occurrence ends (date is 1/1/1900) check_in_start datetime time check in begins (date is 1/1/1900) check_in_end datetime time check in ends (date is 1/1/1900) date_created datetime date and time the collection is created created_by varchar(50) user name logged in when the collection is created date_modified datetime date and time of the last change to the collection information modified_by varchar(50) user name logged in when the collection is last changed 6

7 core_occurrence one row per occurrence of each occurrence type Column Name Data Type Notes occurrence_id int primary key; uniquely identifies one occurrence organization_id int foreign key; joins to orgn_organization.organization_id occurrence_type int foreign key: joins to core_occurrence_type.occurrence_type_id location_id int foreign key: joins to orgn_location.location_id area_id int foreign key: joins to core_area.area_id occurrence_type_template_id int foreign key: joins to core_occurrence_type_template. occurrence_type_template_id occurrence_name varchar(50) name of the occurrence location varchar(200) any value, not tied to location_id value occurrence_start_time datetime date and time the occurrence begins occurrence_end_time datetime date and time the occurrence ends check_in_start datetime date and time checkin begins check_in_end datetime date and time checkin ends membership_required bit 0 = no, 1 = yes occurrence_closed bit 0 = not, 1 = yes head_count int 0 = use checked in count for reporting occurrence_description varchar(1000) notes about the occurrence date_created datetime date and time the collection is created created_by varchar(50) user name logged in when the collection is created date_modified datetime date and time of the last change to the collection information modified_by varchar(50) user name logged in when the collection is last changed location_address_id int unused foreign_key int unused 7

8 core_occurrence_attendance one row per checkin Column Name Data Type Notes occurrence_attenance_id int primary key; uniquely identifies one checkin occurrence_id int foreign key; joins to core_occurrence.occurrence_id person_id int foreign key; joins to core_person.person_id session_id int foreign key: joins to chkn_session.session_id attended bit 0 = no, 1 = yes (0 values come imported data only; Arena does not capture absences explicitly) security_code varchar(8) validation code for receipts check_in_time datetime date and time of the checkin event check_out_time datetime date and time of the checkout event (1/1/1900 for no check out) pager varchar(50) pager identification type tinyint notes varchar(200) core_profile_occurrence one row per occurrence associated with a tag Column Name Data Type Notes occurrence_id int foreign key: joins to core_occurrence.occurrence_id profile_id int foreign key; joins to core_profile.profile_id smgp_group_occurrence one row per occurrence associated with a group Column Name Data Type Notes occurrence_id int foreign key: joins to core_occurrence.occurrence_id group_id int foreign key; joins to smgp_group.group_id 8

9 Connecting Attendance History to Groups In the following discussion groups has the generic meaning of any collection of people, whether tracked as an actual group or as a set of people associated with a tag. WARNING: The queries presented here are not filtered by date range or by any other factor. As a result these queries could run for several minutes even up to an hour or more on your home systems. In real world applications, it is often necessary to add date filters and other restrictions prior to executing the query in order to ensure a reasonable amount of time for the query to run. When you query attendance data, there are often two overlapping but distinct sets of people associated with any group: the set of enrollees within the group and the set of attendees of the group. These are closely related; but because attendance and enrollment patterns change within any group over time, there are sometimes enrollees who do not have any attendance for a set time period, and there are attendees who are no longer enrolled. Some queries focus on only one set of names or the other, but some solutions require data about both sets of names. The best table to start with is smgp_group_occurrence because it is essentially a list of every group (by group_id) that has at least one occurrence associated with it and also every occurrence (by occurrence_id) associated with a group. To this list you can add every attendee who has attended each occurrence of any group by using the core_occurrence_attendance table in this way: groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, attendees.person_id smgp_group_occurrence as groups_to_occurrences left join core_occurrence_attendance as attendees on groups_to_occurrences.occurrence_id = attendees.occurrence_id and attendees.attended = 1 9

10 Finding all the members of a group for any given occurrence is a little trickier because the query must take into account the beginning and ending dates of each person s enrollment in the group. In addition, the leader of any group (the one assigned directly in the group setup information, not added as an enrollee) does not have a beginning or ending date value with which to work. Thus the leader is assumed to be a full member of the group for every occurrence. Here is the query to find all members of the group for each occurrence: groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, members.person_id smgp_group_occurrence as groups_to_occurrences inner join core_occurrence as occurrences on groups_to_occurrences.occurrence_id = occurrences.occurrence_id inner join smgp_member as members on groups_to_occurrences.group_id = members.group_id where occurrences.occurrence_start_time >= coalesce(members.date_joined, '1/1/1900') and occurrences.occurrence_start_time < dateadd(day, 1, coalesce(members.date_inactive, '12/30/9999')) union groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, groups.leader_person_id smgp_group_occurrence as groups_to_occurrences inner join smgp_group as groups on groups_to_occurrences.group_id = groups.group_id where groups.leader_person_id > 0 Once you have both lists you can join one to the other in order to compare the two lists and determine, for each group and each occurrence of that group, whether a given individual is a member of the group or not and whether that individual attended or not. The following query uses the two queries above as common table expressions and then uses a full join to bring the rows together in such a way as to make the comparison possible: 10

11 with group_enrollees as ( groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, members.person_id smgp_group_occurrence as groups_to_occurrences inner join core_occurrence as occurrences on groups_to_occurrences.occurrence_id = occurrences.occurrence_id inner join smgp_member as members on groups_to_occurrences.group_id = members.group_id where occurrences.occurrence_start_time >= coalesce(members.date_joined, '1/1/1900') and occurrences.occurrence_start_time < dateadd(day, 1, coalesce(members.date_inactive, '12/30/9999')) union groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, groups.leader_person_id smgp_group_occurrence as groups_to_occurrences inner join smgp_group as groups on groups_to_occurrences.group_id = groups.group_id where groups.leader_person_id > 0), group_attendees as ( groups_to_occurrences.group_id, groups_to_occurrences.occurrence_id, attendees.person_id smgp_group_occurrence as groups_to_occurrences left join core_occurrence_attendance as attendees on groups_to_occurrences.occurrence_id = attendees.occurrence_id and attendees.attended = 1) group_id = isnull(e.group_id, a.group_id), occurrence_id = isnull(e.occurrence_id, a.occurrence_id), person_id = isnull(e.person_id, a.person_id), member = case when e.person_id is null then 0 else 1 end, attended = case when a.person_id is null then 0 else 1 end group_enrollees e full join group_attendees a on e.group_id = a.group_id and e.occurrence_id = a.occurrence_id and e.person_id = a.person_id This query is essentially a comprehensive all group attendance history view showing every potential attendee of every group/occurrence event and whether that person attended or not. By further joining this view to the smgp_group, core_occurrence, and core_person tables (or any combination that suits your purposes), you can flesh out just about any attendance history query solution you might need for your groups. 11

12 Connecting Attendance History to Tags The same basic approach applies to tags; just substitute profile for group in the table names and append the tag owners along with the regular tag membership. with enrollees as ( tags_to_occurrences.profile_id, tags_to_occurrences.occurrence_id, members.person_id core_profile_occurrence as tags_to_occurrences inner join core_occurrence as occurrences on tags_to_occurrences.occurrence_id = occurrences.occurrence_id inner join core_profile_member as members on tags_to_occurrences.profile_id = members.profile_id where occurrences.occurrence_start_time >= coalesce(members.date_active, '1/1/1900') and occurrences.occurrence_start_time < case when datediff(day, members.date_dormant, '12/31/9999') = 0 then members.date_dormant else dateadd(day, 1, coalesce(members.date_dormant, '12/30/9999')) end union tags_to_occurrences.profile_id, tags_to_occurrences.occurrence_id, tags.owner_id core_profile_occurrence as tags_to_occurrences inner join core_profile as tags on tags_to_occurrences.profile_id = tags.profile_id where tags.owner_id > 0), attendees as ( tags_to_occurrences.profile_id, tags_to_occurrences.occurrence_id, attendees.person_id core_profile_occurrence as tags_to_occurrences inner join core_occurrence_attendance as attendees on tags_to_occurrences.occurrence_id = attendees.occurrence_id and attendees.attended = 1) profile_id = isnull(e.profile_id, a.profile_id), occurrence_id = isnull(e.occurrence_id, a.occurrence_id), person_id = isnull(e.person_id, a.person_id), member = case when e.person_id is null then 0 else 1 end, attended = case when a.person_id is null then 0 else 1 end enrollees e full join attendees a on e.profile_id = a.profile_id and Oe.occurrence_id = a.occurrence_id and e.person_id = a.person_id 12

13 Check In Only Attendance What about attendance taken that is not associated with a tag or a group? I call this check in only attendance, and the only way to isolate those check in items the other two types is to use smgp_group_occurrence and core_profile_occurrence as filters on the attendance history in core_occurrence_attendance. In short, whatever is neither a group check in nor a tag check in must be a check in only check in. Here is how that works: occurrence_type_id = check_in_events.occurrence_type, check_in_attendees.occurrence_id, check_in_attendees.person_id core_occurrence as check_in_events inner join core_occurrence_attendance check_in_attendees on check_in_events.occurrence_id = check_in_attendees.occurrence_id where not exists ( * smgp_group_occurrence where occurrence_id = check_in_attendees.occurrence_id) and not exists ( * core_profile_occurrence where occurrence_id = check_in_attendees.occurrence_id) Notice that this does not account for enrollees, who would essentailly be people who fit the criteria for checking in but who did not check in. Such a list of names is possible to compile in a query result, but the query logic for arriving at that list is complex and beyond the scope of the presentation for today. Notice also that in this case the occurrence type is standing in for the group or tag as the identification for the group into which the attendees checked in. 13

14 Calculate Attendance Totals Per Occurrence or Per Day The following query generates a list of occurrences along with the meeting date, meeting time, and attendance count for the meeting. It is a good example of how to report on all occurrences in one set of results while also being able to distinguish between group, tag, and check in only events. The generic term leaf_id is the column name for the id that could be a profile_id, group_id, or occurrence_type_id value. Why leaf? Because no matter what type of occurrence, it is part of a hierarchical tree structure with one or more branches above it, and at the end of those branches is the leaf into which people are enrolled and/or checked in. Companion queries that identify the branches are made to dovetail with a query like this to make reporting possible on any branch or leaf node of the tree. leaf_type = case when max(groups.group_id) is not null then 'Group' when max(tags.profile_id) is not null then 'Tag' else 'Check In Only' end, leaf_id = max(coalesce(groups.group_id, tags.profile_id, occurrence_types.occurrence_type_id)), leaf_name = max(coalesce(groups.group_name, tags.profile_name, occurrence_types.type_name)), meeting_date = convert(varchar(10), occurrences.occurrence_start_time, 101), meeting_time = max(replace(replace(right(convert(varchar(20), occurrences.occurrence_start_time, 0), 7), 'AM', ' AM'), 'PM', ' PM')), attendance_count = count(distinct attendees.person_id) core_occurrence as occurrences inner join core_occurrence_type occurrence_types on occurrences.occurrence_type = occurrence_types.occurrence_type_id left join core_occurrence_attendance as attendees on occurrences.occurrence_id = attendees.occurrence_id and attendees.attended = 1 left join core_profile_occurrence as occurrences_to_tags on occurrences.occurrence_id = occurrences_to_tags.occurrence_id left join core_profile as tags on occurrences_to_tags.profile_id = tags.profile_id left join smgp_group_occurrence as occurrences_to_groups on occurrences.occurrence_id = occurrences_to_groups.occurrence_id left join smgp_group as groups on occurrences_to_groups.group_id = groups.group_id group by occurrences.occurrence_type, occurrences.occurrence_id, convert(varchar(10), occurrences.occurrence_start_time, 101) The query above reports on attendance per occurrence, but the variation on the next page combines all same day occurrences of the same occurrence type into one row, giving a count of attendees for the entire day. Note that the counting function counts distinct person_id values, meaning multiple check ins by the same person are all counted as one. 14

15 The only difference between the next query and the last one is that the meeting_time column is removed the SELECT clause and the occurrences.occurrence_id column is removed the GROUP BY clause. leaf_type = case when max(groups.group_id) is not null then 'Group' when max(tags.profile_id) is not null then 'Tag' else 'Check In Only' end, leaf_id = max(coalesce(groups.group_id, tags.profile_id, occurrence_types.occurrence_type_id)), leaf_name = max(coalesce(groups.group_name, tags.profile_name, occurrence_types.type_name)), meeting_date = convert(varchar(10), occurrences.occurrence_start_time, 101), attendance_count = count(distinct attendees.person_id) core_occurrence as occurrences inner join core_occurrence_type occurrence_types on occurrences.occurrence_type = occurrence_types.occurrence_type_id left join core_occurrence_attendance as attendees on occurrences.occurrence_id = attendees.occurrence_id and attendees.attended = 1 left join core_profile_occurrence as occurrences_to_tags on occurrences.occurrence_id = occurrences_to_tags.occurrence_id left join core_profile as tags on occurrences_to_tags.profile_id = tags.profile_id left join smgp_group_occurrence as occurrences_to_groups on occurrences.occurrence_id = occurrences_to_groups.occurrence_id left join smgp_group as groups on occurrences_to_groups.group_id = groups.group_id group by occurrences.occurrence_type, convert(varchar(10), occurrences.occurrence_start_time, 101) 15

16 Calculating Percentage Attended Per Attendee In this last example you calculate a percentage of times attended for a given date range. The solution presented here assumes that the first date of attendance for a given person for a given occurrence_type_id value represents the first time he or she could possibly attend, so it disregards earlier dates when calculating the percentage for that person. The query solution below counts the times a person attends within a date range and then uses a subquery to count the number of meetings held during the date range. If the individual s first date of attendance falls within the date range, subquery only counts the number of meetings on or after the first date of attendance. as as datetime; = '1/1/2015'; = '3/31/2015'; occurrences.occurrence_type, attendance.person_id, first_attended_date = min(occurrences.occurrence_start_time), attended_count = count(distinct case when occurrence_start_time then convert(varchar(10), occurrence_start_time, 101) end), meetings_count = ( count(distinct convert(varchar(10), occurrence_start_time, 101)) core_occurrence c where c.occurrence_type = occurrences.occurrence_type and c.occurrence_start_time between case when min(occurrences.occurrence_start_time) then min(occurrences.occurrence_start_time) end attended_percent = case when ( count(distinct convert(varchar(10), occurrence_start_time, 101)) core_occurrence c where c.occurrence_type = occurrences.occurrence_type and c.occurrence_start_time between case when min(occurrences.occurrence_start_time) then min(occurrences.occurrence_start_time) end > 0 then cast(count(distinct case when occurrence_start_time then convert(varchar(10), occurrence_start_time, 101) end) as decimal) / ( count(distinct convert(varchar(10), occurrence_start_time, 101)) core_occurrence c where c.occurrence_type = occurrences.occurrence_type and c.occurrence_start_time between case when min(occurrences.occurrence_start_time) then min(occurrences.occurrence_start_time) end end core_occurrence_attendance as attendance inner join core_occurrence as occurrences on attendance.occurrence_id = occurrences.occurrence_id where attendance.attended = 1 group by occurrences.occurrence_type, attendance.person_id 16