Arena SQL: Query Attendance History Course #A252

Transcription

1 Arena SQL: Query Attendance History Course #A252 Presented by: Tim Wilson Shelby Arena Software Developer 2018 Shelby Systems, Inc. Other brand and product names are trademarks or registered trademarks of the respective holders.

2 Before You Start Our Session SQL (Structured Query Language) is a standardized programming language used for managing relational databases and performing various operations on the data in them. This Session does not cover teaching SQL Structured Query Language. This session focuses on Arena database tables and the Relationship of the data between the different tables dealing with attendance, profiles (tags), and groups. Important you understand Primary Keys (identify data uniqueness) and Foreign Keys (referential integrity and Relationships).

3 Before You Start Our Session When looking at tables in this presentation, please use this legend for identifying key column purposes: column_name BOLD = primary key column_name ITALICS = foreign key (Linked table is listed in the description.) column_name BOLD ITALICS = primary key and foreign Key (Linked table is listed in the description.)

4 Objective Identify Relationships between Attendance, Groups and Profiles (Tags) tables. Identify import tables and columns you need to create attendance records. Attendance records (Dataset) can be used for creating new web based applications or reports. Discuss tracking attendance for Enrollees or Attendees associated with a Group or Profile. Talk about tracking attendance by Check-In or Head Count. Discuss the hierarchal structure associated with Groups and Tags. Review the power of using CTE (common table expressions) in queries to build a group and tag hierarchy with a variable number of hierarchal levels.

5 Attendance Sources Check-In Only, Profile (Tags) and Groups

6 Occurrence Family of Tables Description core_occurrence_type Occurrence Type synonymous with Attendance Type, a set of preferences and parameters that determine who can check in under the specific type. Important Columns occurrence_type_id (core_occurrence_type) type_name Description for all occurrences that are associated with the occurrence_type_id. In some cases used in reports to identify the occurrence.

7 Occurrence Family of Tables Description core_occurrence An individual meeting of an occurrence type. Occurrences are created by the Arena UI and Arena agents. Important Columns occurrence_id (core_occurrence) Primary Key, has relationship with (Attendee Path): occurrence_id in smgp_group_occurrence table. occurrence_id in core_profile_occurrence table. occurrence_type (core_occurrence_type)

8 Occurrence Family of Tables core_occurrence Important Columns (continued) occurrence_name For reporting purposes this name is not normally used because of aggregation. To identify an occurrence name, use: head_count core_occurrence_type.type_name smgp_group.group_name core_profile.profile_name Used by occurrences that do not track individual check-ins and associated with large meetings like worship services, etc.

9 Occurrence Family of Tables core_occurrence Important Columns (continued) occurrence_start_time Date and time the occurrence begins Most reports use this column to filter data for reporting. Note: This is a date and time value; when grouping for aggregation, you may need to mask out the time. ( group by occurrence_start_time) occurrence_end_time Date and time an occurrence ends

10 Occurrence Family of Tables Description core_occurrence_attendance Tracks individual check-in transactions. Links a member (person_id) to an occurrence. (attendee path) Important Columns occurrence_attendance_id (core_occurrence_attendance) Primary Key; however, no other tables reference the key. Uniqueness technically is occurrence_id \ person_id occurrence_id (core_occurrence)

11 Occurrence Family of Tables core_occurrence_attendance Important Columns (continued) person_id (core_person) Associated with an occurrence and follows the attendee path attended 0 When value is zero, the member did not attend occurrence. 1 When the value is one, the member attended the occurrence. When associated with group or profile, don t assume that a record is in the table for every occurrence\person. (potential problem with aggregating percentages)

12 Occurrence Family of Tables core_occurrence.head_count Important Notes Used for large meetings where individual check-ins are not a realistic option. The column holds the total number of attendees. If a report is only aggregating head_count, then the join to core_occurrence_attendance is not needed. Normally if the column value is zero, the attendance type is set up to use individual check-in. However, technically an occurrence could use both head_count and individual check-in. Why?

13 Occurrence Family of Tables Important Notes core_occurrence.occurrence_start_time Most of the Arena production reports filter using the report begin and end input parameters against the occurrence_start_time column. Note: This is a datetime data type and date and time may be present. When using the column in your group by for aggregation, you may have to mask out the time using the convert() or datepart() functions.

14 Group Family of Tables Description smgp_group_cluster Database table that holds the majority of the hierarchy for groups. Attendance is not associated with group cluster. Attendance is only associated with a group. (smgp_group) Used to build input parameters for the hierarchal selection of groups. Information in this table rarely used in the body of a report. Important Columns group_cluster_id (smgp_group_cluster) parent_cluster_id points to group_cluster_id one level above. Referenced in smgp_group table in similar fashion as parent_cluster_id.

15 Group Family of Tables Important Columns cluster_name Name of group cluster smgp_group_cluster On occasion a report may require the full hierarchal path of a cluster\group name. The cluster name is used for most of the hierarchal path with the group name from the smgp_group table being the last name in the path. parent_cluster_id (smgp_group_cluster) Parent_cluster_id points to group_cluster_id one level above.

16 Group Family of Tables Description smgp_group Database table that holds group information, the lowest level of any group hierarchy. This is the only level to which members can be added to a group and attendance tracked. All groups must belong to a group cluster, and each cluster can contain one or more groups. Important Columns group_id (smgp_group) group_id column in smgp_group_occurrence table (path to Attendees) group_id column in smgp_member table. (path to Enrollees)

17 Group Family of Tables Important Columns group_name Name of the group smgp_group When associated with an occurrence, it could also substitute as the name of the occurrence. group_cluster_id (smgp_group_cluster) leader_person_id (core_person) Leader of the group and questionable if the id is in smgp_member

18 Group Family of Tables Description smgp_member Table that identifies the members of a group Important Columns group_id (smgp_group) + person_id (core_person) Member of a group (person_id) and follows the Enrollee path. date_joined and date_inactive Filtered based on core_occurrence.occurrence_start_time to accurately determine if an enrollee is enrolled in the group at the time the occurrence took place. role_luid (core_lookup) Group members role (lookup id to description of role)

19 Group Family of Tables Description smgp_group_occurrence Table that identifies occurrences associated with a group Group path to Attendees Important Columns group_id (smgp_group) occurrence_id (core_occurrence)

20 Group Family of Tables Important Notes All attendance tracking is associated with the bottom level of a group. Attendance is not associated with group cluster level. You can get to a cluster level by going through the parent_cluster_id column in the smgp_group table. Group Leader may or may not be a member in the smgp_member table.

21 Profile Family of Tables Description core_profile Synonymous with Tag. in the Arena UI. Database table that holds information about profiles. Important Columns profile_id (core_profile) profile_id column in core_profile_occurrence table (path to Attendees) profile_id column in core_profile_member table (path to Enrollees)

22 Profile Family of Tables Important Columns profile_name Name of the profile (Tag). core_profile When associated with an occurrence it could also substitute as the name of the occurrence.

23 Profile Family of Tables Description core_profile_member Table that identifies the members of a profile\tag Important Columns profile_id (core_profile) + person_id (core_person) Member (person_id) of a profile and follows the Enrollee path date_inactive and date_dormant Filtered based on core_occurrence.occurrence_start_time to accurately determine if an enrollee is enrolled in the profile\tag at the time the occurrence took place status_luid (core_lookup) Profile member s status (lookup id to description of status)

24 Profile Family of Tables Description core_profile_occurrence Table that identifies occurrences associated with a profile\tag. Profile\Tag path to identify Attendees. Important Columns profile_id (core_profile) occurrence_id (core_occurrence)

25 Profile Family of Tables Important Notes Profile is synonymous with Tag. Un-like Groups where the bottom level of the hierarchy is the only level associated with attendance, Profiles can be associated with an occurrence record at any level in the hierarchy. Aggregation is complicated by additional concerns like: Am I accumulating totals for just this profile or all the profiles associated below this profile? Are totals accumulated as one total or broken out in hierarchal form?

26 What Name? When reporting on an Attendance Type, what name do I use to associate with my aggregation? core_occurrence_type.type_name (best choice requires less joins) core_occurrence.occurrence_name (normally not a good choice) smgp_group.group_name core_profile.profile_name The name associated with the attendance type (core_occurrence_type) in most cases is the name to use in the report. However, if you have a group or profile associated with the attendance, then the group or profile name may be a better choice. Normally the occurrence name is not a good choice because of aggregation. In most cases a Min or Max function would be required in order to use the name associated with core_occurrence.

27 What Name? The query below lists all occurrences and the different names associated with each occurrence: SELECT co.occurrence_id,co.occurrence_name,cotype.occurrence_type_id,cotype.type_name,sg.group_name,cp.profile_name,namesource = CASE WHEN sgo.group_id IS NOT NULL THEN 'Group' WHEN cpo.profile_id IS NOT NULL THEN 'Profile' ELSE 'Attendance Type' END FROM core_occurrence co INNER JOIN core_occurrence_type cotype ON co.occurrence_type = cotype.occurrence_type_id LEFT OUTER JOIN smgp_group_occurrence sgo ON co.occurrence_id = sgo.occurrence_id LEFT OUTER JOIN smgp_group sg ON sgo.group_id = sg.group_id LEFT OUTER JOIN core_profile_occurrence cpo ON co.occurrence_id = cpo.occurrence_id LEFT OUTER JOIN core_profile cp ON cpo.profile_id = cp.profile_id

28 What Name?

29 Questions To Ask Before You Start What Attendance source am I using? Check-In only, Profiles\Tags, or Small Groups What am I aggregating? Individual Check-Ins (class rooms, small groups). Head Count (worship services). Totals\Sub Totals (day, week, month or year). Who am I tracking? Attendees, Enrollees, Visitors. What information about who I am tracking is needed.

30 Hierarchal Group/Profile Structure How do I create a data structure to support the following Group or Profile Hierarchal Structure? Teachers- Level 1 Adult Classes - Level 2 Sunday School Teachers - Level 3 Small Group Teachers - Level 3

31 Hierarchal Group/Profile Structure Tag Id Parent Tag Id Tag Name 1 NULL Teachers 2 1 Adult 3 2 Sunday School 4 2 Small Group Tag Id is the identity associated with the tag name. Parent Tag Id is the identity associated with tag name one level above Tag Id. When Parent Tag Id is NULL the Tag Id represents the top level. The bottom level of a hierarchal structure is identified when the Tag Id is not referenced as a parent id.

32 Hierarchal Profile CTE Query Part 1 WITH profile_hierarchy AS (SELECT tags.profile_id,tags.profile_name,profile_breadcrumb = CONVERT(VARCHAR(MAX), tags.profile_name),id_breadcrumb = CONVERT(VARCHAR(MAX), 'p' + CONVERT(VARCHAR(MAX), profile_id) + '/'),hierarchy_level = 1,parent_profile_id FROM core_profile tags WHERE parent_profile_id is null UNION ALL SELECT tags.profile_id,tags.profile_name,profile_breadcrumb = CONVERT(VARCHAR(MAX), profile_hierarchy.profile_breadcrumb + ' - ' + tags.profile_name),id_breadcrumb = profile_hierarchy.id_breadcrumb + CONVERT(VARCHAR(MAX), 'p' + CONVERT(VARCHAR(MAX), tags.profile_id)) + '/',hierarchy_level = profile_hierarchy.hierarchy_level + 1,tags.parent_profile_id FROM core_profile tags inner join profile_hierarchy ON tags.parent_profile_id = profile_hierarchy.profile_id )

33 Hierarchal Profile CTE Query Part 2... SELECT profile_id,profile_name,profile_breadcrumb,id_breadcrumb,hierarchy_level FROM profile_hierarchy ORDER BY profile_id

34 Hierarchal Profile CTE Query Results

35 Hierarchal Group CTE Query Part 1 WITH group_hierarchy AS (select cluster_id = clusters.group_cluster_id,clusters.cluster_name,cluster_breadcrumb = CONVERT(VARCHAR(MAX), clusters.cluster_name),id_breadcrumb = CONVERT(VARCHAR(MAX), 'c' + CONVERT(VARCHAR(MAX), clusters.group_cluster_id) + '/'),hierarchy_level = 1,parent_cluster_id = null FROM smgp_group_cluster as clusters WHERE clusters.parent_cluster_id is null UNION ALL SELECT cluster_id = clusters.group_cluster_id,cluster_name = clusters.cluster_name,cluster_breadcrumb = CONVERT(VARCHAR(MAX), group_hierarchy.cluster_breadcrumb + ' - ' + clusters.cluster_name),id_breadcrumb = group_hierarchy.id_breadcrumb + CONVERT(VARCHAR(max), 'c' + CONVERT(VARCHAR(MAX), clusters.group_cluster_id)) + '/',hierarchy_level = group_hierarchy.hierarchy_level + 1,parent_cluster_id = clusters.parent_cluster_id FROM smgp_group_cluster AS clusters INNER JOIN group_hierarchy ON clusters.parent_cluster_id = group_hierarchy.cluster_id )

36 Hierarchal Group CTE Query Part 2... SELECT group_id,group_name,cluster_breadcrumb = cluster_breadcrumb + COALESCE(smgp_group.group_name, ''),id_breadcrumb = id_breadcrumb + COALESCE(CONVERT(VARCHAR(max), 'g' + CONVERT(VARCHAR(MAX), smgp_group.group_id)), ''),hierarchy_level FROM group_hierarchy LEFT OUTER JOIN smgp_group ON smgp_group.group_cluster_id = group_hierarchy.cluster_id

37 Hierarchal Group CTE Query Results

38 Best Practices Best Practices when developing queries: 1. Filter large tables down to smaller data set. Attendance and Contributions tables may contain millions of rows. First CTE should use report parameters to reduce rows. Do not bring in the frou-frou (first name, last name, phone number, member status etc.); only work with IDs. 2. Perform Aggregation on the smaller data set. Use results from the first CTE to perform aggregation on IDs ONLY. Avoid large Group By statements. 3. Bring in the frou-frou Final select uses the aggregation CTE and join on other tables using the IDs from the aggregation CTE to bring in the frou-frou.

39 Tim Wilson Shelby Arena Software Developer Tim is a Software Developer at Shelby Systems and has over 22 years of software development experience with a focus and passion for database design and development. Tim has a love for Christ and was excited to join the Shelby Systems team in 2014 and use his skills to help develop software solutions for faith-based organizations.