Mystery Shopping year end report 2015/16 Mystery shops undertaken in 2015-16 Number of Mystery Shoppers involved Value of vouchers issued to shoppers Number of staff hours Rounds of shopping 307 15 1720 41hrs 4 What did we test? Mystery Shoppers contacted Property Managers Housing Officers, They checked responses were given correctly and within time 2013/14 11.05 2014/15 7.12 cost per shop 5.60 (excluding staff cost) How did we carry out the programme? All Mystery Shoppers completed an on line survey for every shop undertaken. All shop scenarios were agreed with the relevant housing team to ensure that no scenarios would raise any work orders. Interim reports were written after each round of shopping. Response definition - A response is recorded when an email, phone message or text is replied to within time or a direct phone call is picked up. Percentage definition - The percentage is worked out by taking the number of shops completed within time from the overall number of shops undertaken for example 35 shops were successfully completed out of a total of 46 shops undertaken this gives a response percentage of 76% overall response rate to all forms of contact 15/16 Data overview The overall response rate to all forms of contact 14/15 The overall response rate to all forms of contact 13/14 76% or 233/307 77% or 146/191 54% or 66/123 Property manager response rate 75% Housing officer response rate 76% Response during office hours 82% Response out of office hours 69%
100 The overall team performance in each round compared with last year 90 80 70 60 50 Round 1 Round 2 Round 3 Round 4 14/15 Overall 14/15 HO 14/15 PM 15/16 Overall 15/16 HO 15/16 PM Response received within time - yes no P/M email out of office hours P/M email during office hours P/M phone out of office hours P/M Phone during office hours H/O email out of office hours H/O email during office hours H/O phone out of office hours H/O phone during office hours H/O text out of office hours H/O text during office hours 90% 84% 48% 75% 80% 92% 65% 74% 61% 82% 10% 16% 52% 25% 20% 8% 35% 16% 29% 18% The number of shops per method is shown in the chart. Property managers were not sent texts, as a consequence there are fewer texts. These are the total number of shops undertaken, not the number of shops responded to. text during office hours, 32 phone call out of office hours, 63 text out of office hours 27 phone call during office hours 61 emial during office hours, 69 email out of office hours, 54 Where there was a response, the numbers of responses in time were as follows. 14-15 94% 96%
Angelyn Francis Chantall Sinclair Coralie Rew Dani Brace Dympna O'Neill Greg Pope Ivie Jay Narwal John Anaba Johnathan Best Kelly Adjetey Kiran Nota/Claire Craig Laura Dymott Lavina Steele Lisa Turner Mark Leahy Michael Noel Monique Green Saima Saifie Sarah Lout Sophie Gill 76% found the staff member helpful and/or friendly Percentage that found the staff member unhelpful 0% 88% were happy with the response Reasons shoppers were not happy with the responses included Rushed msg left on shoppers phone Not put through to staff member but told to go through switchboard Reply text contained no detail No name on answer phone msg Below is the performance of individual staff over the four rounds of shopping 67% 80% 20% 57% 67% 67% 67% 50% 60% 75% 50% 75% 50% 75% 67% 85% 80% 67% 25% 33% 75% 67% 50% 67% 67% 33% 50% 25% 75% 75% 67% 75% 67% 67% 75% 33% 75% 67% 67% 67% 75% 75% 75% 25% 25% 25% 33% 33% A breakdown of shopping by types where there was a response only Below is he break down of each type of shop. Shoppers completed questions on whether the staff member gave their name, job title and company name, had full auto signatures on emails and if emails were sent from a mobile device. Responding to phone calls during office hours No staff name used when answering the phone 4/45 No job title used when answering the phone 15/45 No company name used when answering the phone 9/45 Responding to phone calls out of office hours No staff name used when answering the phone 0 No job title used when answering the phone 5/36 No company name used when answering the phone 4/36
I had to phone Danny Brace, Property Manager out of hours and leave a message for a call back. When I left my message on her voicemail on 5/10/15 it said she was on Annual Leave Friday 31 July to Monday 10 August. This message should have been removed as it was an old message and no longer relevant. Also returned my call after 48 hours, but left polite, friendly and professional voicemail A point to note: Ms Dymott's voicemail information provided out of hours was outdated (details of leave taken in July etc) and should be updated accordingly Eventually received a response. No apology for the lateness of the response Ms Saifie could not take my call when I tried the first time, so I left a voicemail message and she promptly responded with a very polite and friendly phone call. Very pleasant! Four answer phone messages were out of date and one message was rushed so that detail was lost. Responding to emails during office hours No staff name used when replying to emails 0 Number of emails sent from a mobile device 7/61 No job title used when replying to emails 8/61 No company name used when replying to emails 14/61 No phone number in replies to emails 11/61 Responding to emails out of office hours No staff name used when replying to emails 1/46 Number of emails sent from a mobile device 7/46 No job title used when replying to emails 4/46 No company name used when replying to emails 4/46 No phone number in replies to emails 4/46 Very detailed response, very helpful. Luckily the question (about lease) was something I was genuinely interested in so Saima's email was truly helpful. Response email from Michael Noel was addressed to Dear Sir/Madam; I finished my email with my name Miss Wilson. Email could have addressed me as Dear Miss Wilson, not Dear Sir/Madam otherwise all okay.
There was an out of office message received immediately at point of first email sent; and a reply was promptly received on the date Ms Brace returned to work (04 Jan). Dympna O'Neill is on annual leave so Angela responded to email. I think this was the best response I had from all parties as it was concise, authoritative, yet informative. It gives me the information requested with additional details of what is, and also is not, permitted, and why, which would have been very useful if it had been a legitimate query. It was also received quickly and the tone of the email suggests its author knows what they are talking about - from Kareisha Assanah There was nothing on the email to say who it was from or what company nor any other contact details Initially, I received an automatic response which said that staff member was out of office on 14th March (which was out of date) and referring me to her team if matter was urgent. The response from staff member later same day was very informative. I had to email John Abana during office hours to find out how to become a Resident Inspector. John's email response was professional and appropriate. Responding to texts during office hours No staff name used when replying to a text (with a text or ringing back) 4/27 Responding to texts out of office hours No staff name used when replying to a text (with a text or ringing back) 1/17 Shoppers were asked to send a text to the staff member to request a call back. No name left on my answer message, however did tell me when she would be available for me to call her back. This was an out of office hours task, and Ms Sinclair rang back straightaway as she thought it was an emergency! Very polite on the phone and very clear communicator. I explained it was a Mystery Shopping Exercise and she was happy. I received a voicemail from Jay introducing himself as a housing officer from TVHA, saying he was responding to my request for a call back, and to call him on when I had the chance. He left the phone number. He also texted me just after with the same details. When I texted back, thanks, mystery shopper, he responded that it was ok and Happy New Year :) Very quick, positive and friendly response. Laura was a delight to deal with! I'd sent a text requesting a call back and I just got a text in reply saying, " Hello Mrs Ikar, where do you live? Regards Lavina
Jonathan called back immediately and was very friendly on his call, introducing himself with his job title and the company name, said he'd just received my text and asked how he could help me. I explained I was a mystery shopper and he laughed, said it wasn't a problem, but asked if there was anything else he could do to help. Conclusion Having agreed the scenarios before hand and having a larger sample set makes the following conclusions more valid Although there was a one percentage point drop in overall performance, Property Managers achieved 89% in the final round of shopping this year compared with 57% last year. Overall performance was less erratic this year. There is a clear drop in performance for out of hours responses. Signatures on e-mails are still not consistent. Out of office signatures are inconsistent, and in some cases do not exist merely the message content and no signature at all. Replying to out of hours phone messages was the worst performance in both teams. There were some great responses from staff who responded really well. Staff were very good as responding within time. Clearly staff need to ensure that Email signatures and out of office signatures are consistent Answerphone messages are up to date Texts are signed with name and company name Out of hours contact is responded to Testing 0300 phone number We also tested the 0300 456 2929 phone number with some simple scenarios. We wanted to find out where the journey through the automated switchboard would lead, what departments would be engaged, the members of staff spoken to and whether there was a satisfactory conclusion. The queries that were used were as follows; Can you tell me who I need to speak to about mutual exchanges? Can you tell me who I need to speak to about removing a large sofa that has been left where I live? How do I get a My TVH account? Can you tell me who I need to speak to hear more about the right to buy scheme?
The number of times each option was chosen by the mystery shoppers were as follows 32 Option 5 any other enquiries 1 Option 4 home ownership enquiries 7 Option 3 tenant enquiries 1 1 Option 2 payments Option 1 repairs The reasons for selecting the option were generally given as I believed this was the most relevant option relating to my query Clearly the scenarios were not likely to align with options around repairs (even though we class a dumped sofa as a repair order), payments or homeownership enquiries specifically, but homeowners are part of the mystery shopping panel but they chose not to select homeownership enquiries.. Only one call did not receive an answer Two callers were not given the member of staffs name 38 out of 41 calls were answered by the first member of staff, of the remaining three calls One shopper was told to ring back in the morning One got no answer One was passed onto a member of the homeownership team Overwhelmingly shoppers described the member of staff as helpful and/or polite. Beth from Housing advice answered the phone after I'd selected option 3 (tenant enquiries) and she said that she would be able to help me over the phone get a TVH account if she could just take my name and address. It was very easy to get through to the right person who could help and Beth was very professional, polite and helpful. Hayley answered within 15 seconds of my going through to her number. She sounded surprised at my query at first, so perhaps I should have chosen another option (maybe 4), but she said she could help me. She asked my name and first line of address then gave me instructions on how to go about getting the TVHA account by going to the internet address and then what I needed to do. She also checked the system has my correct email address and mobile numbers and gave me the invitation code that I would need to get the account and how it would be verified. She was very polite, friendly and helpful. It was easy to get through, although due to her initial surprise, I think the phonecall perhaps went to the wrong person... On the whole the 0300 system is working. However, more complex scenarios that would be more likely to cover the range of options would test the system more.