Today we undertook our first trial with Advanced Diploma of Building Design students at Newport campus. This was a GPS-based activity, where students braved the blustery weather in groups and car-pooled to a series of 20 buildings around Williamstown, where they had to choose the building style. When they got to within GPS range of the location (accuracy varied according to the device type), the PushLMS system sent a question with 3-5 multiple choice answers to the group's device. Once answered, the system sent a new site address to the group.
So, did it work?
Well, yes...and no as well. A number of factors came into play today, which I will explain shortly. Some we had control over, and some we didn't. What follows is the story of what happened today.
The tech guys got to Newport at 8.15am to do a last test of the GPS tracking and system in general. All good, notwithstanding the fact that we had only just got hold of iPads and iPhones to bolster the range of devices we were testing.
We had arranged to meet the students at 9.15am in the Newport foyer - ostensibly 15 or so students who would form groups of three and to whom we would give a device each. But when we got there, twice as many students were waiting - another teacher had fallen sick, so the decision had been made (unbeknownst to us) to combine the planned class with another we had originally planned to test with too, but for which the timing hadn't worked out.
Some students had their own iPhones so we set them up on the system - in the end we only used four of our devices, even though there were now eight or nine groups of three to five students.
Students were quite excited and some were keen to get into it. But they were patient too as we had to spend some time sorting out the bedlam of phones, groups and student logins with the unexpectedly large number of students.
So off the groups went. I went 'embedded' in a car with one group. We turned up at the first address listed on the device, with 30A as the house number. But the number wasn't to be found, and the house the GPS was pointing to said 28. All the groups were then trying to figure out where 30A was (it was indeed meant to be no 28). Apparently this was due to variations in official records of house numbers supplied by staff to our team. The lesson here is that, although our team had tested locations' GPS tracking the week before, we had not done such real-life testing of locations together with the Building Design staff. This would have picked up the discrepancy.
Then the system abruptly decided to stop working - our PushLMS server had crashed. Students were wandering around the street trying to get the GPS to generate the multiple choice options. A few frantic phone calls to base and we decided to herd everyone back to the campus while the team tried to solve the problem (which ended up being server overloading as two or three students per group had tried to access the system). The lesson: we need to do more server load testing. Although we had tested for 3-4 devices at once, we were not prepared for such a big group, and especially not for multiple students logging in with their own devices too.
Once back at campus, the students took a break while the team got the server up again. We prepared a hard copy version of the questions just in case (which really we should have done in advance as a Plan B). Once up, the team decided to release groups to the site one at a time in a staggered manner to not overload the server. This worked, but lengthened the time to undertake the activity and also meant that some groups were waiting for quite a while before they were able to do the activity.
So off I went again, this time with the first group to go. Here's what I observed 'on the road':
- the activity basically worked very well. Students were actively discussing building styles and referring to the text book as they parked near each building to get the list of question options on their device.
- it took longer than expected. The group I was with took over 90 minutes to do 12 of the 20 questions (until one stuent had to leave to pick up her child)
- there was some confusion about directions on how to get to the next site. Ideally we should have included a paper map with sites marked rather then relying on Google Maps on students' mobiles.
- there appeared to be some glitches in our system such as (in our group's case) the correct answer appearing on the top of the list for most questions. Also one or two questions seemed wrong or impossible to answer (again, which a joint staff-tech test would have uncovered), which threw the students somewhat. A student also suggested that an 'undo' button and ability to go back to check addresses would have been useful.
- students were unsure and anxious about whether or not they were being assessed on their responses to the questions (this was also compounded by the glitches listed above). It seems this was maybe not made clear to them in advance. Apparently attendance was compulsory to pass, but beyond that nobody seemed to know what was or wasn't being assessed.
- at least two students using their own phones ran out of power during the exercise
The big lesson
The big lesson, though, appears to be our assumption of the centrality of mobiles and GPS to this activity. Students pointed out that perhaps a list of sites and questions on paper plus a photocopied Melways map would have worked just as well, and maybe better (ability to plan, no crashes etc). This is strangely ironic because it was the mobile capability of PushLMS that led to this activity being devised in the first place!
Our assumption had been that the GPS-generated questions and the 'treasure hunt' element (not knowing in advance where the next site would be) would be engaging for students. Indeed, they may have been, given their enthusiasm at the outset and their keenness to play with the GPS capability of the system (its reading of distance in metres from sites). But although (being a thankfully good-natured group) they seemed to still retain their enthusiasm after the initial server crash and the waiting around, it is possible to see how anticipation can easily lead to disengagement if the technology isn't working as it should. Most of the 10 or so students I spoke to afterwards suggested that paper would have been easier and better.
Which makes the entire mobile-GPS proposition possibly redundant, even though the activity was praised by all students as worthwhile for their learning. In retrospect this seems quite obvious. But that's looking back, in keeping with what Fisschoff (1975) calls ‘creeping determinism’; the sense that solutions can seem obvious or even inevitable in retrospect, summarised by Malcolm Gladwell (2003) as “What is clear in hindsight is rarely clear before the fact.”
So. we've learned a lot today, including:
- don't get seduced by the technology. Sometimes simpler options will do the job as well or better (we usually know that, but it got us this time!) In this case, there was no time trial involved or competition, so no real benefit to using mobiles except for the gee-whizzery of having a question appear when at a location and perhaps collating results and student locations on the fly (as opposed to the library trial, which will involve holding back questions on the mobiles for a minute when previous questions are answered incorrectly)
- Apple devices read HTML5-based GPS much better. The tablets are by far the best, possibly due to their size and therefore antenna size
- our project assumption (for pragmatic resourcing reasons) about focusing on the tech stuff and letting program areas develop and supply content is blatantly wrong and doesn't work. These projects need to be developed and tested hand-in-hand with subject areas throughout the devt phase. It's very hard for VET teachers who are constantly pushed for time - projects like this are just one more extra activity to fit in. As a result (and we've found this with all the areas we've worked with) activities tend to not be fully thought through and checked carefully, despite teachers' enthusiasm and best intentions.
No comments:
Post a Comment