Today we performed the first full scale trail of the mobile geo location quiz. Two groups of students had be brought together to perform the online assessment. We provided two iPads, two iPhone 3Gs and 2 Samsung Galaxy Ss for student’s to use. Students were also encouraged to use their own devices provided they were equipped with GPS and WebKit/HTML5 compatible web browsers. A group of students soon emerged iPhones in hand. A number of students also had BlackBerrys which we were surprised to find appeared to have a WebKit based web browser and rendered the application as beautifully as Safari or the Android Browser.
The students were given the URL to enter into their phones and then organised into groups. Everything appeared to be working and we concluded it was safe to send all the students out at once. The production team then returned to the command centre we had erected in the conference room where we prepared to watch the students’ progress via the tracking application. A number of lines had started forming on the map when we received our first phone call. It appeared that a group of students had run into a problem with one of the iPads displaying a blank page. Dale and I went outside to meet with this group to help troubleshoot the problem when we began receiving similar reports from other students.
Upon returning to the command centre we became aware that the server we were hosting both the application and the database on had been brought to its knees by the traffic being generated by the devices. While the downloads from all the devices were not excessive, the tracking code was uploading coordinates to the server every few seconds from over 10 devices. This was likely the cause for the server crashing. While we waited for the students to return to base we attempted to migrate the application to another server, this would have worked had the database not still been on the server which had been brought down. While the students waited patiently we restarted the server and waited for it to come back up.
Taking on-board the lessons from the first attempt we decided to stagger the next set of tests and send out one group at a time limiting them to one device per group to minimise server load. This approach appeared to work and the server remained active for the remainder of the tests. One of our team members who went out with the first group to document the test reported back that the application was placing the correct answer at the top of each multiple choice. This bug was something new that hadn’t been encountered during any of our previous testing and has been resolved by randomising the order of the questions each time they are loaded. The test took over an hour and a half to complete due to the number of locations, 20 in all and the students needing to stop an analyse each location to assess the correct response. The number of questions could have been reduced as there was one style that was repeated and some questions which could have been removed.
Due to the length of the test and the reliance the 3G and GPS which are two of the most battery intensive components on a phone aside from the screen, the battery depleted for one of the groups. Luckily they had a spare BlackBerry from which they could log back in and resume the test. It would have been good to provide the students with in car chargers to mitigate the chances the running out of power.
Some of the students commented that they could have done just as well with a paper based assessment and map of the circuit. This has led us to ponder the advantages of the platforms we have employed. Besides the integrated nature of the app, tracking and instant reporting it does appear that the test could have been conducted as successfully with an analogue paper based model. Another option could have been a desk based assessment using Google Street View or similar service.
No comments:
Post a Comment