The first day of the Australian Learning Analytics Summer Institute (ALASI 2015) was packed with things to think about. Lyn Alderman's opening keynote was strong on solid practical advice about ensuring the relevance and quality of higher education courses and drew on her work at QUT. A key message from Lyn's talk was the simple but critical idea to have a dedicated, educationally mindful team mediating between academics and institutional data analysts - sort of like a clutch to ensure the wheels turn smoothly. Lyn used lots of authentic examples to illustrate the work of her team and their course performance model - engaging and good stuff.
After morning tea, I headed off to Identifying and contacting disengaged students in Moodle presented by Jean-Christophe Froissard and Danny Liu from Macquarie. A really well prepared and thoughtful session that had us all using their Moodle plugin on mocked-up course data and playing with a dizzying array of parameters available for teachers to set. I do struggle with the assignment of a risk value to individual students based on a set of parameters, whether teacher selected or not, but the ability to easily flag students based on selectable and flexible criteria (grades, discussion posts, logins etc) and then contact them as individuals or as a group has clear benefits. Lots of good discussion and thoughts from the floor and good to be sloshing about in the muddy intersection of theory and practice.
Interesting and thought-provoking discussions over lunch were followed by a session on Writing Analytics. Simon Buckingham Shum, Simon Knight and Andrew Gibson introduced writing analytics in general and their AWA tool in particular and provided an opportunity to try AWA. The tool is designed to highlight features of reflective and academic writing and Philippa Ryan spoke about using AWA with students in her law class. There were many thoughtful and a few difficult questions from the floor during this session and I think these highlighted the issues and challenges of working in this area really well.
The nub of the problem is that as soon as you define a set of rules (you might say a grammar) to describe a language, there will always be some reasonable expression of the language which will fail to be captured by your rules; Sapir's dictum, "all grammars leak". Why? Well, much like goats, once we have perfectly well understood the rules we then proceed to chew on them and reshape them in order to express our inner goat. Taking a statistical approach to feature identification may help improve the situation but some old goat will always come along with a new expressive feature that just will not be captured.
AWA is designed as a teaching and learning tool and by constraining the context of use this mitigates the problem to a degree: there are some basic, well understood features of academic and reflective writing that we teach and that AWA will capture. It was encouraging to hear Philippa's experience, that when used carefully, in an appropriate context and with guidance, AWA can be used to support students to develop reflective and academic writing skills. Nevertheless, in common with similar tools it can also frustrate and it was good to see examples of this too. Finding the sweet contextual spot where frustration is minimised seems to be key. I did paste a reflective piece from this blog into AWA. Sadly, no reflective features were identified but then again, it is hardly reasonable to expect AWA to speak goat!
Ruth Crick talking about Layers, Loops and Processes: challenges of authority and interpretation
in the formulation of actionable insights in virtual learning ecologies was a highlight. The opportunity to do the learning power survey (see http://clara.learningemergence.com/), reflect on it and see how the ALASI cohort as a whole fared was fascinating. A carefully structured, qualitative approach to understanding key processes involved in learning was illustrated in a very personal way - great stuff.
A delightful jazz trio, the product of their mindful musicianship bouncing off the walls, rendered the tail-end poster session something of a fizz. Nevertheless, fortified with food and drink, a handful of hardy presenters and a few noble delegates carried on regardless and brought to a close an excellent day overall. If you missed our poster, here it is and the abstract here.
Labels: ALASI, Higher Education, Learning Analytics, NLP, Text Analytics
