Carnegie Learning Assistment Interface

Technical Overview

In the initial conception of the CLAI project, our group was chartered to design a front-end for teachers to see the results obtained from the Assistments system, which would be stored in a database. In the system architecture, this database is then our primary link to the Assistments system.

However, at this early stage of the Assistments research project, the architecture is still in early stages of development and very much in flux. At the time of our group's joining the project effort, there hadn't been much initiative yet in processing or analyzing student data, and thus there had been no reason to put a lot of effort into the database yet.
  It was imperative to have real student data (from the beta tests being done in Worcester schools) in our prototypes. It would lend more credibility to our testing in the following ways:
  • If the data displayed in the prototypes was inconsistent or irrational, this would distract the teachers from the test tasks, in that they would be wondering why the information didn't make sense (such as a factoring problem requiring Pythagoras' theorem).
  • If we showed a teacher data from her real life students she would be more involved in the task and our results would be even more indicative of how she would really benefit from the interface.
  • Mocking up a large volume of fake data can take a really long time.
This project is slated as an HCI project, meaning our goal is to work on the user experience and user interface. In a way, working on the database and architecture is out of this scope and not something our group was expected to be working on. However, people in the field of HCI usually come from a very varied background, and sometimes digressions like these come our way and it turns out that the best solution is for us to tackle them ourselves. And so it was in this project.  
We started by designing an ideal database, to represent all the data that we foresaw to be necessary for the interfaces we would like to design for teachers. Our goals were to show teachers how Skills related to Standards by showing how their students performed on related Assistments questions and steps, so that's how we organized our database design.
  • The "School Structure" section holds data to identify students, classes and teachers
  • The "Assistments Activity" represents all the data collected while students are working through assistments questions
  • The "Questions" section is a repository of all the data pertaining to the assistments questions that the students work through
  • The "Skills" and "Standards" sections describe the skills and standards that assistments questions or scaffolds can be categorized into.

CLAI database design diagram: [ Visio ] | [  GIF ]
At the same time, we got in touch with the developers at WPI in order to get access to the data that had been collected in their beta testing. In trying to make use of the data, we felt their database design was not designed in a way that would enable us to pull the relevant information that would be showed to teachers through our prototypes. There was a lot of complexity due to the needs of the software system's architecture and implementation, and also the needs of the researchers who would be doing complex data analysis on the student data. Also, due to the system being in beta and not very stable, a lot of the collected data was corrupt or inconsistent.

WPI database design diagram: [ Visio ] | [  GIF ]
  We were faced with 2 possibilities. One was to take a dump of whatever data the developers had at that moment, find a way to reformat it so that it would fit our database design, and not have to worry about that again. The other was to spend more time communicating with the WPI team in order to understand their system requirements, and design one combined database that would serve the purposes of the software system, the research and data analysis initiative, and the teacher interface. Even though database work was pretty far from the HCI-related experience we wanted to be getting from our Masters project, we agreed that for the good of the Assistments research project as a whole the second option made more sense.
A very large proportion of our time during the first half of the summer semester was spent in designing a database that would be useful for the whole Assistments research project. The structure can be broken down into 6 areas:
  • "Pre/Post Testing" holds data from paper tests given to students before and after they used the Assistments and is part of the research analysis
  • "School Structure" represents students, teachers and classes, and curricula which list which assistments students are working on.
  • "Assistment Activity" holds all the data collected by the system while students are using the Assistments software needed to display data to teachers and to analyze student performance
  • "Assistment Questions" represents all the information needed to define assistments questions and scaffolds. This section may be in flux as more data such as behavior recorder diagrams get transfered into the database or the developers move to a more object-oriented structuring of the questions
  • "Knowledge Components" describes knowledge and skills and their hierarchies, and the mappings to scaffolds and assistments, as determined by the educational experts.
  • "Standards" holds information about how assistments relate to different state standards.

Assistments combined database design diagram: [ Visio ] | [  GIF ]

With the database designed reaching a more stable version it was now possible to implement it in order to have a fully functional back end to feed real student data to our prototypes. This was achieved by querying data from the database, converting it into XML and transferring it through web services, and receiving the web services with ActionScript in order to display it in Flash.
A database following the new design would not be implemented by the Assistments developers, since they were still in the opening stages of their 4-year research project, so it was not high on their priority list. We, on the other hand, were nearing the end of our 2-semester project, and had decided that having real student data was a top priority. So we implemented a subset of the structures that was necessary for us to run our prototypes, outlined here in red. After manually processing the data obtained from the original Assistments database we were able to insert it into our database implementation in order to use it for our user testing.

Implemented database design diagram: [ Visio ] | [  GIF ]
Database structure creation SQL file: [ SQL ]
These tables held all the raw data produced by the Assistments system being beta tested in real schools that was necessary for us to display the information we wanted to display in our prototypes. But in order to convert such data into percentages and scores representing student performance, further processing was needed. This should normally be done in complex SQL nested queries, stored procedures or views, but mySQL (the database software being used for the Assistments system) doesn't have such functionality yet. So we did some of this processing ahead of time and stored it in the database as regular tables, and treated them as views.

View relationships diagram: [ Visio ] | [  GIF ]
View creation SQL file: [ SQL ]
Now the performance data had to be extracted from the database and sent to our Flash prototype. There were several options for ways of doing this, and the simplest one seemed to be web services. This involved writing java code query the database (both tables and "views"), setting up an Apache server with an Axis server to convert the result of the java code into XML, and write ActionScript functions to interpret the XML data into something that Flash can use. These ActionScript functions could then be called from the graphical widgets and interfaces that are our prototype.
 

About Us | HCI Masters Project 2004