Research - Technical Issues

Below are some preliminary thoughts, before diving into the ocean of details of research problems, solutions and opportunities.
The problems and perceived problems should be thought of in the context of the broad issue of ongoing research as the tool for ongoing improving and adaptation of education to the constantly changing society - Looking ahead.

At the heels of the thought of research, comes the thought of experimentation, and the natural negative response to experimentation has already been discussed - indeed not exhaustively, but one has to start somewhere.

Another issue is that of research interference with the optimal course of learning, when the needs of the research seem to dictate an action that would not have been done otherwise. The most trivial example is that of asking students to provide their opinion, which will be part of the input for analysis in the research. The time spent by the students answering questionnaires could have been used for other ways of learning. Note, though that in some education systems, this is already the case. Countries are concerned about their position in the international tests (PISA, etc), and they put a lot of effort into intermediate testing aimed at preparing the kids to the international test. Also, since the learning disabilities are a hot issue, in some education systems tests and assessments take considerable amount of time in the lower classes of primary schools.

Looking ahead again: The problem of non-study-related activities required of the students can be mitigated by being aware of it and trying to minimize the disruptions. We must make sure that any action required or requested of the students, is truly necessary. Part of that, is the use of non-intrusive techniques for feedback and assessment, such as analyzing students’ behavior as it is reflected on normally accumulated records. Such records may include:

• Assessments the students would have taken anyway, right after the event the research is focusing on

• Assessments the students would have taken anyway, a while after the event the research is focusing on - as a sort of follow-up / retention feedback

• Assessments the students would have taken anyway, in fields other than the field the research is focusing on

• Normally scheduled satisfaction assessments of students, parents, teachers, tutors, etc

• Records of students choices of next-course following the event the research is focusing on

• etc.

The usefulness of video recordings was mentioned before, together with its accompanying need for anonymization and Chinese walls. But there is also the technical and logistic issue of analyzing the recorded video - Hours of it? Thousands of hours? Ok, so we can have many researchers analyzing different parts of the video. Such analysis requires a clear methodology, to ensure that different people analyzing different stretches of video, will come up with compatible partial results, so these results can be aggregated into a coherent overall result. If we have a very strict methodology, we will be able to create such a coherent picture, and even experiments that are repeatable - that can be repeated by different research groups, with consistent results. This would make great science, but we are likely to limit the types of results we may get, and therefore miss some of the picture. For example, Counting the number of times a teacher asks a question and waits for an answer from the class is a strict and objective way to analyze a video, but if the researcher is limited to this measurement, we may miss other interesting patterns, such as the length of the median answer from the class, or even the more subtle indications of the median amount of interest the students display. So, even with a good methodology, we are going to find ourselves in an imperfect point between producing consistent results with limited field-of-vision on one end, and producing deeply meaningful results with no repeatability (and hence verifiability) on the other end. I am sure this has been discussed academically in social studies, but I suspect that discussion has been more relaxed than what is needed for a systematic research based education system.

There are also some easier questions regarding analysis of video materials: How to search for an event? How to correlate actions by the same person in different parts of the video? Or maybe different days? Or maybe different research projects? Or their test records? How to block-out the faces for anonymity, where facial expressions are not important? How to count spoken words? How to measure the tone of voice? How to measure stress? How to measure level of interest? How to identify idle time? …
Some of these - the more the better - can be facilitated by technology. My guess - as a technologist - is that they are easier than meets the eye.

Back to more difficult issues - maybe the crux of the education research problem in an education system that allows different ideas and methods: How to make research in one school relevant in another? If the school system is highly standardized, there is no problem. But in a distributed school-system, where different school may have different emphases and goals, this is not trivial. How do we correlate the results in a school that emphasizes academic achievement with the results in a school that emphasizes cooperation between students? One option is to have different research projects and methods, managed by the schools themselves. But this way we will lose the system’s ability to understand the schools, and find ourselves nearing educational anarchy, which is more than what most of us will be comfortable with. The “system”, the state, the regulator have to have some understanding of the schools.

One direction for thinking about conducting a homogeneous study of a heterogeneous population, is to do "qualitative research". This method can generate intuitive understanding in the individuals involved in the research, and it can give rise to some questions to be researched further. But I can't bring myself to thinking about it as producing answers. It is too much of a personal experience by the researchers, and therefore not repeatable, and doesn't lend itself to neither scrutiny nor follow-up. There is a need to come up with quantitative research methods for the "soft" field of education. Tricky.

Another direction would be to break up the research subjects into small components that are measurable and can be compared across different schools, even when the goals of the different schools may have a different nature, such as individual achievement vs. social responsibility. Such component-subjects may include "individual achievement in basic arithmetic learned by rote". This component-subject may be more important in one school, and less important in another. In some schools, it may even be undesirable. But the quality of the teaching/learning of that particular component-subject can be measured, analyzed and compared across many types of schools.

The above suggestions are incomplete and a bit naive. To get better answers, it is necessary to do more study, thinking and mostly - more research about research. Quantitatively.