It's Complicated

The field of education is a complicated field for research. There are many variables that affect the success of a student. For example, thinking about a particular class the student takes, such variables may include: The student's ability in the particular subject being studied, the student's prior knowledge, the student's expectation about their ability, the student's attitude towards the subject, the number of hours the student slept the night before, the student's health that day, whether the student ate that morning or before class, the teacher's ability in the particular subject, the teacher's expectations for the particular student's ability to learn in general or the subject in particular, the teacher's expectations for their own ability to teach the subject, the teacher expectations from the class in general, the amounts of light, noise, oxygen, CO₂ and scents in the class, the class's attitude towards the subject, the teacher and the student, the time of day, the class before the current class, the class scheduled after the current class, the time of year, the quality of the class materials, whether the student brought the class materials to class, etc. etc. etc.
Many of these variables interact with each other. Many are not well understood. Many cannot be easily determined. Many more variables are not known at all.

All this may be further complicated by adding the possible presence of a researcher or research aids such as a video camera. These are likely to affect the behavior of the teacher, the class and the particular student. Also, there are the additional limitations of privacy concerns, ethics, legal issues, lack of funds for research, academic pressures (just publish), researchers own pressures (just complete the degree), and probably more. And then there is the small issue of actually conducting a well planned, validated, executed, analyzed and reviewed research.

These, and similar issues are often alluded to as reasons why education research is by its nature "softer" than some other sciences. (Notice how "soft" the statements here are?) Since it is a soft science - or knowledge domain - it doesn't need to answer to the demands of the exact sciences. Most specifically, conducting a study that is not repeatable and coming up with explanations that are not falsifiable is counts as research.

But,

How special are these characteristics of education research?

Let's consider the amount of variables: In biochemistry, just looking at the issue of metabolism as it is described in wikipedia illustrates the complexity. More specifically, looking at the metabolic network of a bacterium may give us a hint about the complexity of higher organisms:

... and yet, we study human metabolism, conducting repeatable experiments, come up with refutable theories - and often do indeed refute them - and keep building our knowledge. So the complexity does not preclude rigorous scientific handling. Even in exact sciences such as physics we have annoying interplays between variables that are hard to pin down, such as the Three Body Problem, where we don't have a way to determine the behavior of three masses (e.g., stars) moving and attracting each other. And yet, we expect physics research to be very exact, and for the most part we physics research lives up to the expectations. The complexity and incomplete data do make a researcher's life more difficult and interesting, but that's not such a bad thing.

Then there is the problem that the subjects of education research - teachers, students, parents, administrators, etc - are often conscious of the research, and that fact affects their behavior. A researcher in a classroom, a questionnaire, a video camera and other players and tools all end up affecting the very behavior being researched. Thinking about the aspect of Observer Effect concerned with the observed changing behavior as a result of being observed, one can see this is a significant issue in education research. But remembering again that other sciences have to deal with similar issues, such as the Observer Effect in Physics, and seeing physicists courageously dealing with it without giving up h rigor of their research, can inspire us to aim that high in education research, too.

Another aspect of the observer/observee interaction is that of Observer-expectancy effect, which focuses on how the researcher's behavior affects the result of the research. This is a real issue that needs to be addressed. Part of the solution is minimizing the direct interaction between researcher and participants, when possible. Another part of the solution is to use blind and double blind experiment methods. Yet another part is to train researchers and research assistants very well indeed. There are probably more ways I am not aware of right now. The remainder of the problem is too small to justify much tolerance for weak research methods.

Creating a working model of the activity of teaching and learning is a problematic endeavor. Again there are the variables described above, and to make things worse, we are dealing with humans. They are fickle. It's hard to generalize about them, and it is not politically correct to try (could that be part of the issue?) One might say that every model we create would be wrong. I believe that's true, but "Every Model Is Wrong" is a statement we see mentioned in the context of many sciences, including the exact sciences. This statement is usually immediately succeeded with a statement of how models are still useful, even though in the purest sense they are wrong. The weather models are useful in predicting short term weather patterns. It's ok that these models can't predict longer term weather, as long as we are aware of their limitations. Bohr's model of the atom as a nucleus surrounded by electrons zooming around in clear paths is quite wrong, but it is so useful that we still teach it to kids, and it works nicely to explain a lot of chemical and physical behaviors. Philosophically speaking, humans may not have even the potential to conceive of a perfect model to describe anything significant. But even if a model is ultimately wrong, we mustn't give up trying to improve it. If we have an imperfect model, that is useful to describe a very limited subfield of education, this is very good. For example, if we have a model that works only for "teaching and learning English as a Second Language to a group of 10-15 adolescents whose mother tongue is a derivative of Latin", and if that model allows us to correctly predict what techniques would work to improve the teaching and learning, then we have an immensely useful tool in our hands.

Bottom line: Yes, it is complicated. So what?