Monday, March 7, 2011

Teaching the Teachers

I have been thinking about cats today. I almost killed one through the application of a stumbling-pre-dawn misstep this morning. The cat is OK, but it made a sound I would have thought could only have been produced by the vigorous application of severe torture. Every morning one of the cats in my house closely orbits my feet while I stumble to the kitchen; every few months I step on it. You’d think it would learn.

There are things that cannot be grasped with the feline brain, but it is difficult to tell what they are. If we reduce the complexity of the brain to less than that of a cat (no this is not a sendup for a Glenn Beck joke…) to something like a bird it is possible to dispassionately weigh behavior against applied stimulus.

Burrhus Frederic Skinner worked extensively with stimulus-response training of simple-brained animals. He famously taught pigeons to play ping-pong; though they did not play well. Skinner showed that rewards (positive reinforcement) were much more effective than punishments (negative reinforcement) in altering behavior. Skinner’s work has been used as a critical part of the foundation that modern teaching methods are built upon.


Many experienced teachers will tell you that Skinner’s work is wrong.

“What works for birds” they will tell you “Doesn’t work for people”.
They have data to back up their claims. The data is so universal that it can be generalized as follows:

  1. Students who perform exceptionally well at a task, and are rewarded, perform much worse at the task the next time they attempt it. The strength of the reward has no effect on this relationship.
  2. Students who perform exceptionally poorly at a task, and are punished, perform much better at the task the next time they attempt it. The strength of the punishment has little effect on this relationship.

The above generalization can be supported with objective data from a variety of sources; I will provide a relatively dependable way of generating your own later on in this entry. It would appear that the human mind works opposite to the way a bird’s brain works, at least as the workings relate to behaviorism. But if we scratch the surface of the experienced teacher’s (ET) knowledge we see a slightly different story.

ETs believe they are speaking about how their students learn from them, but what they are really relating is what they think they themselves have learned about their students. If the Skinnerism holds their learning would be reinforced better through rewards. To stick with teaching long enough to become an ET requires that the teacher partially responds to the positive stimuli of successfully teaching. What they are correct about in their teaching methods would provide for a more lasting impression in their minds.

The more ET notices the outcome the greater is the impression on their mind. This means that if the deviant ability (either very poor or very good) is exceptional the effect that ET’s teaching method produces is more important to that ET. Similarly, the more ET cares about the student’s progress the more lasting the memory of the effect of ET’s response to the ability.

The problem is that the more ET cares about the student or the exceptional ability the more the ET reinforces in their mind a pattern of responses that does not exist. Once ET recognizes the positive effect of their punishment on their students they have created a delusion that will only be reinforced by continued attention to the data source upon which it is based.

Let me propose a data source that will illustrate this. You will need to supply ET. I suggest that the test is conducted by e-mail. You could get a bunch of teachers to reply to you a couple of times a day in a scenario-based test like this:

Tell the teachers that you are going to test the effectiveness of years of teacher experience on their ability to moderate rewards and punishments to a group of students. It can help if you make clear that neither the rewards nor punishments are severe. The teachers can provide a number from 1 to 10 on how great the reward or punishment should be. Then you provide the teachers with regular performance updates in the form of numbers. The teachers are regularly given a number from -10 to +10 which they are told represents their anonymous students’ performance. After the test the teachers provide answers to a question like: “Did the reward or punishment prove the most effective way to alter the students’ performance?”.


Then to generate the performance numbers you flip coins. Since the mean behavior of a coin toss is HTHTHTHT… we assign a 0 to any sequence of HT or TH. If two heads are flipped in a row we interpret that as a +1. Two tails in a row generates a -1. Three heads generates a +2, four heads a +3, five +4, six +5, seven +6, eight +7, nine +8, ten +9, and eleven heads in a row generates a +10. The same scoring is used for sequences of tails in a row, but the numbers are negative.


Because of the statistical property of “regression to the mean” what ET will see in the above thought experiment is this: whenever ET provides reward for exceptionally good performance they will usually see the performance decline, and when they provide punishment for exceptionally poor performance they will usually see the performance get markedly better. If ET examines their notes and modulates their rewards and punishments they will see that, on average, the punishment does a consistently much better job. The more the teachers are made to care about the outcome the less they will be able to identify the true randomness behind the performance numbers. If one imposes a slight linear increase in the performance values it is almost impossible for even the slightly concerned ET to see the randomness in the values.


I have written on how imposing patterns on the mind leads to conceptual bias, and how these patterns can be reinforced by a lack of data. Here is a situation where the inerrant patterns in data with true randomness reinforces the concept of a very non-random pattern of cultural responses in the mind.

I have framed this entry in the form of a discussion about teachers, but the effects of reward and punishment are used in many aspects of our civilization. Conflicts and wars are often understood as forms of international punishment.

Our spending priorities also mirror what experience-based learning teaches those most interested about rewards and punishments. Estimates had suggested that right about now observed trends would have more of the California budget going towards prisons than education. The estimates had $15.3 billion California state dollars going to education, and $15.4 billion to prisons. The problem economy made that happen sooner. Higher education is the worst hit; 7.5% percent of the CA budget goes to higher education, and 11% goes towards prisons. The per individual discrepancy is even larger; CA pays over $45,000.00 per year for an average prison inmate, and a little over $5,000.00 a year for the average California State University student’s education.

Perhaps California should start hiring their laid off teachers as prison guards. I’m sure ET would see the logic in that.

4 comments:

Joe said...

I really do not like B.F. Skinner since he lead (mostly) US psychology into behaviorism where you can train people by environment by positive or negative reinforcement leading to learning or changed behavior, and standard curriculum in US is still vestiges of that and crap Freudian stuff, the “learned” folk follow the crowd and regurgitate stuff blindly, but as a fan of evolutionary psychology, allot of behavior is just ingrained and people are different by genetics, noting you can do with some. Hate too say it but people are born different, some with higher or lower abilities, skills, aptitudes or what sets their testosterone or dopamine levels off. As a teacher, tabula rasa is limited or over-rated, I realize what you start with is often more important than what you do as far as outcomes - please try to teach the pigeons calculus.

Kahneman and Amos Tversky did some Skinner experiments in the 1970s. The Israeli Airforce rewarded their pilots who did well on flight maneuvers, praise, of base leave, or standard Skinner positive reinforcement, and maybe extra work for those pilots who performed poorly, have to reread the paper. Well, turns out the “good” pilots did worse on subsequent flights, and the “bad” pilots did better. Well, stands that positive reinforcement was a failure and negative reinforcement was a success, Skinner people could not figure out why the psychological methods were not working. But maybe not, just some pilots had a bad day and some had a good one, the reinforcement did nothing, performance had nothing to do with reinforcement. Lets pick from a deck of cards, high card wins, well lets have positive reinforcement on the winners, wine and dine, money rewards, and negative reinforcement and torture the loser picking cards people and redo the experiment, well the prior winners are likely to perform worse next round and the pror losers likely to perform better. Exactly like you note for ”mean reversion.”

If you look at the psch lit, there is also a thing called loss aversion, because of diminishing marginal utility, a gain of say $10 is much less than the equivalent loss of $10. A wolf will say spend much more energy defending a 10 acre domain than to acquire a 10 acre larger domain. More people will pay cash to avoid a $10 credit card surcharge, than will pay cash over a credit card to get a $10 cash discount abet the end net prices are the same. Using framing and loss aversion, glass half empty or half full, decisions over the same things can be manipulated.

Now, for those who are uncertain, the fear of going to hell should motivate more than the reward of heaven. Odd thing is that the bible mentions little of either, but fear of god is a much more motivating force. But look at the cultural reinforcement - take a true believe and the flock, preaching on the benefits of Christian behavior or the punishments of the devil, we would get the same mean reversion problem with the preacher. The good flock gets worse and commits more sins, the bad in the flock gets better and commit less sins, reward and punishment for the preacher (not the flock) would lead to a learned skinner behavior of saying the benefits method does not work, and the punishment method works better, thus teaching/learning methods by the religious should as a meme evolve to the latter towards fear of god method as more effective. Would that work better than the old temple prostitutes or reward method that went out millennia ago?

Joe said...

Add...
There is also a degree of risk aversion and people with higher IQs are less risk averse, maybe better innate skills for judging probability or a more linear utility function. Ok risk of going to hell or to heaven, have faith and believe or do not believe, but every study shows a negative correlation between degree of faith in god and IQ level, the more stupid you are, more likely you believe in god, more intelligent, more likely an agnostic or atheist. Or maybe the less endowed with brains crowd find it a more effective method to follow or copy others than to think since that is a relatively more expensive behavior. Ok, I want to teach my local house plants calculus, and will reward and reinforce them with water or starve them dependent on if they can do the problems answering correctly with number of new buds or leaves. Crap, eventually they all died.

Joe said...

“Higher education is the worst hit; 7.5% percent of the CA budget goes to higher education, and 11% goes towards prisons. The per individual discrepancy is even larger; CA pays over $45,000 per year for an average prison inmate, and a little over $5,000 a year for the average California State University student’s education.” - OK - well, CA is not a person or Santa Claus, more that CA taxpayers pay for CA’s system of overly subsided higher public education as a form of socialism. Why should a CA resident without kids have to foot the bill for the college education of some other? It is plain theft by collectivism and democracy. If you want less of something, like income, just tax it more like cigarettes. The government should not even be in this market, fiscal ups and downs cause big swings which would not happen so often if in the private sector - how many people did Harvard or the local parochial school lay off due to state budget cuts this year? State regulation also creates a morass of bureaucratic waste for higher education, abet not reaching the level for K-12 (look at Wisconsin if they want to keep the high salaries and generous pension payments to government educational employees of the system up at cost to taxpayers as opposed to beneficiaries similar to socialist state owned coal workers versus coal consumers in England under Thatcher - public by vote should not decide). Keep the prisons, they provide a public benefit for all people living in CA by keeping the thugs off the street committing crimes, but release the masses of non-violent drug offenders wasting $45K a year on housing, money better spent on rehab at a fraction of the cost.

adult onset atheist said...

Joe; as interesting as your comments always are your mention of Dan Kahneman in context is especially marvelous. Since he was a psychologist who won the Nobel prize in economics he is almost your compliment; since you are an economist who studies psychology. I almost used quotes from Kahneman’s Air Force training experiences in describing regression to the mean, but went with the thought experiment approach instead. Did you know that Kahneman got his PhD from UC Berkley? So you might view his work as “plain theft by collectivism and democracy”.

Theologically I do not buy your implication that the bible’s god is benign because heaven and hell are not mentioned as often in the bible as they are on Sunday morning. The Old Testament god provides quite a lot of punishment by siding with this or that group of individuals on this side of the curtain of eternal night. I think (without looking it up) that there is even one battle where OTG takes out a million pagan-type warriors in a single battle by siding with Israelites. Of course I am often accused, with good reason, of waxing snarky when I start talking about OTG.

In addition to weighing decisions with aversion to loss, people also weigh heavily towards over-valuing free or nearly free items. There has been work that demonstrates consumer’s willingness to pay more to obtain a free item than the purchase price of a non-free item. Basically there is some level of complexity with the human mind that goes beyond what I am comfortable saying I understand. I have, however, learned calculus.

It is wonderful that I can elicit such terrific response from you when I write about these ideas from simple statistics. I’ve got one more that I am thinking about now. That one is “error by pattern bias” in animals (like humans) with more complex brains. I have not decided on how to frame it though. There are so many ways to present an idea, and whenever I am done writing on an idea I realize that I have chosen one of the ways to present it over all the others.