Category: Learning Theory

Posted on

2017 and onwards.

What will 2017 bring?

I am studying to be an equine behaviourist but so far am unsure whether I want to practice as a behaviourist. The equine part of the equation seems to be the easiest component. Changing peoples long held believes is very difficult, so many don’t even understand the basics of how animals learn. I don’t blame the average horse owner as they are not taught this at riding schools or even in colleges at diploma level.
If people are using pressure to motivate horses they need to understand that it is the relief of that pressure that reinforces the behaviour. This is basic negative reinforcement but I did not learn about this from the British Horse Society or even when I was doing natural horsemanship. I did learn that it is the release that teaches the behaviour but not that it was the use of an aversive stimulus nor was negative reinforcement ever mentioned.
It was only when I investigated clicker training that I learned about positive and negative reinforcement. The more I learned the more convinced I was that positive reinforcement is better for the emotional health of the horse, it gives them a choice. They can say “no” instead of being too afraid to object due to the adverse consequences of non-compliance. Even when I was doing natural horsemanship the horse was not allowed to walk away as this was seen as being “disrespectful”.

Benny taught me so much – he was very adept at escaping the escalating aversives and he introduced me to positive reinforcement.

Mojo is teaching me even more, horses can teach us so much, if we listen, than any human can.
We do not need to subscribe to any particular genre of horsemanship, we need to learn as much as we can from as many sources as possible. Only then can we truly decide what is in the best interest of the horse. To be blinkered or brainwashed by clever marketing is very limiting but unfortunately very common.
So I do find the human animal very hard to understand  – it is the human who has to change if the horse is to have a better life.

Posted on

Reinforcement Spectrum

If we look at the use of negative and positive reinforcement as a spectrum we are all on this spectrum, whether we know it or not.An image of the linear aspects of reinforcement
Some are at the extreme ends of the spectrum, so they only use positive reinforcement or only negative reinforcement.

My personal view is that we can be anywhere along this line as long as we aren’t extreme, as long as we don’t make people who are moving along in their horsemanship journey feel guilty.

Many extremists do make me feel uncomfortable. No-one knows our horse better than we do, what suits one may not suit another. We must tailor how we train to individual horses.

Positive reinforcement is not the choice of many because they lack understanding at this moment in time. So please be understanding of where other people are on the reinforcement spectrum and don’t be an extremist or a pedant.

If we only use one aspect of operant conditioning we may do so due to not understanding how to use all four quadrants correctly.

We do need to be mindful of the emotions involved with negative versus positive reinforcement and learn to read our horses reactions as we train.

Lets us embrace our fellow horse trainers (we are all training every time we interact with the horse). It is up to us to watch what we are reinforcing – if you get unwanted behaviours look at what reinforced the behaviour.

If you need to go to the extreme end of the spectrum using negative reinforcement then ask yourself why? What has fallen apart? How can I retrain this without being extreme?

If we go too far to the positive end our horses may get over aroused and that is not good either.

Moderation in everything is good.

We must look at the whole horse and our husbandry as well, set them up for success.

Just as Iife is never just black or white using the principles of learning theory is not either.

Often when we are young we look at life as idealists and become fixed on things being good or bad. As we mature we mellow in our understanding and reactions and become more rounded and accepting of other peoples points of view.

Please let us be mature in our acceptance of everyone on their path of learning. Whether we totally agree with them or not others often have valuable experiences to share.

Punishment is another spectrum that I may explore on another day.

Posted on

Conditioning the bridge signal

To commence reward based training with Mojo I have first to classically condition the bridge signal. A bridge signal is to tell the horse he has done the correct behaviour, it is applied as soon as the behaviour we want occurs – the reward can then be given once the horse is calm and relaxed e.g not mugging or looking for the reward.

To condition the signal we start with the unconditioned stimulus(UCS) (food or scratches etc) and get an unconditioned response (UCR) – the horse accepting the reward. We then pair the UCS with a neutral stimulus (NS) (the bridge signal) – this can be a verbal noise or a clicker. The NS then becomes a conditioned stimulus and the UCR becomes a conditioned response as it becomes associated with the reward. So the bridge signal says “yes that is what I want”.

I decided to use a clicker to mark as a bridge signal.

Day 1

My first clicker session with Mojo didn’t go too well. My aim was to classically condition him to the clicker – so pairing the sound with the reward. However poor Mojo jumped and ran off when I clicked, it was a rather loud clicker – so now I have a softer sounding one to use to desensitise him to the sound before going any further.

Day 2

I have just been down to see him and his owner was there so we took him to the indoor school and I tried with the quieter clicker and he was fine. A few times with that and I tried the louder one and again no reaction apart from looking for a treat. I got him to stand nice and calm with his head straight. So I now feel happier about it all.

For more information about the terms used here Karen Pryor as a list which I found useful.

Clicker Training glossary

Jo Hughes is in the process of making a video to show how to get the head straight and to help with impulse control – so I will watch that before doing any more with Mojo. I am away for a week so will recommence once I come back. Mojo is out in the field all day and night, so is enjoying just being a horse with his friends. I will of course go and see him in the field before I go.

Conditioning the bridge signalConditioning the bridge signal
Conditioning the bridge signal
Posted on

The 5 Freedoms

How often do we hear people say they use aversive stimuli (pressure/release) because horses use it between themselves? Well yes they do but it is a threat behaviour – they give fair warning too – ears back, a slight shift in weight before a kick. Horses know these signals and can get out of the way. We don’t have ears that move or tails that swish but we can use our larger brains to find a way to communicate that doesn’t involve threats and escalating pressure. It isn’t easy which is why many don’t even try or go back to using aversive stimuli even when they have learned how the horse perceives these aversives.

If we use negative reinforcement and most of us do in some form then we need to be mindful of the fact and make sure we release effectively.
Personally I don’t like to hit my horse with the clip on the end of the rope – which is advocated by some genres of horsemanship, but I do ride traditionally trained horses and so use conventional aids – negative reinforcement, leg on – leg off etc.
It is the understanding of how classical and operant conditioning works that is useful for us, to enable us to choose wisely and ethically.

What I learned at the recent Thinking Horsemans weekend was that we have a duty of care and only we can decide what we are happy to do with our horses, but it is important to be able to read the horses body language – and their emotional state before any training can begin and assess it as we progress.

It was interesting that one speaker suggested that round penning was punishment based – yes horses send other horses away but that is as a punishment, so why do we then think it good to send horses away on a circle so they can be persuaded to follow us?

I love that most reward based training starts with the horse at liberty – giving the horse a choice. I missed the second day but there was a talk about autonomy – with a scale where reinforcement and punishment was at one end of the scale and autonomy at the other end. If a horse is on line it has no choice but to stay (unless like Benny they learn to use their size to leave). Leaving tells us a lot about our relationship.

The 5 Freedoms of welfare can be used to assess whether our training is ethical.

1. Freedom from hunger and thirst. Don’t train when the animal is hungry or thirsty, they will not concentrate and may mug you if you are using positive reinforcement with food rewards.

2. Freedom from discomfort. Check that the horse has all their needs met – environmental (is it a safe area to train in) as well as physical e.g back, teeth,feet and tack are comfortable.

3. Freedom from pain – don’t add aversive stimuli if this will cause pain to the animal – e.g hitting the horse with a whip or line to insist on a behaviour. Don’t use positive punishment unless in an absolute emergency where the horse is inadequately trained.

4. Freedom to behave normally, don’t insist on a horse looking straight ahead if there are distractions – it is normal behaviour to orient towards any possible scary stimulus in the environment.

5. Freedom from fear and distress, are you sure your training doesn’t cause a fear response? Sending a horse out in a round pen uses the fear response and is punishment. Using aversive stimuli to drive the horse forward initiates the flight response, e.g traditional lunging, and natural horsemanship circling.

I am sure you can all think of other examples.

Posted on

Fear

We all know that horses are innately neophobic, any new stimuli or stimuli out of context may well trigger this fear.

We owe it to our horses therefore to learn as much as possible about equine behaviour, body language and emotions. I am just coming to the end of a course on How Horses Learn, Feel and Communicate – run by Jo Hughes.

Whatever variety of horsemanship you follow – it is an excellent course. I have had horses for over 20 years and ridden for 40 years but still learned a great deal.

The module on emotions (affective neuroscience) is fascinating so even if you can’t commit to a 12 week course then there are videos and articles by Jaak Panksepp to explore.

I was not aware of the 7 emotional systems and that any aversive stimulus triggers the FEAR system. So it does seem to bit illogical to train an already fearful animal with aversive stimuli, but this is what traditional and natural horsemanship and even the equitation science people teach.

So if you do nothing else go and learn how the emotions differ between positive and negative reinforcement and decide for yourself. Study equine body language – especially the subtle signs of stress and fear.

No-one is saying you need to change how you interact with your horses but it gives you more tools to decide what is best for any particular horse at any time.

I did find limitations in certain horsemanship programmes when confronted with a horse who so obviously hated pressure (as Benny did). Maybe that was due to my lack of understanding or skill but learning about how pressure/release works and the difference it makes to the horse when you use positive reinforcement made a huge difference to Benny.

Many traditional and natural horsemen and women do use positive reinforcement and the horse world is slowly catching up with the rest of the animal training world.

Science moves on all the time and the Skinnerian models of learning theory may well be superseded, as we learn more about the emotional minds of animals (ourselves included).

So we do need to keep an open mind, don’t be afraid of what you may learn. I learned a lot from traditional and natural horsemanship but it didn’t provide all the answers.

Posted on

Is the avoidance of an aversive a reward?

Is release of pressure a reward?

Following on from the previous post I have been researching the role of dopamine in emotional reactions. A very limited literature search – the subject is vast and I am not an expert. There are of course other hormones and neurotransmitters involved in all processes.

Dopamine is a neurotransmitter involved in movement via the substantia nigra, but dopamine is also produced by the ventral tegmental area of the brain. It is this area that seems to be stimulated when a reward is received or anticipated. Dopamine in these instances make us feel good. (1)

In horse training, when using the addition of an aversive stimulus to initiate a behaviour, e.g traditional and natural horsemanship, it is the removal of the aversive stimulus that is reinforcing. This is negative reinforcement as described in learning theory.

Research has shown that a transient release of dopamine occurs when an aversive stimuli is removed. (1)

So if this is correct then it may be a reward, but horses don’t seem to actively seek the application of an aversive event to gain that “reward”. They actively learn to avoid the aversive stimulus, by complying at the first indication (a cue) that an aversive stimulus might follow if they don’t comply.

Research goes further to explain this phenomenon –

“a new theoretical explanation of conditioned avoidance: (1) fear is initially conditioned to the warning signal and dopamine computes this fear association as a decrease in release, (2) the warning signal, now capable of producing a negative emotional state, suppresses dopamine release and behavior, (3) over repeated trials the warning signal becomes associated with safety rather than fear; dopaminergic neurons already compute safety as an increase in release and begin to encode the warning signal as the earliest predictor of safety (4) the warning signal now promotes conditioned avoidance via dopaminergic modulation of the brain’s incentive-motivational circuitry.” (2)

It is clear from the above that if we use aversive stimuli we must put a cue in place to predict the aversive – so the horse can avoid the application of the aversive.

When we use appetitive stimuli to reinforce a behaviour it is the anticipation of the appetitive that initiates the release of dopamine. (1) So horses actively seek the reward and can get quite animated in doing this and may offer more than we expect.

So in training should we limit the use of aversive stimuli and increase the use of appetitive stimuli?
Horses may feel good if they avoid an aversive stimulus but how do they feel during the conditioning (training) process.

Hence the title “Is the avoidance of an aversive a reward?”

All these are fascinating questions and I do not know the answers – but it does make me more aware of how and why horses learn and how they may feel about the process.

References
(1) https://www.psychologytoday.com/blog/mouse-man/200904/what-is-dopamine
(2) http://www.ncbi.nlm.nih.gov/pubmed/23759871

Posted on

What’s in a word?

What is in a word?

Take learning theory and peoples interpretations, some say release of pressure is a reward some say it is a relief at best.

What is certain is that in operant conditioning terms it is negative reinforcement, therefore for the release/relief to happen there has to be an aversive stimulus applied first. Once the animal complies with our request the stimulus is removed.

So it may be leg aids – use and remove as soon as the horse complies, it may be ground work – apply an aversive stimulus to form a behaviour e.g teaching a horse to back up. Often people teach back up using a light touch on the chest – getting firmer until the horse backs, the pressure is removed on the slightest of movements.

I don’t wish to argue about various methods of horsemanship – it is a personal choice but the more I learn about the emotions involved the more I wish to use positive reinforcement as much as possible.

We do live in a world where most people use aversive stimuli to form behaviours, so our horses need to understand other peoples cues.

Dictionary defintions:

Reinforcement – The action or process of reinforcing or strengthening; The process of encouraging or establishing a belief or pattern of behaviour

Stimuli – A thing that arouses activity or energy in someone or something; a spur or incentive; A thing or event that evokes a specific functional reaction in an organ or tissue

Reward – A thing given in recognition of service, effort, or achievement; A fair return for good or bad behaviour:

Relief – A feeling of reassurance and relaxation following release from anxiety or distress; The alleviation of pain, discomfort, or distress

Release – Allow or enable to escape from confinement; set free: Allow (something) to move, act, or flow freely; Remove restrictions or obligations from (someone or something) so that they become available for other activity: Allow (something) to return to its resting position by ceasing to put pressure on it:

Aversive – Someone or something that arouses a strong dislike or disinclination.

References
http://www.oxforddictionaries.com

http://www.equitationscience.com/aversive-stimuli – some are challenging the notion that horses are not safe unless we use negative reinforcement.

Connection Training
Academy of Positive Horsemanship

Read and decide, which would you prefer for yourself? Also worth looking at the emotions involved in negative versus positive reinforcement.

http://www.thehorse.com/articles/33927/study-human-interaction-shapes-horses-negative-emotions

Jaak Panksepp describes the 7 basic emotional systems all mammals exhibit.
http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3181986/

Posted on

Flood Desensitising

Learning theory is fascinating as is equine behaviour analysis. Learning to look analytically at horse behaviour and the interaction between horse and humans is a sometimes distressing process, seeing videos of flood desensitisation and realising that at some point in the past you have done a similar type of training is very upsetting.

Flooding is a form of habituation and is classical conditioning, designed to stop a horse reacting to a stimulus. Horses are neophobic so are innately afraid of any new stimulus.

Even done well with animals it may never work, we have better ways to desensitise and habituate horses to novel stimuli.

Flooding entails applying an aversive (remember this is anything the horse perceives as a threat) stimulus at full intensity for as long as it takes the horse to habituate. Take note of the word perceives – it is the horse, not us, who decides what is aversive.

Inadvertant flooding can be as simple as clipping a horse whilst restrained – if he calms down during the process he may habituate or he may be worse the next time you try to clip. Many sedate to clip but the fear is still there, this happened to a horse I know – he had been twitched by a previous owner to be clipped, he is now so sensitive to the sight and sound of the clippers it is going to take a long time to regain his trust.

More extreme examples are putting a saddle on a horse whilst it is still anxious about the process and sending him round an arena – so he can buck if he feels the need to get the aversive stimulus off his back.

When watching horses see if they do that to each other and the answer is no – a horse may move another horse from a resource he wants but the pressure is not unrelenting – horses are gentle creatures and really just want to survive with out us chasing them round.

Posted on

Habituation in Horses

Just starting the third module on Jo Hughes Learning Theory course, I think courses like this are a must for all animal trainers, owners and those who have any interest in ethical treatment of animals. Along with the ethology of the species with which you intereact.
This module looks at habituation, desensitisation and the fine lines between this and, flooding and sensitisation.
I know I have been guilty of flooding a horse – e.g trying to desensitise Benny to wormers did not go well when I first got him, made him a lot worse.

If you put a saddle on a horse for the first time and allow it to buck it out or run until he accepts the saddle – with no means of escape – waiting until he is calm and has given in – that is flooding. That may seem obvious but there are much more subtle examples of flooding and some of it goes badly wrong.

Liz admits she flooded Smoke trying to clip him the year before last – it has taken a while to undo. Failed flooding usually ends up with a horse even more sensitised to the fearful stimulus.

Horses are neophobic so any new stimuls invokes the startle/fear respones. They do need to habituate and be desensitised to our envirnoment for them to be and feel safe. Desensitisation is a form of habituation in a controlled manner – stimulus at very low levels and gradually increased so as not to trigger a flight respone.
Flooding triggers a flight response with stimulus at full strength, with no means of escape, until the horse habituates – which may be never!

Posted on

Thoughts on Desensitisation.

As I am doing a course with Jo Hughes of The Academy of Positive Horsemanship I am reading Dr. Jenifer Ziligs book How to Train Animals 101. The more I learn the more there is to be considered when training horses. Everything we do with them is teaching them something.

Although the course hasn’t got to desensitisation yet I have been seeing more and more videos of people flood desensitising horses.

Some ways to desensitise:
Habituation – exposure over time, also called passive desensitisation. A horse over time habituates to his environment and the weaker the stimulus the more rapid and persistent is the habituation.
This process can be very slow and if we wish horses to habituate to aversive stumuli e.g for possible medical intervention then counter-conditioning with systematic desensitisation is useful.

Counter-conditioning – this is classical conditioning using a reward to change the horses perception of a stimulus. “associating stimuli of opposite value, the combination of which acts to nullify the value towards neutral” (Dr. J. Zeligs – Animal Training 101)

Also 2 ways to approach stimulus exposure:

Systematic desensitisation – using small approximations of the stimulus, can be used with counter conditioning.

Flooding – unrelenting exposure to the stimulus – this is not a recommended technique.

More on flooding by Helen Spence – Flooding and Learned Helplessness

Also from Dr.Sue McDonnell about “sacking out” often used as a euphemism for flooding.
“Unpredictable Fear”

Recently I read this statement:

“Sometimes a horse will put up the greatest resistance just before he comes through. I call it ‘the darkest hour before the dawn’. When the horse has tried all the avenues of his natural instinct of self-preservation and puts up its greatest resistance; is when people will usually give-up or get mad. If they would just be patient, the horse is about to come around.'” – Tom Dorrance.

I would like to deconstruct this statement and ask why the horse feels he needs to take flight or to fight.

The horse has 4 reactions to a fearful situation: first they may freeze – look at the threat, then they may try to flee, if flight isn’t successful – as in a confined space – they may try to fight the threat.
If the threatening predator is still present after all these attempts the horse may give in and freeze or try appeasement behaviours to look less of a threat to the predator.
Then they appear submissive but can actually be in a state of learned helplessness.

In the above scenario the human is the predator – so what happened to the natural horsemanship principle that we should not act like a predator?

The above is a classic way to flood desensitise a horse – don’t give up until the horse gives in – as we all know flooding done incompletely may make the horse put up an even bigger fight next time. The problem is that flooding may never work even if done to the point of the horse giving in.

Why put a sensitive animal like a horse through such a process? If this was done to a deer or other prey animal it would be called cruelty.

So next time you go to a clinic or watch a video of horses being trained try to analyse what is actually happening not what the trainer says is happening.