Topic 3.1: The Chain Rule

Set in concrete

This little motor spins about two hundred times every minute, and it drives a machine built a lot like one by the artist Arthur Ganson. The motor turns a worm gear, and the worm turns a big wheel fifty times more slowly. That wheel turns the next worm, and so on, twelve times down the line.

By the third wheel, it’s turning about as slowly as the hour hand on a clock, and you can’t see it move at all. The very last gear sits inside a solid block of concrete, which holds it completely still. Somehow the motor keeps running without breaking anything. How can a motor run all day long when the gear at the far end is stuck in concrete?

By the end of this lesson, you’ll be able to explain this whole machine with one multiplication. That multiplication is the chain rule. You’ll use the same rule to find the derivative of a function tucked inside another function, and to read one straight off an AP-style table.

Before we work it out, take a guess. If the motor keeps spinning two hundred times a minute, and the concrete weren’t there, how long would that last gear need to make one full turn? Let’s start with a smaller machine, one you can turn with your own hand, so we can see where that number comes from.

Rates multiply

Here’s a crank with a big gear, and it drives a middle gear with a third as many teeth, so the middle gear spins three times for every turn of the crank. A second pair of gears doubles the speed again, so let’s turn the crank exactly once and count. The middle gear makes three turns, and the last one makes six. Three times two is six, so the speeds multiply as they pass down the chain, and that’s really all a gear train does.

Let’s give these turning things names. We’ll call the crank’s turns x, the middle gear’s turns u, and the last gear’s turns y. The middle gear turns three times for each turn of the crank, so d u d x is three. The last gear turns twice for each turn of the middle one, so d y d u is two.

The last gear compared with the crank, d y d x, is those two rates multiplied together, two times three, six. The units show why it works. Turns of u cancel, just like the middle gear drops out, and you’re left with turns of y per turn of x.

The long machine works the same way, except that every stage multiplies the speed by one fiftieth instead of by three. The time for one turn is just one over the speed. The motor makes two hundred turns a minute. How many minutes does the third wheel need to make one full turn?

Every gear after that turns fifty times more slowly again. The fourth needs three weeks for one turn, the sixth about a hundred and fifty years, and by the ninth it’s nineteen million years. By the time you reach the gear in the concrete, one full turn would take about two point three trillion years. That’s around a hundred and seventy times longer than the universe has existed.

Even with time sped up a thousand times, the fourth gear barely creeps and the rest look frozen. In fifty years of running, the last gear would turn by only a tiny fraction of the width of an atom.

A little give in the metal soaks that up, and the concrete never notices. Gears are the easy case, because each pair multiplies by the same number wherever the machine is, so let’s swap one gear for a link that isn’t a gear at all.

A link that isn’t a gear

Now let’s put a pin on the middle gear, and the pin rides in a slot that pushes a pen straight up and down. This time x and u are angles, in radians, not turns. The pin rides around a circle and the slot copies its height, so if the gear has turned through an angle u, the pen’s height is sine of u, in units of that circle.

The paper underneath moves along with the crank, so as I turn the handle, the machine draws its own graph. The crank is at x, the gear is at three x, and the pen is at sine of three x. The curve is y equals sine of three x, a composite function, because the gear feeds three x into the sine.

Let’s put three number lines beside the machine, for the crank, the gear and the pen, and watch a tiny turn travel down the chain. Start at zero and give the crank a tiny nudge. The gear turns three times as far. Here in the middle of its stroke, the pen’s height changes about as much as the gear’s angle, so it changes about three times as much as the crank’s angle.

Let’s try the same nudge with the gear at pi over three. It still turns three times as far as the crank, but now the pen moves only about half as far as the gear. Multiplying those two rates gives about one and a half.

As the nudge shrinks, that half settles on exactly cosine of pi over three. That’s the derivative of sine, cosine of u, read where the gear is. The gear always multiplies by three, while the pen’s multiplier changes as the gear turns. To find that multiplier, we have to use the gear’s angle, not the crank’s.

Here’s the strangest moment on this machine. Turn the crank yourself, and find a spot where the pen stops moving for an instant, even though you’re still turning.

If you’d read the cosine at the crank’s angle instead, at pi over six, you’d get about zero point eight seven and you’d swear the pen was still moving. You have to read the multiplier where the gear actually is.

Now you build one. Swap in a different middle gear, and set the crank so the pen’s height starts out changing four times as fast as the crank’s angle. Let’s take this curve to the board and write down exactly what the machine just showed us.

The chain rule

A small nudge in x makes a nudge in u, and that makes a nudge in y. We multiply delta y over delta u by delta u over delta x to get the ratio for the whole chain, delta y over delta x. As the nudge shrinks to nothing, each of those ratios turns into a derivative. That gives us the chain rule, d y d x equals d y d u times d u d x.

You’ll also see it written with primes. If y is f of g of x, then its derivative is f prime of g of x, times g prime of x. That works as long as g has a derivative at x and f has one at g of x.

In words, you take the derivative of the outside function, evaluated at the inside, so the inside stays exactly as it is, and then you multiply by the derivative of the inside. That g of x tells you where to read the outside derivative, just as the gear’s angle told us which pen multiplier to use.

Let’s do the machine’s own function, y equals sine of three x, where the inside is three x and the outside is sine. Here f is sine and g is three x. The derivative of sine is cosine, so f prime of g of x is cosine of three x, with the three x left alone inside it.

Then we multiply by the inside’s derivative, three, which makes three cosine of three x. Watch the slope as the probe rides along the curve, and you can see three cosine of three x drawing itself underneath. At zero the slope is three, three units of pen height for each radian of crank, and at pi over six it’s zero, right where the pen stopped.

Let’s try a point we haven’t seen yet. For y equals sine of three x, what’s d y d x at x equals pi over twelve?

The chain rule also saves you a lot of algebra. You could find the derivative of three x plus one to the fifth by multiplying out five copies of it, but nobody wants to do that. Call the inside u, three x plus one, so y is u to the fifth. The outside gives five u to the fourth, and the inside gives three.

Multiply them and put the inside back, which gives five times three x plus one to the fourth, times three, or fifteen times three x plus one to the fourth. The mistake almost everyone makes at first is stopping at five times three x plus one to the fourth, and that answer has forgotten the three, the gear in the middle.

Outside in

Before you can use the chain rule, you have to spot the chain, so let’s look at three functions that look almost the same. In sine of x squared, the x squared sits inside the sine, so sine is the outside and x squared is the inside. Sine x, all squared, works the other way around, because the squaring happens last, so the square is the outside and sine x sits inside it.

x squared times sine x isn’t a chain at all, because neither function is inside the other, so that one’s a job for the product rule. The quickest way to tell is to ask what you’d do last if you were working it out with a number, because whatever you do last is the outside. Which one is the derivative of sine of x squared?

Let’s do a couple together. For e to the x squared, the outside is e to the something, and its derivative is itself. We keep e to the x squared and multiply by the inside’s derivative, two x. For the natural log of sine x, where sine x is positive, the log is the outside, so we get one over sine x. Then we multiply by cosine x, the derivative of the inside, and cosine over sine is cotangent of x.

Chains can have more than two links, like the long machine. Sine squared of three x, which means sine of three x, all squared, has three links. The square is the outside, so it gives two sine three x, times the derivative of what’s inside the square. That inside, sine three x, is a chain too, the machine’s own function. Its derivative is cosine of three x, times three from the gear.

Multiply all three, and you get six sine three x cosine three x. The slip to watch for is stopping at two sine three x cosine three x, which forgets the three, the last gear in the chain.

Now you try one with three links. If f of x is cosine squared of two x, what’s f prime of pi over eight?

Thin air

This is the Crew Dragon launch from March 2019, the same rocket you may know from precalc, and this time we’re watching a different number, the air pressure all around it. As the rocket climbs, the air gets thinner, so the pressure keeps falling, and you can even see the exhaust spreading wider as the air stops squeezing it.

The pressure depends on the altitude, and the altitude depends on time, so the pressure is a function inside a function, P of h of t. That means the rate the pressure falls is a chain too, the pressure lost per kilometer of height, times the kilometers the rocket climbs each second.

For the air, we’ll use a simple model, P of h equals one hundred e to the negative h over seven, in kilopascals, with h in kilometers. Its derivative takes the chain rule too, because the inside, negative h over seven, has a derivative of negative one seventh. That gives P prime of h, which tells us how much the air pressure drops for each kilometer the rocket climbs.

Sixty seconds after liftoff, the rocket is about eight kilometers up. It’s climbing at about zero point two eight kilometers a second. At that height, the air pressure drops about four and a half kilopascals for every kilometer.

Multiply the two rates, and the kilometers cancel, which leaves about negative one point two five seven kilopascals per second. On the exam you’d say it in a full sentence. At sixty seconds after liftoff, the air pressure around the rocket is decreasing at about one point two five seven kilopascals per second.

Forty seconds later, at T plus one hundred, the rocket is climbing more than twice as fast. At that moment, how fast do you think the air pressure around it is falling, compared with T plus sixty? The rocket keeps climbing faster, yet each extra kilometer now removes much less pressure. You have to read the outside rate at the altitude the rocket has actually reached.

Now you design one. Suppose a rocket passes through this same altitude, about eight kilometers up. Set its climb rate so that the air pressure around it falls at exactly half a kilopascal per second. Up here, the plume balloons out wide, because there’s almost no air left to hold it in.

On the exam

On the AP exam, the chain rule often shows up with no formulas at all, just a table of values like this one. Say h of x is f of g of x, and they ask for h prime of one. The chain rule says h prime of one is f prime of g of one, times g prime of one.

First find g of one in the table. It’s two, so we read f prime in the row for two. f prime of two is six, and g prime of one is five, so h prime of one is six times five, thirty. The trap is reading f prime at one instead of at g of one, which gives negative two times five, negative ten. That even has the wrong sign, so it would tell you h is going down.

Now we can also say whether h is increasing or decreasing at x equals one. A full-credit answer writes the chain rule with the numbers, then says h is increasing there because h prime of one, thirty, is positive. Now you try one from the same table. What’s h prime of three?

Every link

Let’s put it all together. When one thing depends on another, which depends on a third, the rates multiply link by link. With functions, that’s the derivative of the outside, read at the inside, times the derivative of the inside, and for longer chains you keep peeling from the outside in.

That’s the whole secret of this machine, and it really is one multiplication. Twelve stages that each multiply by a fiftieth make a last gear so slow that over a whole lifetime it would barely move at all, and the concrete never notices.

The same rule powers backpropagation, the method behind how most neural networks learn. It sends the chain rule backward through every link of the network to work out how much each tiny setting affects the final answer. The next time you see one function inside another, think of these gears. Follow the inside’s output into the outside, and multiply the rates you find there.