# Differential equations, a tourist’s guide | DE1

The mathematician Stephen Stogartz said, "Since Newton, humans have known that the laws of physics are always expressed in the language of differential equations." Of course, this language goes far beyond the boundaries of physics, And can interpret the world around you and add color to it. In the next few videos, I want to give a series of journeys on this topic. In order to take a look at this part of the field of mathematics, At the same time, I also enjoy digging into the details of special cases. I will assume that you understand the basics of calculus, such as derivatives and integrals. In future videos, we will use some basic linear algebra, but not much. Differential equations are easier to describe changes than absolute quantities. It is easier to describe why the population size is growing or shrinking than it is to describe why it has a specific value at a specific point in time.

And it’s easier to describe the degree of change in someone’s love than where it happened. In physics, more specifically Newtonian mechanics, motion is usually described by force. And force determines acceleration, which is a statement of change. These equations have two forms: Ordinary differential equations (ODEs), which are functions with a single input value, usually expressed in time, And partial differential equations (PDEs) are functions with multiple input values. As for partial differential, we will discuss it in depth in the next video; You will often need to use it to think about the overall numerical continuity over time.

Such as the temperature of each point in a solid, or the velocity of a fluid at each point in space. The ordinary differential equations we are focusing on now only involve a finite set of values ​​that change over time. It does not have to be time itself, the independent variable can also be other things, But the time-varying example is the most typical and common in differential equations. Physics provides us with a good playground for us to explore with simple examples, And when we delve deeper into it, we won’t be confused or misled by the slightest difference. Let's try our best and think about the trajectory of something you throw in the air.

Gravity near the surface causes something to accelerate downward at an increase of 9.8 m/s per second. Now let’s interpret its true meaning: If you observe the force applied by some object and record the speed per second, These vectors will add an additional 9.8 m/s per second. We call this constant "g". Although this is relatively simple, it also gives an example of a differential equation. The y coordinate is given as a function of time. Its derivative gives the vertical component of velocity, Taking the derivative again gives the vertical component of acceleration. For brevity, let us write the first derivative as y-dot and the second derivative as y-double-dot. Our equation states that y-double-dot =-g is a constant. You can solve this problem through integration, which is basically a backward calculation. First of all, you need to find the speed first. What kind of function will change-g after taking the derivative? It should be-gt, right? More precisely, it should be-gt + Vo. Don’t forget, different functions can produce the same derivative, ` So you have additional degrees of freedom depending on the initial conditions. So what function is there to get this after taking the derivative? It's -(½)gt ^ 2 + Vo t, right? Similarly, a constant must be added to prevent the derivative from changing.

The constant is determined by the initial position. Now that you did it, we solved a differential equation about the rate of change. When the effect of the force depends on the position of the object, things become more interesting. For example, when studying the movement of planets, stars, and satellites, gravitational acceleration is no longer considered constant. Given two objects, the force between them acts on the same straight line, And the force between is inversely proportional to the square of the distance between each other. Similarly, the rate of position change is speed, but now the rate of change of speed is acceleration, which is a function of position. So you have the actions between these interacting variables. This narrative also reflects the interaction of two moving objects. In differential equations, this fact is often reflected: The puzzle you face is a puzzle that involves finding a function defined in its own derivative or a higher order derivative. In physics, the most common use is the second-order differential equation, This means that the highest derivative you find is the second derivative. Higher-order differential equations will have third-order derivatives, fourth-order derivatives, etc.; The puzzle will be more complicated.

When you think deeply about this problem, it feels like you are solving an infinite continuous jigsaw puzzle. In a sense, you have to find an infinite number of numbers, and each number corresponds to a point in time. But they are limited by specific methods: these values ​​are intertwined with their own rate of change and rate of change of the rate of change.

In order to feel more clearly what these are being studied, I hope you take a moment to think about a seemingly simple example: a pendulum. What is the change of the angle θ formed with the vertical line over time? This is often used as an example of simple harmonic motion in physics classes, It means that it oscillates like a sine wave. Furthermore, the period is 2π√( L / g), Where L is the length of the pendulum and g is the acceleration of gravity. However, these formulas are actually lies. Or more precisely, it is an approximation at a small angle. If you measure an actual pendulum, You will find that when you pull it farther, the cycle will be longer than the physical formula mentioned in high school. And when you pull it farther, the function graph of θ(t) no longer looks like a sine wave. In order to understand what actually happened, First, let us set up a differential equation.

We will set the distance x along this arc to measure the position of the pendulum. The angle we care about is measured in radians. We can express x as L θ, where L is the length of the pendulum. As usual, gravity decreases with acceleration g, But because the pendulum can limit the movement of this mass, we must observe the acceleration component along the direction of movement. Here is a small geometric exercise for you.

It tells you that there is a small angle equal to our θ. So the gravity component on the opposite side of this angle will be -g sin (θ). When the pendulum is to the right, we set θ to be positive, and to the left to be negative, And this negative sign in acceleration means It always points in the opposite direction of the displacement. So we get a second derivative of acceleration x Is-g sin (θ). Similarly, we can simply examine this formula to feel its physical meaning. When θ = 0, sin(θ) = 0, So there is no acceleration component at this time. When θ = 90°, sin(θ) = 1, So the acceleration component at this time is the same as the free fall.

Okay, let’s take a look, because x = L θ, This means that the second derivative of x is-g/L sin (θ). In order to get closer to the real conditions, let us add the factor of air resistance, which can be set to be proportional to the speed. We write it as-μ θ-dot, and μ is a constant, determined by how fast the pendulum loses energy. This is a particularly intriguing differential equation. It is not easy to solve, but it is not too difficult to give any reasonable explanation. At first you might think that the sine function is related to the sine wave pattern of the pendulum. Ironically, you will eventually find the opposite is true. The equation with a sine is why the real pendulum does not oscillate with the sine wave pattern. If this sounds mysterious, think about this situation. The sine function treats θ as an input value. But in the approximate solution, you can see that the output value has the θ value as the oscillation of the sine wave.

Obviously something suspicious is brewing. This is an example I like, even if it’s relatively simple, But this reveals an important fact about differential equations, and what you are facing is exactly: The world’s invincible explosion problem! In this example, if we remove the damping term, we can reluctantly write down the analytical solution, But it is really ugly enough, these functions will involve integration and weird inverse integration problems you have never heard of. And when you look back, logically speaking, to find the solution is to be able to perform calculations. And establish a dynamic model that can be understood and studied. In this case, the problem of how to calculate and understand these new functions has been beaten. Again, if we add back the damping term, there is no known way to write down the correct analytical solution. Well, for any new problem, you can define a new problem function to solve the problem. If you want, you can even say it yourself. But again, unless it allows you to calculate and understand the answer, it is meaningless. Therefore, when studying differential equations, we often have to take shortcuts and skip the real answer, because we can't really get it.

Then directly follow the equation to build an understandable and computational model. Let's see what the pendulum will look like. What does it look like in your mind? Or do you have any visualization software for you to observe directly, To help you understand the various possible ways of movement of the pendulum? And how do the laws govern the development of the entire process from the beginning? You can try to imagine the graph of θ(t), Learn a little about the interaction between slope and curvature. And by visualizing all possible states on a 2D plane, can we observe any simple and universal laws? . You can fully present the state of a simple pendulum with two values: angle and angular velocity. You can choose to change any value at will without affecting the other one. But acceleration is purely a function of these two values. So every point on the 2D plane completely describes every moment of the pendulum. You can think of these as all possible initial conditions for the pendulum. If you know the initial angle and initial angular velocity, this is enough to predict how the system will develop over time.

If you haven't used them, you can refer to this chart. The inward spiral you are looking at is a fairly typical pendulum trajectory. So take a moment to think carefully about what these represent. Note that at the beginning, as θ decreases, θ-dot on the y-axis becomes negative, This seems to make sense, because as the pendulum moves to the left, it moves faster as it gets closer to the bottom. Remember, even if the speed vector of this pendulum points to the left, the value of the speed represents the vertical component in space. One concept is particularly important. This state space is an abstract concept, which is different from the pendulum that exists in the physical world. Since our modeling loses some energy due to air resistance, this trajectory will spiral inward, It means that the peak value of speed and displacement will gradually decrease with each swing. And what we want to say is that when θ and θ-dot return to zero, the pendulum will return to its origin. With this space, we can think about the role of differential equations in the vector field.

Let me tell you what I mean. The pendulum state is a vector [θ, θ-dot]. Maybe you think of it as an arrow or a point. No matter what it is, it has two coordinates anyway, each of which is a function of time. Take the derivative of this vector and you will get the rate of change, which represents the direction and velocity of the points in the graph. This derivative is a new vector [θ-dot, θ-double-dot], And we regard it as the position attached to the relevant point in this space. Let's take a moment to explain what this means. The first part of this rate of change is θ-dot, which is also a coordinate in space. So the higher the position in the figure, the more the point tends to move to the right. Conversely, the lower the position, the more the point tends to move to the left. The vertical component is θ-double-dot, and we can rewrite the differential equation with θ and θ-dot. In other words, the first derivative of our state vector is a function of some vector and has a complicated relationship with the second coordinate. Do the same thing to all points in space, and you can see the tendency of state changes at different points. In order not to dazzle you, we appropriately reduce these vectors, and then use different colors to indicate its magnitude. Please note that we have transformed a second-order equation into a system consisting of two first-order equations. You can even change the name θ-dot, To emphasize that they are two independent values, and they affect each other's rate of change. This is a common technique in the study of differential equations, Rather than thinking about changes in a single value at a higher level, We prefer to think of it as a vector value of the first derivative. In this form, we get a great visual presentation to help us think about the meaning of the equation. As our system continues to evolve from the initial state, our point will continue to move along the trajectory in this space.

The speed at this point is consistent with the vector in this field. Again, the speed here is not the same concept as the speed of the pendulum in the physical world. This is a more abstract rate of change of θ and θ-dot. You might think this is a little bit magical, Think carefully about what these trajectories mean? And how does this represent how the pendulum moves? For example, in areas where θ-dot is high, The vector guides the point to move to the right and then falls into the vortex. This corresponds to the process of a pendulum with a high initial speed, after it has completely rotated several times, then it swings back and forth.

For something more fun, when I adjust the air resistance μ, such as increasing it, You can quickly see that the trajectory develops faster into the spiral, which means that the pendulum decelerates faster. This feels very clear, it doesn't seem very good. But please think about it, when you only stare at the equation, there is no coordinate chart, and no pendulum. What is the acceleration of the movement of the system to the vortex as the value of μ increases? You can't see anything. So drawing these vector fields with software allows you to observe them more intuitively. This is really amazing, any system of ordinary differential equations can be described by a vector field like this. So this method is also used very frequently. However, they usually have more dimensions. For example, the famous three-body problem, This question is discussing how the three masses move in 3D space, And their gravity affects each other, and then you have their initial position and speed. Each mass has three coordinates to describe its position and three others to describe its momentum, So this system has 18 degrees of freedom, so these possible states are 18-dimensional spaces.

Isn't this weird? The single point of roaming and the unimaginable 18-dimensional space, As the vector moves step by step in accordance with the laws of nature, All are determined by the position and momentum of the 3 masses in the 3D physical space. (By the way, in terms of application, you can use its symmetry to reduce the dimensionality in your settings The points with more degrees of freedom remain the same in the higher-dimensional state space). In mathematics, we call this space "phase space". You will often hear me use this term extensively in all kinds of state change systems, But you should know that in physics, especially Hamiltonian mechanics, the word is often reserved. That is, the space of position and momentum is represented by an axis.

So physicists will agree that the 18-dimensional space used to describe the three-body problem is a phase space. But they may ask us to make some changes to the pendulum setting to comply with the use of this term. For those you saw in the block collision video, We will be called phase space by mathematicians when we work there, But physicists may prefer other terms. Just know its specific meaning. This seems to be a simple idea, depending on how good you are at absorbing modern methods of thinking about mathematics, But it’s worth noting that it will take some time to humanize, Only in this way can people truly embrace the way of thinking about dynamic space, especially when the dimension becomes very huge. In his book Chaos, James Gleick describes phase space as "the most powerful modern scientific invention." One reason it is so powerful is that you can not only ask questions about a single initial state, but you can also know the spectrum of the entire initial state.

All the possible trajectories collected are reminiscent of flowing liquid, so we call it phase flow. Give an example to illustrate why phase flow is an effective way to think about stability, The beginning of our space corresponds to the stationary state of the pendulum, And this point here represents the state of the pendulum upright and balanced. These are the so-called fixed points, And a natural question to ask is whether they are stable? That is to say, will a small push cause the trend to return to a stable point or stay away from it? Physical intuition tells us that the answer is obvious, But how do you see these only through equations? What if they are telling completely different stories? We will discuss how to calculate the answer to this question in the next video.

And related computational intuition, and thinking about the small area of ​​the space around the fixed point, And discuss the contraction or expansion of the phase flow at its point. And speaking of attraction and stability, let's briefly talk about love. The quotation from Stogatz I mentioned earlier is an eye-opening column from the New York Times, which mentions the mathematical model of love, This example is worth talking about. We are not only talking about physics. Imagine that you have been pursuing a favorite object, but this process is always frustrating. At this point, maybe you can turn your head and look at the physics side, Maybe it can keep your mind away from this romantic affair, Think deeply about the pendulum clock equation that broke your heart, and you will suddenly realize the dynamics of the endless pursuit of the abyss.

When you feel that the object seems to have a good opinion of you, your own preference for them will also increase. But when they look cold, your preference for them will drop. In other words, the rate of change in your feelings is directly proportional to their feelings. But your sweetheart is just the opposite: When you are not interested in them, they will be attracted instead, but once you are interested, they will ignore you again. The phase space of this equation looks quite similar to the central part of the pendulum diagram. The two of you will wander back and forth between endless attraction and repulsion.

This is not only applicable to the swing of the pendulum and your feelings, but it is also mathematically verified. In fact, when the relationship between each other progresses too fast, your partner will want to slow down. This is the so-called expectation and fear of harm. This is like the friction of a pendulum, and the two of you are destined to fall into the contradiction between the spirals of your heart. I have heard the bells of the wedding. The point is that these are two very different dynamic laws, one comes from physics involving one variable, And the other one comes from… er… the two variables of chemistry, in fact, their structures are quite similar, It is easy to see from their phase space. The most notable thing is that although the two equations are different, for example, there is no sine in your pursuit equation, But its phase space exposes its potential similarities. In other words, what you are learning now is not just staring at the pendulum all the time, A small part of your research may also have the opportunity to be used elsewhere.

Okay, so the phase diagram is a good way to build an understanding strategy, But what about the actual answer to our equation? Well, one method is to simulate how the world works, but using finite time steps instead of defining infinitesimals and limits in calculus. The basic idea is that if you have a point on this phase diagram, Take the vector on the phase diagram and a short period of time Δt. Specifically, this vector is multiplied by Δt. As before, when drawing this vector field, in order to avoid confusion, each vector will be artificially reduced. After doing this repeatedly, the final position will be an approximation of θ(t), and t is the sum of all your short periods of time.

If you look at the picture as it is now, then consider the meaning of its pendulum movement. You must find this very inaccurate. But that is only because the value of Δt is too large. If we take the value to 0.01, you can get a more accurate approximation. It just needs to be repeated several times. In this case, 1000 steps are required to calculate θ(10). Fortunately, we live in an era with computers, so a task that needs to be repeated 1,000 times can be done with a few simple lines of language. So let us write a small python program to calculate θ(t). All that needs to be done is to design a θ and θ-dot function that can return the second derivative of θ. Start by defining the initial values ​​of the two variables θ and θ-dot, I will equal θ to π/3, which is 60°, and θ-dot to 0.

Next, write a loop between 0 and 10, and set Δt to 0.01. In each step of the loop, increase the value of θ by multiplying θ-dot by Δt, and increase the value of θ-dot by multiplying θ-double-dot by Δt. The θ-double-dot can be obtained by calculating the differential equation. After all these small steps, simply return the θ value. This is called the numerical solution of the differential equation. Numerical methods can be more complex and refined, and can strike a balance between accuracy and efficiency, But this cycle also gives a basic idea. So even if you can’t find an accurate answer, it feels bad, In such a helpless situation, they still have the meaning of studying differential equations. In the next video, we will talk about a few solutions to find an accurate solution.

But the topic I want to focus on is whether these accurate solutions can also help us study more general unsolvable problems. But it got worse. Because there is a limit to obtaining accurate analytical solutions. A great field from the last century-chaos theory, Reveal what restrictions allow us to use these systems to make predictions and know if there are accurate solutions. For some systems, after some inaccurate measurements, We know that even if the initial conditions are slightly different, it will lead to a completely different trajectory. We even have a full understanding of why it happened. For example, the well-known three-body problem has confusing concepts in it. So looking back at the previous quote, the result of using this language to fill the universe seems quite cruel. Either we can’t solve it, or we know the answer but it’s useless for long-term predictions. This is indeed cruel, but it is also very reassuring.

It gives us hope to observe this complex world and present it in mathematics, And be able to reveal unobtrusively between the model and reality..