Controllers with PID and
Movitation for Neural Networks

Abstract

Controllers are useful for tasks such as temperature regulation and synchronizing electrical motors. Controllers govern such processes to reach some end goal. PID control is a commonly used implementation of a controller, but is not the only alternative. Another type of controller is based on neural networks. I experimented with the performance of PID control on several varieties of simulated motor systems, and discuss the differences I expect to see in a neural network controller.

Controllers

A basic controller takes from a sensor and supplies output to a device that affects the sensor indirectly. Its goal is to bring the sensor to some desired value. In temperature regulation for example, the controller cannot modify the temperature directly. It does it through a device such as a heater or cooler.

Before we continue, let us familiarize ourselves with some controller terminology. Examples are given for the temperature controller described above.

Controller terms

• Controlled system: The system that the controller controls
  i.e. the temperature regulator
• Process Variable: What is being affected
  i.e. the temperature
• Manipulated variable: What affects the process variable
  i.e. the heater or cooler
• Setpoint: Desired value for the process variable
  i.e. 20° C
• Error: Difference between the setpoint and the current value of process variable
  i.e. –5° C

PID Control

PID is one way to implement a controller. PID stands for “Proportional, Integral, Derivative”. Each of the three terms refers to a specific strategy of controlling the manipulated variable. The image below, taken from the PID page on Wikipedia, shows how all three components are used together.

The contributions of each component are summed together, and the result is added to the manipulated variable. Except before the components are summed, each is weighted by a constant term that determines its significance among the three. The constant terms for proportional, integral, and derivative controls are K_P, K_I, and K_D.

Weighting constants

The K_P, K_I, and K_D constants are preset before the PID controller runs. The method by which they are set is called tuning. Tuning often happens manually, where a human tries different values and chooses the best ones.

The individual components of PID control

Proportional control

Proportional control is commonly used in non-PID controllers. Proportional control adjusts the manipulated variable proportionally to the current error. The larger the error, the more it tries to minimize it. Proportional control is implemented according to the following formula.

The following image, also from the Wikipedia PID page, shows the process variable as a function of time after the setpoint was changed from 0 to 1. The reference signal represents the setpoint, and other curves represent different values of the the constant term K_P.

A human tuning this controller for K_P is likely to select the value K_P = 2, since it makes the process variable settle on its setpoint the quickest.

Integral control

Integral control takes into account the error accumulated up to the current point. The adjustment is described by the formula below. So the longer the process variable is far away from the setpoint, the more integral control will adjust it toward the setpoint. Here are some visual examples of how integral control influences the response of the controlled system.

Derivative control

Derivative control predicts the process variable's value in the future, and adjusts the manipulated variable accordingly. The following formula describes this. The faster the process variable converges on the setpoint, the more derivative control tries to keep the process variable at its current value. It effectively counteracts the adjustments made by the proportional and integral controls. The image below shows various system responses for various values of K_D.

Derivative control is effective at countering oscillation.

P, PI, PD, PID controllers

Not all three PID components are necessary for a stable system. Sometimes a subset of these is sufficient. For some systems P, PI, and PD works and full PID is overkill. It all depends on the properties of the system being controlled.

PID quirks in digital systems

In digital systems input from the process variable is discrete, so we need to approximate derivatives and integrals we compute derivatives and integrals.

Derivative control

The derivative term is approximated as follows.

The higher the sampling timestep Δt (the delay between updates of the process variable), the higher the inaccuracy of the derivative term. This is especially evident when the process variable converges on the setpoint, and the error becomes small. In that case the approximated derivative term is relatively large and causes the process variable to oscillate.

As a result derivative control is either disabled or processed through a low-pass filter on digital systems.

Integral control

An integral is approximated by a Riemman sum. But in integral control, we are integrating from t = 0 to an arbitrarily large t. Similarly the resulting sum can get arbitrarily large. This means integral control will probably dominate the controller and destabilize the system.

To prevent this, I limited the resulting sum to a maximum allowed value.

Test system

I implemented several basic, simulated systems that are meant to test the performance of a PID controller. The systems all resemble a single motor, and the goal is to spin it at some desired RPM.

In the real version of the systems, the manipulated variable is likely a voltage applied to the motor. But to simplify things in my adaptation the manipulated variable is the process variable itself.

Each system attempts to capture some property of a motor. A motor is likely to have friction and momentum, so there are systems that implement them. A list of systems I implemented is listed in the table below.

	Acceleration	Resistance	Cap on max acceleration
Very simple/ ideal motor	discontinuous	none	no
Max Acceleration	discontinuous	none	yes
Simple friction	discontinuous	constant	no
Complex friction	discontinuous	variable	no
Momentum	continuous	variable*	no

* continuous acceleration continuous implicitly gives variable resistance

Implementation and results

I implemented the test systems using Matlab. Code is available in the Appendix.

I manually tuned my PID controller on each system listed in the above table and selected values for K_P, K_I, and K_D that I felt gave the best results.

I show graphs of the system response below, motor RPM vs. time. System response is measured against some reference setpoint, which first starts out at 0 RPM, then jumps to 100 RPM, then to 150 RPM. The goal of the PID controller is to make the system to be as close as possible to the setpoint.

The reference setpoint is shown in blue, and the system response in red.

Simple motor

The simple motor is by far not meant to represent a motor accurately. It is mostly here for example.

K_P = 0.5, K_I = 0, K_D = 0

The simple motor is also an example of where a P-only (proportional) controller is sufficient.

Maximum acceleration

This motor simulation imposes a cap on the maximum torque on the motor. In this context, torque is the change in RPM per unit time. As a result the slope of the red line below never exceeds a certain value.

K_P = 0.25, K_I = 0.1, K_D = 0

Simple friction

The simple friction system works by scaling each change in RPM by a constant k where k < 1. In the actual code k = 0.2.

The graph below shows an example of a poorly tuned PID controller using the same K_P, K_I, and K_D values from the system above. The system below never reaches stability past 0 RPM.

K_P = 0.25, K_I = 0.1, K_D = 0

There are no universal values for K_P, K_I, and K_D that will work well for all systems.

Simple friction, take two

This time the controller is tuned on the simple friction system, and the best constant values are shown below.

K_P = 1, K_I = 0.3, K_D = 0

Compared to the last example, this one illustrates the importance of tuning the controller.

Complex friction

This type of friction is modeled after air friction. In particular, it resembles air friction on an object that increases proportionally with the velocity of the object through the medium.

Although there is no such air friction in a motor system (or at least it is minimal), it is beneficial to see how the controller will perform on it.

The larger the RPM is, the larger is the motor's resistance to changes in RPM.

K_P = 10, K_I = 1, K_D = 0

Since resistance is higher at high RPMs, the system is sensitive on the first step up to 100 RPM, and then relatively insensitive on the second step up to 150 RPM. As a result the controller was difficult to tune on this system, since the controller would tend to overshoot in the first step and undershoot in the second. Even the tuned system has this behavior, as you can see in the graph above.

Momentum

In all previous cases the torque on the motor was discontinuous. If the torque on the motor were plotted against time on the same graph, it would a flat line at 0, then jump to some positive value like a step function.

In this motor system, the torque is continuous. The system does this by limiting the change in torque to some maximum value. This is strikingly similar from the motor with a cap on maximum torque, except that this one limits the change in torque.

K_P = 13, K_I = 0, K_D = 80

Momentum in this system effectively makes the red line want to keep going in the direction it was already going in. This makes for a system that is sluggish and hard to control.

The derivative term is activated here for the first time in all examples. Without it, the system would be unstable. Specifically, when the error is high it would accelerate the RPM to a high value and would not be able to slow down before it passes the setpoint. It would end up overshooting the setpoint by more and more each time, and spin out of control.

The derivative term slows the system down when it predicts that it would get close to the setpoint. This is exactly what we need to make the system stable.

There is one minor problem, though. If you look closely at the red line after it settles, you'll see that it oscillates slightly about the setpoint. That is the effect of derivative control on digital systems which I mentioned earlier.

PID Disadvantages

PID controllers are far from ideal. They need to be tuned to a specific system for good performance. Also the constants K_P, K_I, and K_D are static. They are preset before the PID controller runs, and stay constant during the controller's operation. This is especially disadvantageous for controlling the “complex friction” system, as its sensitivity is a function of the motor's RPM. Hence it is probably better to have the constants actually vary with the RPM.

A neural networks approach

Neural networks can be used instead of, or in conjunction with, PID control. For instance, a neural network can act as the controller itself. Or alternatively, it could be used to tune a PID controller.

Neural networks controller

Hagan and Demuth describe the process involved behind training a neural network to control a system.

They say the process is similar to the one used to approximate an unknown function using a neural network, and suggest to use a multi-layer perceptron for this purpose. By making the error of the network the difference between the actual function value and the network output, they have the network train to approximate the function.

On page 5 their paper, several graphs represent different response curves for the neural network with different neuron weights. Because a neural network is trained to optimize its weights to best satisfy the training data, this could mean the network trains to achieve the best response curve.

The authors suggest several architectures of integrating the neural network into the controller-system unit, with varying advantages and complexities.

Neural networks to tune PID controllers

An alternative to the above is to have a neural network act as the human in tuning a PID controller. It could possibly even perform a better job than the human, simply because it can evaluate far more combinations of K_P, K_I, and K_D and settle the best one.

Future work

One possible direction for future work is to test Hagan and Demuth's various neural network-based controller architectures against the several simulated motor systems here.

Another alternative is to actually implement a network to train a PID controller on a specific system. If I were to design such a network, I would start with a time-delay multi-layer perceptron architecture with two layers and make it have three outputs, K_P, K_I, and K_D. I would feed it 10 RPM values, where each is one timestep behind the previous. One of those 10 values would be the current RPM. After running the network on some step function of the setpoint, I would feed it the sum from the PID integral component as error.

References

[1]   Wikipedia. PID Control. http://en.wikipedia.org/wiki/PID_controller.
[2]   Martin T. Hagan, Howard B. Demuth. (June, 1999). Neural Networks for Control. http://hagan.okstate.edu/HaganDemuthACC99.pdf

Appendix

Code

maxaccelMotor.m

function [stateout] = maxaccelMotor(state, desired)
    current = state(1);
    difference = desired - current;
    % Up to a maximum force can be applied
    force = sign(difference) * min(abs(difference), 5);
    % Momentum coefficient * current
    stateout = [current + force 0];

simplefrictionMotor.m

function [final] = simpleFrictionMotor(current, desired)
    difference = desired - current;
    force = 0.2 * difference; % Damp the force applied
    final = current + force;

complexfrictionMotor.m

function [stateout] = complexFrictionMotor(state, desired)
    current = state(1);
    difference = desired - current;
    coeff = 0.2 / max(current, 2);
    coeff = min(coeff, 0.5); % Set maximum value
    coeff = max(coeff, 0.01); %
          Set minimum value
    force = coeff * difference; % Damp
            the force applied
    stateout = [current + force 0];

momentumMotor.m

function [stateout] = momentumMotor(state, desired)
    current = state(1);
    accel = desired - current;


    % Momentum coefficient * current
    prevforce = state(2);

    forcediff = 1*accel - prevforce;
    % Force can only change up to 5 units
    force = prevforce + sign(forcediff)*0.3;
    stateout = [current + force force];

main.m

% RPM goes from 0 to 200

minRPM = 0;
maxRPM = 200;

halfRPM = minRPM + floor((maxRPM - minRPM) / 2);
quarterRPM = minRPM + floor((halfRPM - minRPM) / 2);

dt = 1;
time = [0:dt:299]; % In milliseconds
%signal = halfRPM*ones(1, length(time));
signal = [zeros(1, 100) halfRPM*ones(1, 100) (halfRPM+quarterRPM)*ones(1,100)];
%signal = [halfRPM*ones(1, 100) (halfRPM+quarterRPM)*ones(1,100)];

% Holds history of all RPMs
RPMs = zeros(1, length(time)); % Start with zero


% ---PID CONTROLLER---

% Proportional, integral, and derivative coefficients
prpK = 13; intK = 0; drvK = 80;

% Sum of integration
intSum = 0;
% Maximum limit for numeric integration
intLimit = floor(halfRPM / 2);

% Previous error stored for derivative use
prevError = 0;

% Hold state for complex motor simulations
state = [0 0];

% PID control
for i=1:length(signal),
    desired = signal(i); % Desired RPM
    current = RPMs(max(1, i-1)); % Current RPM from before
    prev = RPMs(max(1, i-2)); % Immideately preceding RPM
    error = desired - current; % Error between desired and current

    final = current;

    % Add proportional term
    proportional = prpK * error;


    % Add integral term
    intSum = min(intSum + error, intLimit);
    integral = intK * intSum;


    % Add derivative term
    derivative = drvK * (error - prevError / dt);

    prevError = error;

    final = final + proportional + integral + derivative;


    % Simple motor
    %RPMs(i) = final;

    % Maximum acceleration motor
    %state = maxaccelMotor(current, final);

    % Simple friction motor
    %state = simpleFrictionMotor(state, final);

    % Complex friction motor
    %state = simpleFrictionMotor(state, final);

    % Momentum-based motor
    state = momentumMotor(state, final);

    RPMs(i) = state(1);
end


p1 = plot(time, signal);
set(p1, 'Color', 'yellow');
hold on
p2 = plot(RPMs);
set(p2, 'Color', 'cyan');
axis([0 length(time)-1 -10 maxRPM])
set([p1 p2], 'LineWidth', 3)
ylabel('RPM');
xlabel('time (ms)')
hold off