Dare to Question the Obvious.

A few years ago, I was doing a combination of training course and control troubleshooting services in a pulp and paper mill. My student and I had spent a good deal of the week hooking up to control loops and troubleshooting them. We discovered the usual assortment of faulty valves, transmitter damping and improper tuning. However, the very last task of that week has always stuck in my mind.

It was a Friday morning, and I was wrapping things up with my student. I was destined to take a plane home later that afternoon. After the regular production meeting adjourned, my student and I were asked to “tune” the Machine Chest Consistency controller. Apparently, this loop had not been performing very well since the last shutdown, a few of weeks earlier. To the uninitiated, consistency measures the percentage of paper fiber in stock by mass.

The Machine Chest Consistency controller is critical to the stable operation of a paper machine. It is controlled by injecting dilution water from a header into the suction of the stock pump.

I was somewhat apprehensive about this assignment since I generally do not like to tune loops on a Friday, let alone on the last day of my site visit. But, being the Looney Tuner that I am, my student and I proceeded to connect up to both the consistency and the controller output signals.

We started out by putting the controller in manual and making 1% bumps on the output. Small bumps or step changes determine the precision of the control valve – a valve must be precise within 1%. We performed a succession of 1% bumps and each time the meandering nature of the consistency signal temporarily fooled us into believing we saw a response. We then graduated to 2% bumps; still no response, and then 5% bumps, same result.

At this moment, I suspected the usual culprit, a faulty valve. My student and I then proceeded to perform a “finger test” on the valve. The “finger test” requires two people, one in the control room and the other at the valve, with one or two fingers on the valve stem. The person at the valve then asks the person in the control room to perform a series of small changes, all the while feeling for any movement at the valve. This is a very effective method to determine not only whether the valve actually moves, but how it moves as well. In our situation, the valve response to 1% steps was both precise and crisp. I then checked for another usual suspect, a closed hand valve on the dilution valves…but alas, it was open. The dilution line was also hot.

I was somewhat apprehensive about this assignment since I generally do not like to tune loops on a Friday, let alone on the last day of my site visit.

I returned to the control room and, feeling somewhat frustrated, performed 20% bumps (amidst some gasps in the control room). We finally saw a small “dent” in the consistency. Something was wrong, and time was running out - my plane was leaving in 4 hours. I checked other obvious suspects: the white water dilution pump was running, however the dilution header pressure was very low. For proper consistency control, a dilution header pressure should be 40 to 45 PSI. In our case, it was 10 PSI, hardly enough for effective dilution control. I stepped out of the control room, and this time, I went to the dilution pump. Yes, the pump was running (check the obvious things first), and all the hand valves were open on the header. I returned to the control room scratching my head.

As I explained my findings to my growing entourage, I glanced at my watch: I had half an hour to solve this puzzle or make new plane reservations. At that time, someone told me that they had hastily replaced the motor on the dilution pump during the shutdown two weeks ago. “Hastily…” I thought…and slowly, a question began to germinate inside my head. “No”, I thought, “surely they would have checked that also”…

I then braved the obvious question aloud: “Did you verify the rotation of the pump after the motor was changed?” Some nervous glances were exchanged, and everyone quickly stormed out of the control room towards the pump in question. A quick verification confirmed that the pump had been rotating backwards for two weeks.

I made my plane with fifteen minutes to spare.