Alright, I've been away from the nuclear biz for a few years now, but coverage of an event at TVA's Browns Ferry (BFNP) site sounds misleading to me.
"Data storm" blamed for shutdown.First, some background. Look here for information on
boiling water reactors (BWR) in general. The players in this event were the reactor recirculation pumps (which tripped) and the condensate demineralizers ("polishers"). The net effect was as if kidney failure caused a seizure, which the doctors in the control room dealt with via manual scram (intentional shutdown of the reactor) from the control room.
Now, the doctors. Historically anyone who operates a reactor or directly supervises them needs a pretty exclusive credential as a "reactor operator" (RO) or "senior reactor operator" (SRO) per requirements from the Nuclear Regulatory Commission (NRC, occasionally US-NRC). They receive this credential after intensive training on their plants both with books and with team drills using a control room that is a replica of their facility right down to the colors and shapes of the switches, buttons and displays. They can lose it, and of course the jobs that require it, by failing to act as required by training and procedure.
One tenet of reactor management is that shutdowns like this are dissected in detail, particularly automatic ones. For the latter you can bet that the operators on shift at the time will be asked why they didn't avoid the situation if possible, or else shut down the reactor themselves before the automatic logic took over. An unsatisfactory answer will be received as if a doctor at a malpractice trial had said "no, I didn't use anaesthetic because I figured the patient would pass out anyway from the pain".
Now let's talk about the reactor recirculation (RR) pumps. They're not found in pressurized water reactors (PWR), but they're familiar to fossil plant operators. The idea is to increase the coolant flow through the reactor, because that permits it to generate more power (think turbochargers).
There's always some flow under power operation, because steam is flowing out and feedwater is flowing in to maintain a constant level of coolant in the reactor well above the fuel. But if water is recirculated, you can have more flow through the core than what feedwater alone would provide, and this increases power output. And that power varies in proportion to the RR pump speed, so they have variable frequency drives to control their motors (at BFNP, anyway - more modern BWRs control recirc flow in other ways).
When you shut down a plant, the idea is to minimize its power production. Accordingly the RR pumps are shut down early in the process in the course of a planned shutdown of a BWR, and having the RR pumps unavailable for whatever reason (short of large leaks) does not affect reactor safety.
Now, the condensate polishers. They're typically located far from their reactor in the basement of the building which houses the turbine. They clean the reactor coolant, which in a BWR is demineralized water. By the time the water reaches a BWR's condensate polishers it has been through the condensate and feedwater systems, the reactor, the turbine, and the condenser, and along the way it has picked up contaminants from corrosion products and others. This means that you don't hang around the condensate polishers - they're "hot" radiologically.
They have no safety function. David Copperfield could make them all disappear and the plant could be shut down safely. You won't operate the plant that way for long, but nuclear safety regulations are not concerned with that - that's the owner's problem.
Also like kidneys, you have more CPs than you need. In normal operation you'll have as many running as it takes to support full power operation, and the others will either be out of service or being regenerated.
If CPs are the kidneys, the radwaste system is the bladder. CPs that I am familiar with contain tiny beads of special polymeric resins which preferentially absorb various ions, and they don't last forever - they're either regenerated or replaced at intervals. The analogy is left as an exercise.
Either process involves valve manipulations and other activities near the CPs. Automation of this saves radiation exposure to operators and forces use of the proper procedures. And nowadays the cool kids are using programmable logic controllers (PLC) to control such logic - what valve opens when, and for how long, in response to what measurement, etc.
PLCs are like highly proprietary computers with a very narrow focus and specialized I/O devices. They're much more reliable than PCs (thank God for that), and, being proprietary, they need not make compromises in the name of compatibility with other manufacturers. They can be programmed and can communicate like computers in various network topographies. They can drive or be driven by analog signals, switch and relay contacts or by signals sent over their network. They're compact and can be replaced by yanking cards out of a rack. And as a consequence they offer entirely new ways to fail.
I don't know the details of the configuration at BFNP, but it sounds fair to assume that the PLCs that control the CPs somehow share an information network with the reactor recirculation pumps. Otherwise how could the problem with the CPs impact the RR pumps?
Well, if so this interconnection is not quite as crazy as it may sound. For one, all of the systems in the power generation system must work in concert. Once a BWR is generating power, you manipulate control rods to a certain state as dictated by the nuclear engineering staff to optimize fuel consumption. Then you pretty much leave them there, and control power via the speed of the reactor recirc pumps. More power means more flow through the CPs, and they only tolerate so much flow, so whether automated or not the CP status is considered in the operation of the RR pumps.
I understand that BFNP was modified recently, which is probably when the PLCs were put in. These devices were in their infancy at the time these plants were originally designed and built, and BWR units built later still had traditional 4-20mA analog controllers and relay ladder logic for most systems. When the new design was proposed, I'm guessing that some old operators and engineers were saying "WTF? Nothing happens so fast with CPs that we can't deal with it manually and we don't need any strange new control systems with their new failure modes. This isn't some stealth bomber that simply can't be flown without computerized controls and which spends most of its time shut down anyway - what is this crap doing here?" At which time they found that they were troglodytes and were overruled.
But the bottom line is that nothing happened that impacted the public beyond reading about it in the paper. It cost TVA a bundle because shutdowns always do, planned or otherwise, so you can bet that they're looking really hard at preventing recurrences if only for the most venal capitalistic reasons. There'd better be some engineers and instrument techs burning serious midnight oil at the company that provided the PLCs and on the staff at BFNP. And you don't have to be a baseball fan to know that the most effective way to keep your errors down is to have fewer chances - nobody wants this to recur.
So why did we hear of a "data storm"? I don't like that formulation because it invites speculation that somehow the Internet was involved, and in turn, that someone from outside the plant could manipulate the plant to its detriment.
Yeah, fat chance. If I thought that were happening I'd throw a fit myself. No RO or SRO I ever met would stand for that - they're responsible for the plant, they have to know what's going on, and they'd raise Cain if anything that mattered could bypass them. And I can't imagine the NRC liking it.
The TVA spokesperson confirms the separation here:
"The integrated control system (ICS) network is not connected to the network outside the plant, but it is connected to a very large number of controllers and devices in the plant," Johnson said. "You can end up with a lot of information, and it appears to be more than it could handle."
That's not to say that no computers are ever attached to such control systems. Some are used to monitor and log various plant statuses to help in diagnosing problems and analyzing trips, and they are invaluable for this. But in my experience they are connected through isolators such that they could fail completely, short out, etc, without harming the original circuits. They don't control anything related to power production, and no safety related plant functionality is lost if such systems are unavailable.
Original link from
Instapundit.
UPDATE: What's this, traffic? It must be a link back from Instapundit. Thanks, Glenn! Incidentally, I made some minor changes above to make it resemble English a bit more closely. I don't have to be literate, I'm an engineer!
