After-action Tracker

In April of 2025, the City of Richmond released the Interim After-Action Assessment report from HNTB, the independent firm investigating the January water crisis that affected residents across the region. The report detailed 15 short-term recommendations (for implementation in three to six months) and 6 long-term recommendations. See below for status updates on each recommendation.

Updated April 16, 2025.

Recommendations

The following actions are recommended for implementation in the next three (3) to six (6) months.

 

 Done!
1. Operate the WTP in Summer Mode all the time or at least during storm events that have risks of power outages.

 Done!
2. Develop a Bus Tie/ATS failure plan, ensure all electrical staff are properly trained on the plan, and clearly display the plan on each bus tie cabinet (SG 6 and SG 7).

 Done!
3. Review staffing plans and consider staffing the WTP with mechanical and electrical staff during storm events that have risks of power outages. If staffing at this level is not feasible, at minimum implement all other recommendations and develop severe storm event response protocol with requirement that maintenance staff on call during storm events can respond in 30 minutes or less.

 Done!
4. Provide a filter effluent valve UPS with a parallel duplicate backup UPS in each plant, all with a minimum runtime of one (1) hour, and ensure that both function as intended to close all filter effluent valves on loss of power. Size the UPS to close all filter valves simultaneously. In lieu of a backup UPS, a small backup generator could be considered to provide backup power in the event of the first UPS failing. The UPS sizing and operation should meet the requirements of Chapter 7 of the Virginia Electrical Code (2020) for Legally Required Systems.

 Done!
5. Install a SCADA UPS with a minimum runtime of at least one (1) hour. The UPS sizing and operation should meet the requirements of Chapter 7 of the Virginia Electrical Code (2020) for Legally Required Systems.

 Done!
6. Change the SCADA programming in Plant 1 to match that of Plant 2 so that the filter effluent valves are set to manual, and the manual set point is set to zero when SCADA is on UPS power.

 In progress
7. Develop an SOP for operators to manually shut down the SCADA system if a failure appears imminent to safeguard against a hard shutdown.

 Done!
8. Verify filter effluent valve fail safe positions are set to close or reprogram to close.

 In progress
9. Add clearwell high level floats that signal control system to override filter effluent valve commands to close the valves. An engineered design is required to ensure that the WTP operates as designed with the addition of these floats.

 In progress
10. Ensure all filter valve actuators are rated as watertight and provide seal-tight fitting and conduit drain fitting prior to the seal-tight fitting within 10-inches of the actuator body. Install a breather fitting near the basement ceiling. Inspect the seal-tight fittings initially after one (1) month and then annually to check if water intrusion has occurred.

 In progress
11. Install visual indicators of filter effluent valve positions with remote open/close switch for each at locations in each control room or basement that are safely accessible by operators in the event of minor flooding.

 Done!
12. Review and re-evaluate organizational structure of operations and maintenance staff at the WTP.

 In progress
13. Develop written SOPs for plant operation, establish a comprehensive training system for staff on SOPs, and implement a regular update schedule for the SOPs.

 In progress
14. Develop standardized agenda for start of shift or shift change meetings with the input of plant staff. Agenda should include safety and emergency operating plan reminders as well as a log or record of all shift meetings.

 Done!
15. Implement seasonal risk assessment. Involve all WTP staff to identify potential risks to assets based on seasonal conditions and remind them of emergency procedures.

 In progress
16. Implement reoccurring formal training for WTP management staff, including emergency response.

 Done!
17. Review safety program for all staff. Refer to 12VAC5-590-560 for minimum requirements.

 In progress
18. Expand DPU Emergency Operations Manual to include scenario-specific and process-specific actions for plant staff to follow during emergency events. Ensure plan is kept current and readily accessible per 12VAC5-590-505.

 Done!
19. Develop an emergency staffing plan for DPU facilities in cases of emergency situations. Ensure operations, maintenance, and leadership staffing have coverage while providing adequate relief.

 Done!
20. Evaluate existing Master Plan, Capital Improvement Plan, and other planning efforts for the WTP. Determine which recommended projects have been completed and which have been deferred. Of the deferred projects, develop a plan to re-prioritize the implementation of these projects based on criticality.

 Done!
21. Perform a holistic review of the planning, engineering, and procurement processes for capital projects within DPU. Determine inefficiencies in these processes and develop an action plan to address or correct the found inefficiencies.

 In progress
22. Develop a crisis communication plan. The plan should include the following at minimum: Establish clear protocols for communicating with staff and internal stakeholders during a crisis, ensuring all staff are informed and aligned with the DPU's response strategy.

 In progress
23. Develop clear protocols for communicating with external stakeholders and to ensure key stakeholders (including regulatory agencies) and wholesale customers (Henrico, Chesterfield, and Hanover) are notified immediately in the event of a WTP outage, using multiple communication channels, as appropriate, to ensure timely dissemination of information. Regular training of relevant staff on this plan and expectations of communication responsibilities during an emergency. Regularly review and update contact information to maintain accuracy and effectiveness. (The City has engaged another consultant to assess its emergency communications during crisis response).

 Not yet started
24. Install dewatering pumps, such as hydraulic-driven pumps, which avoid high suction head issues affecting the priming of the existing dewatering pumps. Dewatering pumps should also have higher flow rates than the existing dewatering pumps, 3,000 to 6,000 gpm is recommended, and permanent piping that routes water away from the plant basements.

 Done!
25. Review staffing plans and consider the addition of a float operator to each shift, so that typical staffing is four (4) operators per shift. If there is an issue getting coverage for an operator that needs a day off, there are always a minimum of three (3) operators.

 In progress
26. Raise as many critical electrical systems above the plant basements as practical.

 In progress
27. Provide an automatic transfer system for the existing backup generator system.

 Not yet started
28. Seal clearwells as much as possible by repairing any cracks or spalling in the concrete and sealing any pipe penetrations and hatches to reduce the flow rate at which water from the clearwells can flood the basement.

 In progress
29. Restructure PM schedule to reduce PM overlap for the same set of assets.

 In progress
30. Develop and implement an asset management plan that includes maintenance and replacement of water system assets, both in the WTP and out in the distribution system.