Contingency Plan Creation
This page discusses IT Contingency Plan Development. This section outlines the creation of the contingency plan in step 6 of the previous page covering disaster recovery planning phases. It considers the phases of the disaster or incident that the plan is intended to cover.
Ways to aid contingency planning:
- Store backup media off site.
- Store enough software and licensing information to rebuild all primary systems off site.
- Standardize hardware, software and peripherals to allow recovery to be easier and faster.
- Document all servers hardware, software, operating system, applications running, services running, configuration
- Document all information about vendors of hardware and software including your customer number, tech support information, sales information and contacts, contract numbers especially service contract numbers and expiration dates. Bank account numbers and passwords and IDs for accounts.
- Document the network with drawings and documentation including documentation about IP addresses, netmasks of the network, ISP router information, ISP DNS information.
- List and keep offsite IT policies and procedures including security policies for re-establishment of policies and procedures should the primary facility be destroyed.
- Use service failover technologies including:
- Dual power supplies in servers
- Server load balancing or server clustering
- Keep a copy of on site developed code off site. Keep a copy of all documentation, backups, and software off site in case the primary site is destroyed. This policy and the backup policy should complement each other. The backup policy should provide for proper labeling of backup media along with setting a schedule for performing backups and the type of backups. It should provide for a limited time to retain media, testing the restore capabilities, and define what data should be backed up.
An important part of contingency and disaster recovery planning includes where to get new equipment. Options include:
- Keep additional replacement equipment (enough to get minimal functionality started) in another facility
- Make arrangements with suppliers to get emergency drop shipments
- Keep contact information and account information about suppliers in a fireproof safe off site.
1. Supporting information
This section lists supporting information and items to consider when creating a contingency plan.
- The reason for the contingency plan and contingency plan objectives
- Lists organizations that are affected by the plan and lists related plans and policies that either support or are supported by the IT Contingency plan.
- Scope - Outlines situations and conditions that are covered and not covered by the plan. It should consider what lengths of disruptions are covered. It may not address minor disruptions and may or may not cover total destruction of all organizational property. The scope indicates locations and systems covered by the plan. It discusses assumptions such as manpower being available.
- Identify regulatory requirements
- Identify systems covered in the contingency plan. Include network diagrams with locations of network devices, system locations, and system descriptions. Systems to cover include:
- Websites including management of domain names
- LAN/WAN equipment including cabling, routers, switches, firewalls.
- Mainframe systems
- List the teams for specific recovery roles and team members whether by name or job position. Team members and new staff must be informed about what team they are on and what their duties are. There should be a line of succession of duties should some team members not be available when needed.
Disaster recovery team definitions and assigning members to these teams will help define the roles and responsibilities of staff when dealing with the disaster recovery process. Possible Disaster Recovery Teams:
- Management Team
- Telecommunications Team
- Procurement Team to acquire equipment and supplies - The plan should consider the fact that this team will need access to vendor contact information and other information required to make purchases.
- Hardware Salvage Team
- Transportation and Relocation Team
- Network Operations Recovery Team
- Server Recovery Team
- Operating System Administration Team
- Application Recovery Team
- Damage Assessment Team
- Alternate Site Recovery Coordination Team
- Original Site Restoration/Salvage Coordination Team
- Administrative Support Team
- Test Team
- Physical Security Team
Depending on the size of the organization, not all of these teams may be created or some teams may be consolidated together, however the functions of the teams must be provided for.
2. Notification Phase
Notification procedures should be created for situations covered by the IT contingency plan whether there is advanced warning or not. Items to cover may include hurricanes, tornadoes, flood, fire (major and limited to one system), earthquake, bombing, theft, virus infection. Consider how small an incident will be covered by the plan. For example the failure of one complete system may be covered, but the failure of one hard drive on a computer that causes data loss may not be covered. The time to recover may be a consideration whether to cover the event in the plan. Notification procedures should provide for:
- Prompt notification
- Notification of the proper team or teams depending on the incident - The personnel to be notified should be clearly identified with contact information. Backup people to be notified should be listed in a contact list or the procedure.
- Methods of notification including phone, email, pager but must make sure for specific named incidents that notification is timely. Methods of notification may provide for employees listening to a particular radio station to determine whether they should come to work in the event of a snowstorm or other emergency.
- The notifier should know what information to provide such as type of emergency or incident, whether there is loss of life possible or loss of property, and the severity of the incident.
- The damage assessment team should assess damage and determine the cause of the problem, the potential for additional damage or disruption, the area and equipement affected, the status of physical infrustructure such as the building, power, heating, functional status of equipment, type of damage, items to replace, and estimate of time to get services back to normal.
- The damage assessment team should compare the damage assessment ot the plan activation criteria and activate the IT Contengency plan if procedures call for it. Criteria may consider the disruption duration, how critical affected systens are to the organization, the amount of damage, and safety.
3. Recovery Phase
Procedures should give priority to systems that are more essential to the business impact analysis (BIA). The allowable system outage times should be considered. The recovery phase focuses on:
- Contingency measures for temporary IT processing capability.
- Repair or replacement of the original system or facilities.
- Restoration of capabilities at the original or new facility.
Recovery phase procedures should assign tasks to the proper teams for obtaining access to new or damaged facilities, getting supplies and hardware, installing hardware, notifying business partners that use the affected systems, getting and loading data from backup media, restoring operating systems and applications, and testing functionality.
4. Reconstitution Phase
During this phase, operations are transferred back to the original or new replacement facility. Includes:
- Being sure power, environmental controls, water, security, and other equipment is available at the facility.
- Installing computer systems including hardware and software.
- Establish connectivity with network components.
- Test system operations
- Back up the data on the contingency systems and load it onto the restored systems.
- Shut down temporary systems and cease contingency operations.
- Remove sensitive materials from the temporary site.
- IT personnel return to the main facility.
IT contingency planning is designed to sustain and recover IT services after emergencies. NIST lists other plans including:
- Crisis communications plan - To help protect people and property during emergencies or disasters by sending status reports to the public and personnel.
- Occupant emergency plan - Procedures to minimize loss of property and life during emergencies or disasters.
- Business Continuity plan (BCP) - Tries to sustain business function (payroll, accounts payable,purchasing) during and after disruption. Includes IT where essential for the business functions.
- Continuity of Operations Plan (COOP) - A business plan designed to sustain the organization's essential functions for up to 30 days after a disaster or emergency. This plan is not specific to IT but may include IT functions if they are critical for this plan.
- Continuity of support plan/IT contingency plan - Part of incident recovery, addresses IT system disruptions
- Cyber incident response plan - IT responses to emergencies and other incidents - Part of incident recovery.
- Disaster recovery plan - Part of incident recovery providing details used to recover IT capabilities.
- Business Recovery Plan (BRP) - Tries to restore business after a disaster or emergency
Once a disaster happens or a test is run against the plan, the results should be evaluated, problems identified, corrections made and changes to the plan implemented by the disaster planning team. Information from all departments should be considered.