F5 (FIPS) Federal Information Processing Standards

• FIPS HSM (Hardware Security Module): A physical module installed in the F5 BIG-IP device, providing secure key storage and cryptographic operations.
• FIPS Partition: Logical separation within the HSM for key isolation and access control.
• FIPS User Roles: Includes Security Officer (SO) for module administration and Crypto Officer (CO) for key management.
• Key Management Tools: F5 tmsh/GUI utilities for generating, importing, and managing FIPS-protected keys.
• SSL/TLS Profiles: Configured to use FIPS keys for secure communication.

• F5 BIG-IP device with FIPS HSM installed and licensed
• Physical access for initial HSM setup (smart cards, PINs, etc.)
• Administrator access to BIG-IP TMOS (root for CLI and/or admin for GUI)
• Backup of existing keys/certificates (if migrating)
• Knowledge of compliance requirements (FIPS 140-2/3 level needed)
• All firmware/software updates applied to F5 system

1. Initialize the FIPS HSM:
   • CLI: Run fipsutil init
     • Set the Security Officer (SO) password (required for future administrative actions).
     • Define the security domain label. This label must match exactly on all devices in an HA pair or sync group.
     • Optionally configure the Crypto Officer (CO) role with a unique PIN.
   • GUI: Navigate to System > Device > FIPS to verify initialization status post-reboot.
2. Generate or Import Keys:
   • CLI:
     • Generate keys: tmsh create sys crypto key <key-name> security-type fips
     • Import keys: tmsh install sys crypto key <key-name> from-url <url> security-type fips
   • GUI:
     • Navigate to System > Certificate Management > Key Management
     • Click Create or Import and select FIPS as the security type.
3. Configure SSL Profiles:
   • CLI: tmsh create ltm profile client-ssl <profile-name> cert-key-chain { <chain-name> { cert <cert.crt> key <key.key> } }
   • GUI:
     • Navigate to Local Traffic > Profiles > SSL > Client (or Server)
     • Create/Edit a profile and select the FIPS-protected key under Certificates/Keys.
4. Apply Profiles to Virtual Servers:
   • CLI: tmsh modify ltm virtual <vs-name> profiles add { <ssl-profile-name> }
   • GUI:
     • Navigate to Local Traffic > Virtual Servers
     • Edit the target Virtual Server > SSL Profile (Client/Server) section.
     • Select your FIPS-enabled SSL profile from the dropdown.
5. Test Connectivity:
   • CLI: openssl s_client -connect <VIP>:443 (verify cipher suite and certificate chain)
   • GUI:
     • Navigate to Local Traffic > Virtual Servers
     • Check the Availability column for active status.
     • Use the browser's lock icon to inspect certificate details.
6. High Availability (HA) Setup:
   • Risk: Mismatched security domain labels break key synchronization.
   • Steps:
     1. CLI: On Device 1, run fipsutil init with security domain label "FIPS_HA_GROUP".
        GUI: Verify under System > Device > FIPS.
     2. CLI: On Device 2, run fipsutil init with identical label and credentials.
     3. CLI: Verify sync with tmsh show sys crypto fips (compare HSM IDs and labels).
        GUI: Navigate to Device Management > Device Groups to confirm sync status.

• Check Key Storage:
  • CLI: Run tmsh list sys crypto key. Keys stored in the HSM will show security-type fips in the output.
    Alternative CLI: tmsh show sys crypto fips key (shows FIPS keys and attributes).
  • GUI: Navigate to System > Certificate Management > Key Management. Look for keys marked as "FIPS" or "Hardware" in the storage column or details.
• SSL Handshake Verification:
  • CLI: openssl s_client -connect <VIP>:443. Inspect the output to verify the certificate chain and cipher suite; ensure only FIPS-approved ciphers are used.
  • GUI: Go to Local Traffic > Virtual Servers, select the relevant Virtual Server, and check under SSL Profiles that FIPS keys are assigned.
    You can also use a browser to connect and inspect the certificate and cipher via the lock icon.
• FIPS Mode Status:
  • CLI: fipsutil status (should show "FIPS mode: enabled").
    Alternative CLI: tmsh show sys crypto fips (provides additional FIPS module status and health).
  • GUI: Navigate to System > Device > FIPS (if available) to review FIPS status and health indicators.
• Compliance Reports:
  • Export Configuration (UCS):
    • CLI: tmsh save /sys ucs /var/tmp/<filename>.ucs (saves a full backup, including FIPS configuration).
    • GUI: System > Archives, then click Create to generate a UCS archive.
  • Export Logs:
    • GUI: System > Logs > Local Traffic or System > Logs > Audit. Use the Export button to download logs.
    • Remote Syslog (optional): Configure under System > Logs > Configuration > Remote Logging to send audit logs to an external syslog server.
  • Compliance Reports (ASM/PCI):
    • GUI: Security > Reporting > Compliance. Click Printable Version or Export to download as PDF.

Use this section to diagnose and resolve common F5 FIPS issues.

• Common Issues:
  • Key not found in HSM (Hardware Security Module)
  • SSL handshake failures
  • Module not in FIPS mode
  • Role/PIN authentication errors
  • FIPS self-test or integrity check failures
• Diagnostic Commands & GUI Paths:

  # Check FIPS module status
  fipsutil status
  # GUI: System > Device > FIPS
  
  # List FIPS keys
  tmsh list sys crypto key
  # GUI: System > Certificate Management > Key Management
  
  # Test SSL handshake
  openssl s_client -connect <VIP>:443
  # GUI: Local Traffic > Virtual Servers > [Your VIP] > SSL Profiles
  
  # Check logs for errors
  tail -f /var/log/ltm
  tail -f /var/log/secure
  # GUI: System > Logs > Local Traffic / Audit
  
  # Run FIPS integrity check (before rebooting if self-test failed)
  python /usr/libexec/sys-eicheck.py
  
  # Show FIPS module info
  tmsh show sys version
        

• Flowchart: F5 FIPS Troubleshooting

  Start
  │
  ├─ Is FIPS HSM detected?
  │   ├─ No → Check hardware installation, cabling, and licensing
  │   └─ Yes
  │
  ├─ Is FIPS module initialized?
  │   ├─ No → Run fipsutil init and set up roles/PINs
  │   └─ Yes
  │
  ├─ Are keys present in HSM?
  │   ├─ No → Generate/import keys using tmsh or GUI
  │   └─ Yes
  │
  ├─ SSL handshake fails?
  │   ├─ Yes
  │   │   ├─ Check SSL profile configuration (GUI: Local Traffic > Profiles > SSL)
  │   │   ├─ Ensure correct key/cert assignment
  │   │   └─ Validate backend trust and cipher compatibility (must be FIPS-approved ciphers)
  │   └─ No → Success!
  │
  ├─ FIPS self-test or integrity check failure?
  │   ├─ Yes
  │   │   ├─ Run python /usr/libexec/sys-eicheck.py
  │   │   ├─ Review /var/log/secure for failure details
  │   │   ├─ Do NOT reboot until resolved; rebooting locks the volume if unresolved
  │   │   └─ If unable to recover, boot from alternate partition or re-image system
  │   └─ No
  │
  └─ Compliance/Audit issues?
      ├─ Review logs and configuration
      └─ Export reports for auditors (GUI: System > Logs > Audit, System > Archives)
        

• Best Practices & Recovery Tips:
  • Always back up your configuration (UCS) before making FIPS changes.
  • Maintain at least two boot volumes for recovery in case of self-test failures.
  • Do not install the FIPS license on backup volumes unless required for recovery.
  • If the system halts at boot, use the GRUB menu to select a different partition or re-image if necessary.
  • Consult F5 documentation and support for persistent or unclear failures.

Key risks when enabling FIPS on live F5 BIG-IP systems and mitigation strategies.

• Service Disruption and Downtime:
  • Device Reboots & Self-Tests: FIPS activation requires a reboot. If self-tests (system integrity, cryptographic checks) fail during boot, the system halts entirely, causing downtime for all services.
    Example Failure: A failed sys-eicheck.py integrity test during reboot locks the boot volume[5][2].
  • SSL/TLS Cipher Changes: FIPS disables non-compliant ciphers (e.g., RC4, MD5). Clients using deprecated ciphers will fail SSL handshakes[3][5].
• Key & Certificate Compatibility:
  • Key Migration Complexity: Existing non-FIPS keys must be regenerated or imported into the HSM. Errors here break SSL/TLS termination.
    Example: Software-stored keys become invalid in FIPS mode[1][3].
  • HA Synchronization Failures: Mismatched security domain labels or credentials in HA pairs prevent FIPS key sync, breaking failover[5].
• Operational Risks:
  • Configuration Errors: Incorrect security domain labels, SSL profile assignments, or HA setup can disable services.
  • Loss of Access: Lost Security Officer (SO) passwords or security domain labels require full HSM re-initialization, forcing key regeneration[1].
• Application Compatibility:
  • Legacy Client Breakage: Older clients (e.g., TLS 1.0/1.1) or non-FIPS cipher suites (e.g., AES-128-CBC-SHA) lose connectivity[3].
  • Automation Failures: Scripts/APIs managing keys or SSL profiles may fail if not updated for FIPS restrictions[5].
• Recovery Complexity:
  • Boot Failures: Failed self-tests require booting from an alternate partition or re-imaging, prolonging downtime[2][5].
  • Limited Troubleshooting: FIPS mode restricts access to certain diagnostic tools and logs[5].

| Risk Category              | Criticality | Mitigation Strategy                                                                 |
|----------------------------|-------------|-------------------------------------------------------------------------------------|
| Service Disruption         | High        | Schedule during maintenance windows; test FIPS activation in lab first              |
| Key Migration              | High        | Back up all keys/certs pre-migration; use tmsh install sys crypto key ... fips |
| HA Sync Failures           | Critical    | Initialize devices sequentially with identical security domain labels               |
| Legacy Client Breakage     | Medium      | Audit client cipher support pre-implementation; use F5 SSL Orchestrator if needed   |
| Boot/Recovery Failures     | Critical    | Maintain 2+ boot volumes; avoid FIPS licenses on backups[5]                         |
  

• Backup: Create a UCS archive via tmsh save /sys ucs before changes[5].
• Test: Validate FIPS configuration in a non-production environment.
• Document: Securely store SO passwords, security domain labels, and HSM PINs.
• Monitor: Check /var/log/secure and /var/log/ltm post-implementation.