Table of Contents
- Overview
- Core Components
- Supported Platforms and Data Collection Methods
- Collector Configuration Examples
- Collector Workflow Overview
- Organizational Best Practices
- Use Cases
- Troubleshooting Tips
- Conclusion
SuzieQ Network Observability: Collector
Overview
SuzieQ Network Observability: Collector is a modern, open-source data collection engine purpose-built for multi-vendor network environments. By connecting to switches, routers, firewalls, and cloud instances, the collector gathers a consistent, time-stamped record of operational network data. This foundational dataset powers analytics for troubleshooting, compliance, and efficient operations across complex or hybrid networks.
What Is the SuzieQ Collector?
The SuzieQ Collector is a software component that remotely connects to network devices and retrieves their current state information—such as interface status, routing tables, protocol neighbors, and hardware/software details—without requiring agents or proprietary tools. It works seamlessly across platforms like Cisco, Arista, Juniper, Cumulus, Palo Alto, SONiC, and standard Linux hosts.
Why Is It Important?
- Unified Visibility: Networks often include devices from many vendors. SuzieQ Collector normalizes data, making it searchable and comparable regardless of hardware or software.
- Operational Accuracy: Continuous or on-demand polling provides both real-time and historical insights, enabling root-cause analysis and compliance verification.
- Efficiency and Scalability: The collector automates what would be a slow, error-prone manual process, supporting environments ranging from small labs to massive, distributed networks.
- Agentless and Secure: By using well-established protocols (SSH, REST APIs), it fits into existing security policies and requires no disruptive software installations on network equipment.
How Does It Work?
- Inventory Driven: You define your devices in a YAML, CSV, or dynamically from an external source (like NetBox). This inventory includes IPs, credentials, and optional metadata like namespaces.
- Connection & Polling: The collector initiates connections using SSH or API calls to devices, authenticating with provided credentials to retrieve current network state.
- Data Normalization: Retrieved data is transformed into a consistent internal structure, eliminating vendor differences.
- Storage: Normalized records are written in Parquet files—an efficient big-data format—for fast querying, analysis, and long-term retention.
- Workflow Flexibility: The collector supports continuous monitoring (scheduled intervals) or one-off “snapshot” collections, depending on operational needs.
By transforming raw network state into structured, actionable information, SuzieQ Collector allows engineers and operators to solve problems faster, automate health checks, track configuration drift, and maintain robust documentation for modern infrastructures.
Core Components
These are the essential building blocks that enable the SuzieQ Collector to gather, normalize, and store network operational data across multi-vendor environments:
- Inventory Parser: Reads the inventory file (YAML, CSV, or dynamic sources like NetBox) to produce a structured device list, including hostname, OS type, credentials, and organizational grouping (namespace).
- Connection Manager: Establishes secure connections to each device using SSH or REST APIs without requiring agent installation. Handles authentication, session creation, and communication reliability.
- Poller Engine: Orchestrates the periodic polling of network devices. Executes show commands or API queries to collect operational state data (interfaces, routing, protocols, etc.).
- Normalizer: Converts diverse data from various vendors into a standard, device-agnostic format—ensuring consistency across network platforms. Enables unified search, analysis, and reporting regardless of underlying device vendor.
- Data Writer: Persists all gathered and normalized records into Apache Parquet files. This columnar storage approach offers efficient querying and supports large-scale data analysis.
- Logging & Error Handler: Records poller activity, connection status, data parsing results, and failures for auditing and troubleshooting. Provides verbose logs to assist with operational issues.
Supported Platforms and Data Collection Methods
SuzieQ Collector offers broad multi-vendor device compatibility and flexible ways to acquire operational state data across the network. Here’s how support and collection work, step-by-step:
-
Platform Support:
- Arista EOS (version 20.0 or later)
- Cisco IOS, IOS-XE, IOS-XR
- Cisco NXOS (N7K 8.4.4+, N9K 9.3.1+)
- Cumulus Linux
- Juniper Junos (QFX, EX, MX, SRX, Evolved OS)
- Palo Alto PanOS (version 8.0+)
- SONiC devices
- Linux servers
Support for new devices is frequently added in response to enterprise and community needs.
-
Data Collection Methods:
- Agentless Connection: No software is installed on network devices. All data gathering is performed remotely.
- Protocols Used: Uses SSH or REST API to securely access and retrieve state details.
- Inventory Driven: Reads device details from static (YAML, CSV) or dynamic sources (NetBox). The inventory acts as the single source of truth for device reachability and credentials.
- Operational Data Collected: Includes interfaces, routing, protocols (OSPF, BGP, LLDP), software versions, configuration, and topology information.
-
Flexible Polling Modes:
- Continuous Mode – Scheduled, periodic polling to build a time series of network state.
- Snapshot Mode – One-off data collection from all devices for immediate, point-in-time analysis.
This flexible foundation allows network engineers to observe, audit, and analyze operational state regardless of vendor or device type, with minimal setup.
Collector Configuration Examples
The SuzieQ Collector is designed to work with both simple and dynamically generated inventories. Below are step-by-step examples that demonstrate how to configure the collector using YAML and basic CLI options for typical deployment scenarios.
-
Step 1: Create an Inventory File (YAML Format)
The inventory lists devices, their operating systems, access credentials, and namespace details. Here's a basic static inventory file:
namespace1: devices: - name: leaf1 hostname: 192.168.10.11 os: eos username: admin password: adminPass! - name: spine1 hostname: 192.168.10.21 os: eos username: admin password: adminPass!
Devices are grouped by namespace to support multi-tenant or multi-environment deployments.
-
Step 2: Run the Collector Using the Inventory
Execute the collector and point it to the inventory file:
suzieq-poller -I ./inventory.yaml -p parquet-output-directory
This command triggers the polling process and stores the normalized output under the specified directory using Parquet format.
-
Step 3: Use Snapshot Mode for One-Time Collection
Snapshot mode is used to perform an immediate one-time data pull across all devices:
suzieq-poller -I ./inventory.yaml --once
Useful for testing, diagnostics, or point-in-time audits.
-
Step 4: Adjust Polling Interval (Optional)
For continuous collection, the polling interval can be set in seconds:
suzieq-poller -I ./inventory.yaml --run-mode daemon --poll-interval 300
This example sets polling to occur every 5 minutes.
-
Step 5: Use NetBox as a Dynamic Inventory Source
If managing inventory in NetBox, SuzieQ can pull device data directly using a dynamic plugin:
suzieq-poller -I netbox.yaml
This enables real-time inventory synchronization and eliminates manual device list updates.
These examples provide a flexible starting point for configuring the SuzieQ Collector in single or multi-site environments, whether using static files or dynamic inventory systems.
Collector Workflow Overview
The SuzieQ Collector performs network data acquisition in a structured, repeatable workflow. Below is a step-by-step breakdown of how it functions from start to finish:
-
Step 1: Load the Inventory
The collector begins by parsing the inventory file, which contains hostname, OS type, authentication credentials, and namespace metadata. This defines the scope of devices to examine.
-
Step 2: Establish Device Sessions
Using SSH or REST API, the collector authenticates into each device with the credentials supplied. This step opens a secure session to begin gathering network state.
-
Step 3: Collect Operational State
The poller retrieves interface status, routing tables, protocol neighbors, MAC tables, software versions, and other essential state data. This collection is vendor-specific but driven by a unified internal process.
-
Step 4: Normalize Collected Data
After retrieval, the collector transforms all collected data into a standard model that abstracts vendor differences. This uniform format enables centralized analysis and consistent querying across platforms.
-
Step 5: Persist Data for Analysis
The normalized output is stored in structured Parquet files on disk. These files serve as a historical record and can be queried by the SuzieQ analytics engine or external data platforms.
-
Step 6: Log Status and Errors
Each collection cycle records connection success, failures, device responsiveness, and data parsing status. These logs help in auditing and troubleshooting network issues or unreachable devices.
This workflow allows SuzieQ to provide reliable, vendor-neutral network visibility with minimal manual effort.
Organizational Best Practices
Implementing SuzieQ Collector effectively requires thoughtful organization and maintenance to maximize network observability and ease ongoing operations. The following step-by-step best practices help ensure successful deployment and management:
-
Define Clear Namespaces:
Group devices logically using namespaces based on geography, function, or environment. This segmentation simplifies filtering, reporting, and targeted troubleshooting.
-
Maintain an Up-to-Date Inventory:
Regularly review and refresh the inventory to reflect network changes. Automate this process when possible by integrating with dynamic sources such as IPAM or CMDB systems.
-
Configure Reasonable Polling Intervals:
Balance polling frequency with network stability and storage considerations. High-value or critical devices may require more frequent collection, while less critical devices can have longer intervals.
-
Secure Device Credentials:
Store access credentials securely, using environment variables or vault solutions. Avoid hardcoding sensitive information directly in inventory files.
-
Monitor Collector Logs Regularly:
Use logs to detect and quickly resolve connection issues, parsing errors, or device unreachability. Automated alerting on repeated failures can improve responsiveness.
-
Leverage Historical Data for Trend Analysis:
Use stored Parquet data over time to analyze network behavior, identify patterns, and detect anomalies before they impact operations.
-
Document Collector Configuration and Processes:
Maintain clear documentation of inventory structures, polling schedules, and collector usage instructions. This aids team collaboration and onboarding.
Following these organizational practices ensures the SuzieQ Collector remains efficient, scalable, and valuable for ongoing network observability initiatives.
Use Cases
SuzieQ Collector addresses a range of practical scenarios to enhance modern network operations. Below is a step-by-step overview of how it can be leveraged in real-world environments:
-
Historical State Analysis
Retrieve the exact status of an interface, device, or protocol at any moment in the past. This allows for precise root cause analysis of outages or performance problems.
-
Change Validation and Compliance Auditing
Quickly compare the state of devices or configurations before and after maintenance windows. Validate that changes have been applied correctly and that compliance baselines are maintained.
-
Automated Documentation
Continuously generate up-to-date inventory, topology diagrams, and device configurations, creating trustworthy documentation for audits and knowledge sharing.
-
Proactive Health and Protocol Checks
Monitor protocol status (e.g., OSPF, BGP, LLDP) and device health on an ongoing basis. Surface issues before they cause operational impact, and support automated alerting workflows.
-
Network Security Investigations
Identify devices running vulnerable software, spot unauthorized configuration changes, and trace the appearance of rogue VLANs or interfaces across the infrastructure.
-
Exploratory Analysis and Reporting
Query the collected dataset for answers to ad hoc questions—such as finding all devices with a specific software version, tracking MAC addresses, or analyzing usage trends over time.
With these practical applications, SuzieQ Collector enables network teams to evolve from manual troubleshooting to automated, data-driven operations and strategic visibility.
Troubleshooting Tips
When running the SuzieQ Collector, various conditions can affect connectivity, data retrieval, and processing. Below is a step-by-step guide to resolve common issues and keep your collection workflow running smoothly:
-
Verify Device Reachability
Confirm that the target devices are accessible on the network and responding to SSH or API requests. Use tools like
ping
ortelnet
to validate connectivity. -
Check Inventory File for Errors
Ensure that your YAML or dynamic inventory is correctly formatted. A single syntax error can prevent the collector from parsing any device entries.
-
Confirm Credentials and Permissions
Double check that the usernames and passwords in the inventory grant sufficient access rights for data collection. The user must have privilege to run show commands or retrieve state.
-
Use Verbose Output for Debugging
Add the
--loglevel debug
flag to the collector command to enable detailed logs. This helps identify where in the process the failure is occurring.suzieq-poller -I ./inventory.yaml --loglevel debug
-
Review Collector Logs
Logs provide timestamped insight into connection attempts, errors, and polling progress. Reviewing logs regularly can uncover repeated failures or configuration gaps.
-
Validate Device OS Support
Ensure that the devices you're collecting from are among the supported platforms. Devices outside of support scope may fail to respond or return unusable data.
-
Monitor for Poller Timeouts
If polling is taking too long or failing silently, consider increasing the timeout threshold. This can help with devices that respond slowly or have command queuing delays.
-
Test Snapshot Mode Individually
Use the
--once
flag to isolate individual collection issues without running in daemon mode. This is useful for testing new inventory entries or configurations.
Following these tips will help you identify and fix common issues encountered during collector setup or execution, keeping your network data pipeline reliable and responsive.
Conclusion
Throughout this blog on the SuzieQ Network Observability: Collector, we explored how SuzieQ provides a powerful and vendor-neutral solution for gathering detailed operational state across your network. From inventory-driven configuration to real-time and historical data analysis, the collector acts as the foundation for scalable, repeatable, and insightful network observability.
We started by breaking down the core components that drive SuzieQ's collector engine—such as the inventory parser, poller engine, and data normalizer. We then explored the wide range of supported platforms and how the collector connects dynamically using SSH or APIs without requiring agents on devices. You saw step-by-step how to configure the collector, whether working with static YAML files or pulling live data from a tool like NetBox.
We followed that with a look at the polling workflow—from loading inventory to persisting normalized data. We also shared organizational practices to make SuzieQ's deployment smoother and maintainable, such as setting up namespaces and securing credentials properly.
You were also introduced to several practical use cases, including analyzing device history, automating network documentation, validating configuration changes, and supporting security investigations. And to round it out, we offered troubleshooting tips for addressing common errors and improving collector reliability.
If you're looking to modernize your network observability while simplifying your data pipeline, SuzieQ Collector is a strong, flexible choice that fits comfortably into modern environments and operational workflows.
Thanks for reading—happy collecting, and may your networks stay visible and resilient!