SuzieQ uses a highly efficient columnar format for storing network state data, optimized for querying and analytics. Here's how this works under the hood:

• Parquet File Format: All network data snapshots are stored as Apache Parquet files. This format provides high-performance read/write operations by storing data in a compact, column-oriented structure.
• Columnar Architecture: Parquet files store data in columns rather than rows. This accelerates analytical queries by allowing SuzieQ to read only the specific attributes it needs, such as interface state or routing protocol status.
• Time Series Data Representation: Each snapshot of device state is written as a new row into its corresponding Parquet file. This produces a trackable timeline of state changes across interfaces, routes, protocols, and more.
• Directory-Based Table Structure: Parquet files are organized into separate logical folders—one for each device table, such as interfaces, BGP neighbors, and ARP entries. This keeps data modular and simplifies access across namespaces or device types.
• Scalable and Efficient for Observability: SuzieQ’s use of Parquet allows it to scale to thousands of devices while still enabling low-latency queries and detailed time-range filtering. The format supports compression and parallelism out of the box.

This is what a standard SuzieQ data store looks like after initial setup. Each directory serves a specific role in organizing persistent data, configuration, and runtime artifacts:

• parquet/ This is the main location where all collected network data is stored. Data from each device or table—interfacing, routing, BGP, etc.—is saved here using Parquet format, organized by timestamp and namespace.
• tmp/ Temporary directory used for logging, intermediate files, cache, and any ephemeral runtime data that is not meant to be permanently saved.
• suzieq-cfg.yml The configuration file that instructs SuzieQ how to operate. It includes polling intervals, data paths, namespaces, and logging levels for each module.
• inventory.yml The list of devices SuzieQ will query. This file defines hostnames, access methods, authentication parameters, and how the devices are grouped by namespace.

suzieq_project/
├── parquet/
├── tmp/
├── suzieq-cfg.yml
└── inventory.yml

This structure helps teams maintain clean separation between configurations, collected data, and transient logs—making large environments easier to manage and automate.

The configuration file used by SuzieQ—typically named suzieq-cfg.yml—defines how data is collected, stored, and grouped. Here's a breakdown of the most common settings to get started:

• data-directory: The local path where SuzieQ will write structured Parquet data. This should match the folder used in your directory structure for long-term observability and historical analysis.
• temp-directory: A path used for temporary files, runtime logs, and metadata during polling operations. This folder is cleaned up automatically during normal operation.
• inventory-file: The file path listing the network devices SuzieQ should collect data from. It includes device IPs, credentials, and groupings based on namespaces or roles.
• logging-level: Controls the verbosity of operational logs. Options include DEBUG, INFO, WARNING, and ERROR depending on how much runtime visibility is required during polling.

data-directory: ./parquet
temp-directory: ./tmp
inventory-file: ./inventory.yml
logging-level: WARNING
poller: 
  period: 300

This YAML layout provides a clean way to instruct SuzieQ how to interact with your infrastructure. Modifying the frequency or paths allows you to tailor the observability stack to your environment's size and architecture.

SuzieQ gathers network state through a scheduled polling engine and saves it into the data store for search and analysis. Here's how the polling and ingestion process works step by step:

• Poller Component: This module connects to each device listed in the inventory and runs commands or API calls to collect operational state. Supported methods include SSH, REST, and gNMI, depending on the device type.
• Data Normalization: After raw data is collected, SuzieQ transforms it into a vendor-neutral format. This ensures all outputs—such as ARP tables, routing entries, interface lists—follow a consistent schema.
• Polling Interval: The poller runs at a specified time interval, defined in seconds by the period setting in the config file. A typical setting is every 300 seconds for general monitoring purposes.
• Time-Stamped Snapshots: Each polling cycle results in a snapshot of device state saved into Parquet files. These snapshots are time-indexed for historical review and convenient querying.
• Multi-Vendor Coverage: SuzieQ can poll a wide mix of platforms including Cisco, Juniper, Arista, Cumulus, Palo Alto, SONiC, and some Linux-based routers. All data is unified in the same structure regardless of vendor.

With this approach, SuzieQ builds a rich data lake of network state over time, enabling performance tracking, outage forensics, and compliance audits without manual logging.

Once network data is collected and stored, SuzieQ gives you multiple ways to query and analyze it. These access methods support real-time troubleshooting, historical analysis, and automation pipelines:

• Command Line Interface (CLI): A Python-based terminal tool that allows quick, filterable queries across the collected data. CLI supports tab completion and flexible syntax, making it ideal for engineers and SREs.
• REST API: An HTTP-based API server can be enabled to allow programmatic access to data. This is useful for integrating SuzieQ with dashboards, ticketing systems, or custom reporting workflows.
• Web GUI: A graphical frontend is available for users who prefer visual filtering and timeline-based inspection. It connects directly to the underlying Parquet data set for instant lookups.
• Output Formats: SuzieQ supports a variety of output modes including plain text, JSON, CSV, Markdown, and even pandas DataFrames. This gives teams flexibility when saving results, sharing reports, or building automated tasks.
• Namespace Filtering: Data can be segmented by namespaces to isolate results by region, site, or tenant. Most access methods allow passing a namespace argument to reduce noise and improve search precision.

Whether you're building a dashboard, writing a compliance script, or manually exploring device history, SuzieQ provides a consistent and flexible interface into your network’s data.

SuzieQ’s data store is built for long-term stability, scalability, and ease of ongoing management. Below are typical strategies and considerations to keep your deployment running smoothly over time:

• Scalability: SuzieQ’s architecture allows you to scale from a small lab to networks containing thousands of devices. By using efficient Parquet storage and modular file structures, the data store handles expansion without degrading performance.
• Performance Optimization: The file-based, columnar approach means that you can run searches and queries quickly, even as your environment grows. Regularly monitor resource usage—such as disk I/O and available space—to ensure continued fast operation.
• Backup and Retention: Periodic backups of the entire data directory are recommended. Set a retention schedule based on your organization’s compliance or analytic needs. Retaining historical data enables audits and forensics but should be weighed against storage costs.
• Reliability: SuzieQ operates without a centralized database daemon, reducing potential points of failure. If data is ever corrupted or lost, restoration is as simple as copying Parquet files back into place.
• Usability and Maintenance: All configuration for data paths and polling can be updated by modifying a YAML file. Log files provide insight into polling errors or runtime issues—review these regularly to address problems proactively.
• Security and Access: Ensure that file permissions on your data and configuration directories are set appropriately to restrict access only to authorized users. For larger environments, use backups with encryption to protect sensitive network data.

By following these practices, you position your SuzieQ deployment for seamless upgrades, reliable long-term operation, and adaptable storage as your network workload evolves.

Keeping SuzieQ’s data store reliable and efficient involves adopting proven troubleshooting methods and routines. Here’s a step-by-step guide to preventing issues and recovering quickly if they appear:

• Configuration Verification: Always confirm that directory paths in your config file point to valid and writable locations. A misconfigured data-directory or inventory path can prevent data collection entirely.
• Disk Space Monitoring: Regularly check storage usage in your parquet/ folder. If disk space runs low, data snapshots may fail to write and older snapshots could be at risk.
• Log Analysis: Review log files located in the tmp/ or specified log directory to catch errors and warnings. Event logs reveal issues like connectivity problems, failed polling cycles, or unsupported device responses.
• Inventory Consistency: Ensure the device inventory file is kept up to date. Remove decommissioned devices and verify all new devices are correctly grouped (for namespaces) to avoid missed data and incomplete coverage.
• Polling Consistency: Confirm that all scheduled polling cycles are running as intended by periodically comparing actual snapshot timestamps with the poller’s interval. Gaps can indicate a misconfigured period, network issues, or authentication errors.
• Schema Updates: If upgrading SuzieQ or integrating custom sources, validate that schema versions are aligned across your environment. Changes in data fields or table formats can cause compatibility issues.
• Regular Backups: Establish a workflow to backup Parquet files in the data directory before making major changes or software upgrades, ensuring simple rollbacks if a problem arises.
• Automated Alerts: Set up basic monitoring or alerting for failed polling, excessive log errors, or storage thresholds to address issues before they affect intended analysis.

By following these practices, teams can minimize downtime, simplify root cause analysis, and maximize the value of SuzieQ’s observability features.