Command: operator debug
The operator debug command builds an archive containing Nomad cluster
configuration and state information, Nomad server and client node
logs, and pprof profiles from the selected servers and client nodes.
If no selection option is specified, the debug archive contains only cluster meta information.
Usage
This command accepts comma separated server-id and node-id IDs for
monitoring and pprof profiling. If IDs are provided, the command will
monitor logs for the duration, saving a snapshot of Nomad state
every interval. Captured logs and configurations are subjected to
redaction, but may still contain sensitive information and the archive
contents should be reviewed before sharing.
If an output path is provided, debug will create a timestamped
directory in that path instead of an archive. By default, the command
creates a compressed tar archive in the current directory.
Consul and Vault status and version information are included if configured.
If ACLs are enabled, this command will require a token with the 'node:read' capability to run. In order to collect information, the token will also require the 'agent:read' and 'operator:read' capabilities, as well as the 'list-jobs' capability for all namespaces. To collect pprof profiles the token will also require 'agent:write', or enable_debug configuration set to true.
General Options
-address=<addr>: The address of the Nomad server. Overrides theNOMAD_ADDRenvironment variable if set. Defaults tohttp://127.0.0.1:4646.-region=<region>: The region of the Nomad server to forward commands to. Overrides theNOMAD_REGIONenvironment variable if set. Defaults to the Agent's local region.-no-color: Disables colored command output. Alternatively,NOMAD_CLI_NO_COLORmay be set.-ca-cert=<path>: Path to a PEM encoded CA cert file to use to verify the Nomad server SSL certificate. Overrides theNOMAD_CACERTenvironment variable if set.-ca-path=<path>: Path to a directory of PEM encoded CA cert files to verify the Nomad server SSL certificate. If both-ca-certand-ca-pathare specified,-ca-certis used. Overrides theNOMAD_CAPATHenvironment variable if set.-client-cert=<path>: Path to a PEM encoded client certificate for TLS authentication to the Nomad server. Must also specify-client-key. Overrides theNOMAD_CLIENT_CERTenvironment variable if set.-client-key=<path>: Path to an unencrypted PEM encoded private key matching the client certificate from-client-cert. Overrides theNOMAD_CLIENT_KEYenvironment variable if set.-tls-server-name=<value>: The server name to use as the SNI host when connecting via TLS. Overrides theNOMAD_TLS_SERVER_NAMEenvironment variable if set.-tls-skip-verify: Do not verify TLS certificate. This is highly not recommended. Verification will also be skipped ifNOMAD_SKIP_VERIFYis set.-token: The SecretID of an ACL token to use to authenticate API requests with. Overrides theNOMAD_TOKENenvironment variable if set.
Debug Options
-duration=2m: Set the duration of the log monitor command. Defaults to"2m". Logs will be captured from specified servers and nodes atlog-level.-interval=2m: The interval between snapshots of the Nomad state. If unspecified, only one snapshot is captured.-log-level=DEBUG: The log level to monitor. Defaults toDEBUG.-max-nodes=<count>: Cap the maximum number of client nodes included in the capture. Defaults to 10, set to 0 for unlimited.-node-class=<node-class>: Filter client nodes based on node class.-node-id=<node1>,<node2>: Comma separated list of Nomad client node ids, to monitor for logs and include pprof profiles. Accepts id prefixes, and "all" to select all nodes (up to count = max-nodes).pprof-duration=<duration>: Duration for pprof collection. Defaults to 1s.-server-id=s1,s2: Comma separated list of Nomad server names, "leader", or "all" to monitor for logs and include pprof profiles.stale=<true|false>: If "false", the default, get membership data from the cluster leader. If the cluster is in an outage unable to establish leadership, it may be necessary to get the configuration from a non-leader server.-output=path: Path to the parent directory of the output directory. Defaults to the current directory. If specified, no archive is built.-consul-http-addr=<addr>: The address and port of the Consul HTTP agent. Overrides theCONSUL_HTTP_ADDRenvironment variable.-consul-token=<token>: Token used to query Consul. Overrides theCONSUL_HTTP_TOKENenvironment variable and the Consul token file.-consul-token-file=<path>: Path to the Consul token file. Overrides theCONSUL_HTTP_TOKEN_FILEenvironment variable.-consul-client-cert=<path>: Path to the Consul client cert file. Overrides theCONSUL_CLIENT_CERTenvironment variable.-consul-client-key=<path>: Path to the Consul client key file. Overrides theCONSUL_CLIENT_KEYenvironment variable.-consul-ca-cert=<path>: Path to a CA file to use with Consul. Overrides theCONSUL_CACERTenvironment variable and the Consul CA path.-consul-ca-path=<path>: Path to a directory of PEM encoded CA cert files to verify the Consul certificate. Overrides theCONSUL_CAPATHenvironment variable.-vault-address=<addr>: The address and port of the Vault HTTP agent. Overrides theVAULT_ADDRenvironment variable.-vault-token=<token>: Token used to query Vault. Overrides theVAULT_TOKENenvironment variable.-vault-client-cert=<path>: Path to the Vault client cert file. Overrides theVAULT_CLIENT_CERTenvironment variable.-vault-client-key=<path>: Path to the Vault client key file. Overrides theVAULT_CLIENT_KEYenvironment variable.-vault-ca-cert=<path>: Path to a CA file to use with Vault. Overrides theVAULT_CACERTenvironment variable and the Vault CA path.-vault-ca-path=<path>: Path to a directory of PEM encoded CA cert files to verify the Vault certificate. Overrides theVAULT_CAPATHenvironment variable.
Output
This command prints a summary of the capture and the name of the timestamped archive file produced.
Examples