Create distributed search groups

You can group your search peers to facilitate searching on a subset of them. Groups of search peers are known as "distributed search groups." You specify distributed search groups in the distsearch.conf file.

For example, say you have a set of search peers in New York and another set in San Francisco, and you want to perform searches across peers in just a single location. You can do this by creating two search groups, NYC and SF. You can then specify the search groups in searches.

Distributed search groups are particularly useful when configuring the monitoring console. See Monitoring Splunk Enterprise.

Configure distributed search groups

You define distributed search groups in distsearch.conf.

For example, to create the two search groups NYC and SF, create stanzas like these:

[distributedSearch]
# This stanza lists the full set of search peers.
servers = 192.168.1.1:8089, 192.168.1.2:8089, 175.143.1.1:8089, 175.143.1.2:8089, 175.143.1.3:8089
[distributedSearch:NYC]
# This stanza lists the set of search peers in New York.
default = false
servers = 192.168.1.1:8089, 192.168.1.2:8089
[distributedSearch:SF]
# This stanza lists the set of search peers in San Francisco.
default = false
servers = 175.143.1.1:8089, 175.143.1.2:8089, 175.143.1.3:8089

Note the following:

• The servers attribute lists groups of search peers by IP address and management port.
• The servers list for each search group must be a subset of the list in the general [distributedSearch] stanza.
• The group lists can overlap. For example, you can add a third group named "Primary_Indexers" that contains some peers from each location.
• If you set a group's default attribute to "true," the peers in that group will be the ones queried when the search does not specify a search group. Otherwise, if you set all groups to "false," the full set of search peers in the [distributedSearch] stanza will be queried when the search does not specify a search group.

Use distributed search groups

To use a search group in a search, specify the search group like this:

sourcetype=access_combined status=200 action=purchase splunk_server_group=NYC | stats count by product

This search runs against only the peers in the NYC location.

Distributed search groups and indexer clusters

This feature is not valid for indexer clustering, except for limited use cases in certain complex topologies.

In indexer clustering, the cluster replicates the data buckets arbitrarily across the set of search peers, or "cluster peer nodes". It then assigns one copy of each bucket to be the primary copy, which participates in searches. There is no guarantee that a specific peer or subset of peers will contain the primary bucket copies for a particular search. Therefore, if you put peers into distributed search groups and then run searches based on those groups, the searches might contain incomplete results.

For details of bucket replication in indexer clusters, see Buckets and indexer clusters in Managing Indexers and Clusters of Indexers.

These are some examples of indexer cluster deployments where distributed search groups might be of value:

• Multiple indexer clusters, where you need to identify the peer nodes for a specific cluster.
• Search heads that run searches across both an indexer cluster and standalone indexers. You might want to put the standalone indexers into their own group.