What exactly is Consul?
Consul is an open-source platform for service networking developed by HashiCorp. Consul is a service mesh solution with a full-featured control plane for service discovery, configuration, and segmentation. Each of these features can be used separately or in tandem to create a complete service mesh. Consul requires the use of a data plane and provides a proxy as well as a native integration mechanism. Consul includes a built-in proxy that allows you to get started right away, but it also supports third-party proxy integrations such as Envoy.
What are the key features of Consul?
Consul was designed with DevOps and application developers in mind, with a focus on modern, elastic infrastructures.
Discovering Services- Clients of Consul can register a service, such as api or mysql, and other clients can use Consul to find providers. Using DNS or HTTP, applications can quickly locate the services they require.
Examine your health- Consul clients keep track of cluster health and assist service discovery components in avoiding sending traffic to unhealthy hosts.
KV Store- Dynamic configuration, feature flagging, coordination, and leader election are just a few of the applications for Consul’s hierarchical key/value store.
Communication between services that is secure
Secure Service Communication- Consul can generate and distribute TLS certificates for services in order to establish mutual TLS connections. Intentions can be used to specify which services are allowed to talk to each other. Service segmentation can be easily managed with intentions that can be changed in real time, rather than using complex network topologies and static firewall rules.
Datacenters in multiple locations- Consul is capable of supporting multiple data centers. This means that users of Consul will not have to worry about adding more abstraction layers in order to expand to multiple locations.
What is the purpose of Consul Architecture?
Consul, as can be seen, is in charge of three different servers. The Raft algorithm is used in the working architecture to select a leader from among the three servers. These servers are given names like Follower and Leader. As the name implies, the follower is responsible for adhering to the leader’s decisions. All three servers are linked together for any communication.
Each server communicates with its own client using the RPC concept. Clients can communicate with one another using the Gossip Protocol. TCP or the gossip communication technique can be used to make the internet communication facility available. This communication is done directly with one of the three servers.
In Consul, what is the Raft Algorithm?
Raft is a log management algorithm that uses a consensus algorithm to manage a replicated log. It’s based on the CAP Theorem, which states that when there’s a network partition, you have to choose between consistency and availability. Not all three fundamentals of the CAP Theorem can be met at any given time. For any two of them, at the very least, a tradeoff must be made.
Multiple servers, usually in odd numbers, make up a Raft Cluster. For example, if we have five servers, the system can withstand two failures. At any given time, each server is in one of three states: Leader, Follower, or Candidate. There is only one leader in a typical operation, and every other server is a follower. These followers are in a passive state, which means they don’t make requests on their own but rather respond to those made by the candidate and the leaders.
What Exactly Is A Checkpoint? / Does Consul Call Home?
Consul makes use of Checkpoint, a HashiCorp service that checks for updates and critical security bulletins. Checkpoint receives only anonymous data that cannot be used to identify the user or host. An anonymous ID is sent, which aids in the de-duplication of alert messages. This pseudonymous ID can be turned off. Using the Checkpoint service is, in fact, optional and can be turned off.
Is Consul a UDP Broadcast Or Multicast System?
Consul employs the Serf gossip protocol, which is based on unicast TCP and UDP. In a multi-tenant or cloud network environment, broadcast and multicast are rarely available. As a result, both Consul and Serf were created to avoid any reliance on those abilities.
Is Consul Eventually Consistent Or Consistently Consistent?
The service catalogue and the gossip protocol are two important subsystems of Consul. All nodes, service instances, health check data, ACLs, and KV information are stored in the service catalogue. It has a high level of consistency and has been replicated using the consensus protocol.
The gossip protocol is used to keep track of which nodes are in the cluster as well as to detect node or agent failure. By its very nature, this data is eventually consistent. The service catalogue is updated when the servers detect a change in membership or receive a health update.
The answer to the question is nuanced as a result of this split. Almost every client API interacts with the service catalogue and follows a strict set of rules. Updates to the catalogue may arrive via the gossip protocol, which is eventually consistent, so the catalog’s current state may lag until the state is reconciled.
Is Delta Updates For Watchers Or Blocking Queries Supported By Consul?
A delta or a change-only response to a watcher or a blocking query is not currently supported by Consul. The API simply allows for a full-result edge-trigger return. A client should save the results of their most recent read and compute the delta on the client side.
Instead of attempting to support the delta calculation, Consul offloads it to clients by design. This eliminates the need for costly server maintenance as well as race conditions between data updates and watch registrations.
Consul uses which network ports?
Consul uses all of the ports listed in the Ports Used section of the Configuration documentation.
Is it necessary for Consul to set resource limits for user processes?
A Consul client agent should only require a small number of open file descriptors. Each connection to the client agent (for example, for a blocking query) will open a connection, and there will typically be connections to one of the Consul servers. Watch handlers, health checks, log files, and other applications require a small number of file descriptors.
You should plan on the above requirements for a Consul server agent, as well as an additional incoming connection from each node in the cluster. Although this should not be the case in most cases, in the worst-case scenario, if one of the other servers fails, the other client agents should all connect to the same server, so planning for this possibility is beneficial.
The default limits are usually sufficient for Consul, but before arbitrarily increasing the limits, you should carefully examine your own environment’s specific needs and identify the root cause of any excessive resource utilization.
What is the size limit for per-key value sizes in Consul’s Key/Value Store?
The maximum size of a key’s value is 512KB. This is strictly enforced, and any client attempting to store more than that limit in a value will receive an HTTP 413 response. It’s important to keep in mind that the Consul key/value store isn’t intended to be used as a general-purpose database. For more information, see Server Performance.
What Information Is Shared Among Consul Datacenters?
Data is not replicated between different Consul datacenters in general. When a request is made for a resource in another datacenter, the local Consul servers send an RPC request to that resource’s remote Consul servers and return the results. If the remote datacenter is unavailable, those resources will be unavailable as well, but the local datacenter will be unaffected. A limited subset of data can be replicated in some special cases, such as with Consul’s built-in ACL replication capability or external tools like consul-replicate.
What Is A Consul Server And How Does It Work?
Consul is a service discovery and configuration system that is distributed, highly available, and datacenter-aware. It can be used to present services and nodes in a flexible and powerful interface that allows clients to have an always-up-to-date view of the infrastructure to which they belong.
What Is Hashicorp Consul and How Does It Work?
HashiCorp Consul is an open source tool that provides service discovery, health checks, load balancing, a service graph, mutual TLS identity enforcement, and a configuration key-value store to address these new complexities. Consul is an ideal control plane for a service mesh because of these features.
What Is Docker Consul and How Does It Work?
We’re excited to announce the availability of our official Consul Docker image. Consul is a modern datacenter runtime that allows you to discover, configure, and orchestrate services. Consul is the first tool in the Consul family to have an official Docker image.
What Exactly Is A Consul Agent?
Consul’s core process is the Consul agent. The agent is in charge of maintaining membership information, registering services, performing checks, and responding to queries, among other things. The agent must be installed on each node in a Consul cluster. Any agent can run in either client or server mode.
- Interview Questions & Answers Complete Guide for Artifactory - January 11, 2022
- Interview Questions & Answers Complete Guide for Consul - January 10, 2022
- Interview Questions & Answers Complete Guide for Nexus - January 7, 2022
Consul on Kubernetes
»Q: Can I upgrade directly to a specific Helm chart version or should I upgrade one patch release at a time?
It is safe to upgrade directly to a specific version. Be sure to read the release notes for all versions you’re upgrading through and look for any breaking changes.
»Q: Can I upgrade in place or should I spin up a new Kubernetes cluster?
It is always safer to spin up a new Kubernetes cluster but that is not an option for most teams. Consul supports upgrading in place.
Non-production environments should be upgraded first. If upgrading a Consul version, Consul data should be backed up.
»Q: How can I run tcpdump on Consul servers?
First, add the following to your
values.yamlfile so you cankubectl execinto the Consul server containers as root:server: securityContext: runAsNonRoot: false runAsGroup: 0 runAsUser: 0 fsGroup: 0Copy
Run a
helm upgrade(see Upgrade Consul on Kubernetes for full upgrade instructions).Now,
kubectl execinto a server pod:Copy
Install
tcpdump:Copy
»Generic Consul Questions
»Q: What is Checkpoint? / Does Consul call home?
Consul makes use of a HashiCorp service called Checkpoint which is used to check for updates and critical security bulletins. Only anonymous information, which cannot be used to identify the user or host, is sent to Checkpoint. An anonymous ID is sent which helps de-duplicate warning messages. This anonymous ID can be disabled. In fact, using the Checkpoint service is optional and can be disabled.
See
disable_anonymous_signatureanddisable_update_check.»Q: Does Consul rely on UDP Broadcast or Multicast?
Consul uses the Serf gossip protocol which relies on TCP and UDP unicast. Broadcast and Multicast are rarely available in a multi-tenant or cloud network environment. For that reason, Consul and Serf were both designed to avoid any dependence on those capabilities.
»Q: Is Consul eventually or strongly consistent?
Consul has two important subsystems, the service catalog and the gossip protocol. The service catalog stores all the nodes, service instances, health check data, ACLs, and KV information. It is strongly consistent, and replicated using the consensus protocol.
The gossip protocol is used to track which nodes are part of the cluster and to detect a node or agent failure. This information is eventually consistent by nature. When the servers detects a change in membership, or receive a health update, they update the service catalog appropriately.
Because of this split, the answer to the question is subtle. Almost all client APIs interact with the service catalog and are strongly consistent. Updates to the catalog may come via the gossip protocol which is eventually consistent meaning the current state of the catalog can lag behind until the state is reconciled.
»Q: Are failed or left nodes ever removed?
To prevent an accumulation of dead nodes (nodes in either failed or left states), Consul will automatically remove dead nodes out of the catalog. This process is called reaping. This is currently done on a configurable interval of 72 hours. Reaping is similar to leaving, causing all associated services to be deregistered. Changing the reap interval for aesthetic reasons to trim the number of failed or left nodes is not advised (nodes in the failed or left state do not cause any additional burden on Consul).
»Q: Does Consul support delta updates for watchers or blocking queries?
Consul does not currently support sending a delta or a change only response to a watcher or a blocking query. The API simply allows for an edge-trigger return with the full result. A client should keep the results of their last read and compute the delta client side.
By design, Consul offloads this to clients instead of attempting to support the delta calculation. This avoids expensive state maintenance on the servers as well as race conditions between data updates and watch registrations.
»Q: What network ports does Consul use?
The Ports Used section of the Configuration documentation lists all ports that Consul uses.
»Q: Does Consul require certain user process resource limits?
There should be only a small number of open file descriptors required for a Consul client agent. The gossip layers perform transient connections with other nodes, each connection to the client agent (such as for a blocking query) will open a connection, and there will typically be connections to one of the Consul servers. A small number of file descriptors are also required for watch handlers, health checks, log files, and so on.
For a Consul server agent, you should plan on the above requirements and an additional incoming connection from each of the nodes in the cluster. This should not be the common case, but in the worst case if there is a problem with the other servers you would expect the other client agents to all connect to a single server and so preparation for this possibility is helpful.
The default ulimits are usually sufficient for Consul, but you should closely scrutinize your own environment’s specific needs and identify the root cause of any excessive resource utilization before arbitrarily increasing the limits.
»Q: What is the per-key value size limitation for Consul’s key/value store?
The default recommended limit on a key’s value size is 512KB. This is strictly enforced and an HTTP 413 status will be returned to any client that attempts to store more than that limit in a value. The limit can be increased by using the
kv_max_value_sizeconfiguration option.It should be noted that the Consul key/value store is not designed to be used as a general purpose database. See Server Performance for more details.
»Q: What data is replicated between Consul datacenters?
In general, data is not replicated between different Consul datacenters. When a request is made for a resource in another datacenter, the local Consul servers forward an RPC request to the remote Consul servers for that resource and return the results. If the remote datacenter is not available, then those resources will also not be available from that datacenter. That will not affect the requests to the local datacenter. There are some special situations where a limited subset of data can be replicated, such as with Consul’s built-in ACL replication capability, or external tools like consul-replicate.
»Q: Can Consul natively handle protecting against other processes accessing Consul’s memory state?
Consul does not provide built-in memory access protections, and doesn’t interact with the host system to change or manipulate viewing and doesn’t interact with the host system to change or manipulate application security.
We recommend taking any precautions or remediation steps that you would normally do for individual processes, based on your operating system.
Please see our Security Model for more information.
»Q: Are the Consul Docker Images OCI Compliant?
The official Consul Docker image uses Docker image schema V2, which is OCI Compliant. To check the docker images on Docker Hub, use the command
docker manifest inspect consulto inspect the manifest payload. Thedocker manifest inspectmay require you to enable experimental features to use.»Q: What browsers are supported by the Consul UI?
Consul currently supports all ‘evergreen’ browsers, as they are generally on up-to-date versions. This means we support:
We do not support Internet Explorer 11 (IE 11). Consul follows a similar alignment with Microsoft’s own stance on IE 11, found on their support website.