Perforce Slow Sync issues
Network latency
Sync Performance issues are not obvious with locally connected hosts, as the network latency is low. However, as latency increases, performance worsens.
To correct the issue, follow these steps on the client machines exhibiting the issue:
http://kb.perforce.com/article/1191/slow-sync-on-remote-windows-clients
Network problems:
How do I determine if slow Perforce response time is caused by network problems?
A network issue can be suspected if Perforce commands run quickly on the local machine but run slowly across a network. You can also compare the lapse time against the usage time, for example, if the information is available in the Perforce logs.
Any of the following can cause slow responses:
- Misconfigured domain name server (DNS)
- Misconfigured Windows name server (WINS) or Windows domain
- Slow network response
1. p4 info
A good initial test is to run the p4 info command. If this command does not respond nearly immediately then there is a network related problem.
The p4 info command uses the P4PORT setting to contact the Perforce Server. If the P4PORT setting uses the hostname of the server machine, a DNS lookup is required to fetch the server IP address. If the DNS server is failing or the network is slow, the lookup process takes time. You can use the IP address directly to avoid the DNS lookup.
2. Hostname vs. IP Address
On a client machine, try using the Perforce Server’s IP address in the P4PORT setting. This avoids the DNS lookup used to convert the hostname to an IP address. Here is a P4PORT example which uses the server hostname:
P4PORT=hostname:1666
Using the IP address directly, the P4PORT setting then looks like this:
P4PORT=1.2.3.4:1666
If you do not have the Perforce Server machine’s IP address handy, you can use the ping command to find it. Here is an example of how you would run the ping command:
3. ping hostnam
The output of the ping command lists the IP address for the hostname.
If the p4 info command responds immediately when you use the IP address in the P4PORT setting, you have a misconfigured DNS.
“p4 info” vs. P4Win
The p4 info command is processed by the Perforce Server. As the Perforce Server compiles the output information for the command, it does a reverse DNS lookup on the client’s IP address. A forward DNS lookup might be fast, while a reverse DNS lookup is slow. One way to determine if a reverse DNS lookup is slow is by using P4Win, the Perforce Visual Client. The “Show Connection Info” operation in P4Win does not perform a reverse DNS lookup.
You can compare the response of P4Win’s “Show Connection Info” with the response from the command line p4 info. If a “Show Connection Info” operation is fast while a p4 info is slow, there is a reverse DNS lookup problem on the Perforce Server machine.
4. The “hosts” file
To work around DNS problems, add hostname IP address entries to the hosts file. This task is typically performed by a Systems Administrator; be sure to follow your company’s standard procedures.
Rather than using an IP address for the P4PORT setting, you can add a hostname IP address entry in your hosts file. The host file can be tricky to find. Here are a few places it can be:
Windows Server 2000 (and later):
C:\Winnt\System32\Drivers\etc\hosts
Unix:
/etc/hosts
The following is an example entry for the hosts file:
1.2.3.4 hostname
If you have determined there is a reverse DNS problem, you need to add an entry for your client hostname in the machine’s hosts file where the Perforce Server resides.
5. Wildcards on Windows
In some cases, p4 commands using unquoted file patterns with a combination of depot syntax or client syntax and wildcards can cause delays.
p4 files //depot/*
p4 files //client/*
The delay occurs because the wild card expansion routines for Windows mistakes the depot name or client name for a network system name. You can prevent the delay by putting double quotes around the file pattern:
p4 files “//depot/*”
Network Topology
Most networks use 100 Mbit technology. The maximum theoretical transfer rate of this type of network is 10 Mbytes per second. In most cases slower transfer rates are experienced. If a network is saturated, transfer rates can drop off significantly, maybe as low as 4 Mbytes per second. In cases of network saturation use of network routers or switches can help.
6. Client on a Network Filesystem
It is possible that the p4 executable itself is on a networked file system that is performing very poorly. To check for slow access to the executable, try running:
p4 -v
The p4 -v command simply prints out the version information and does not attempt to access the network. If you get a slow response, network access to the p4 executable itself may be the problem. Try copying or downloading the p4 executable onto a local filesystem.
7. Server on a Network Filesystem
Using a network filesystem for the Perforce Server must be thought through very carefully. Just as in the scenario above with the Perforce client stored on a network filesystem, data to and from the server application must go through the network twice. The Perforce Server also uses file locking on vital data files. Not all network filesystems have efficient locking implementations and some are buggy.
If the network is saturated and the transfer rate is down to about 4 Mbytes per second, the Perforce Server cannot satisfy client requests. Each client request uses part of the network, and the Perforce Server must share that resource in order to access the network filesystem. A cheap hard drive these days will provide a 20 Mbyte per second transfer rate. A good SCSI hard drive can transfer as much as 160 Mbytes per second. If possible, try to avoid a network filesystem related server bottle neck.
Try and play with “p4 ping” by progressively increasing the message count, receive and transmit lengths (one at a time). If you hit the same wall repeatedly, then this could very well be indicative of some throttling taking place somewhere
8. p4 -z tag info
9. Reference.
http://kb.perforce.com/article/40/isolating-network-problems
http://kb.perforce.com/article/1191/slow-sync-on-remote-windows-clients
http://kb.perforce.com/article/1462/slow-sync-performance-on-windows
- DevOps Certified Professionals (DCP)
- Site Reliability Engineering Certified Professionals (SRECP)
- Master in DevOps Engineering (MDE)
- DevSecOps Certified Professionals (DSOCP)
URL - https://www.devopsschool.com/certification/
My Linkedin - https://www.linkedin.com/in/rajeshkumarin
- Best AI tools for Software Engineers - November 4, 2024
- Installing Jupyter: Get up and running on your computer - November 2, 2024
- An Introduction of SymOps by SymOps.com - October 30, 2024
