Take the 2-minute tour ×
Super User is a question and answer site for computer enthusiasts and power users. It's 100% free, no registration required.

I am trying to do p4 sync folder/...

It's not very fast and it's using only 2% of the CPU. I am not happy with the performance.

I am trying to write a function in bash, given a parameter n, it forms a queue of all files and folders n levels deep and forks of a process of p4 sync -f $thatDirectory/... or p4 sync -f $thatFile.

How do I start about this? Is there any utility that already does this?

Thanks in advance.

share|improve this question

3 Answers 3

up vote 2 down vote accepted

I have zero experience with Perforce, but this sync sounds like an I/O-bound operation. Parallelizing it in multiple processes isn't likely to help, because the bottleneck is the disk or network.

If the disk is the bottleneck, parallelizing might actually make it slower, because you'll have lots of processes accessing different files simultaneously, causing the disk heads to thrash around.

Before you jump into scripting, do some analysis to figure out what's making it slow. Use tools like strace and Wireshark to study read/write calls and network traffic, for example.

share|improve this answer

sometimes the p4d (the server) process has been started with a '-f' flag that will run it as a single threaded process.

couple of points to note

try using the -p option to sync instead - it means less db writes and that has to be a good thing !

you could try a 'go n levels deep and sync from there down' type of thing, something like :

$ p4 dirs //<path>/* | while read path ; do

                p4 dirs $path/* |

                while read subpath;

                do

                                p4 dirs $subpath/* |

                                while read subsubpath;

                                do

                                                p4 <cmd>  $subsubpath/... ;

                                done;

                                p4 <cmd> $path/...  # to catch the files at this level

                done;

                p4 <cmd> //<path>/... # to catch the files at this (top) level done

 

in your case, you'd have <cmd> equal to "sync" + any other args.

 

share|improve this answer

I would stear away from the sync per files... especially so if you have multiple users trying to do this from an embedded tool or something similar. We completely killed our server here on multiple occasion with stunts like these.

Perforce does however offer an option to parallelize your sync for you --parallel, it does require some server side adjustments but you should give it a try.

you could shard in multiple invocation of p4.exe but I would try and keep them small, at least not beyond your available core count.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.