El módulo Net::CascadeCopy de Alex White provee la utilidad ccp que permite la copia paralela de ficheros. La paralelización sigue una estructura arbórea. El siguiente extracto de la documentación explica el método:
A frequent solution to distributing a file or directory to a large number of servers is to copy it from a central file server to all other servers. To speed this up, multiple file servers may be used, or files may be copied in parallel until the inevitable bottleneck in network/disk/cpu is reached. These approaches run in O(n) time.
This module and the included script, ccp, take a much more efficient approach that is O(log n). Once the file(s) are been copied to a remote server, that server will be promoted to be used as source server for copying to remaining servers. Thus, the rate of transfer increases exponentially rather than linearly. Needless to say, when transferring files to a large number of remote servers (e.g. over 40), this can make a ginormous difference.
Servers can be specified in groups (e.g. datacenter) to prevent copying across groups. This maximizes the number of transfers done over a local high-speed connection (LAN) while minimizing the number of transfers over the WAN.Sigue un ejemplo de ejecución:
pp2@europa:~/src/perl/perltesting$ ccp -s -f ShellUI.tex -g bno:beowulf,nereida,orion 2009/03/31 09:07:01 Adding group: bno: beowulf, nereida, orion 2009/03/31 09:07:01 Starting: (bno) localhost => beowulf 2009/03/31 09:07:02 Succeeded: (bno) localhost => beowulf (1 seconds) 2009/03/31 09:07:02 BNO: completed:1 running:0 left:2 errors:0 failures:0 2009/03/31 09:07:02 Starting: (bno) beowulf => nereida 2009/03/31 09:07:03 Starting: (bno) beowulf => orion 2009/03/31 09:07:04 Succeeded: (bno) beowulf => orion (1 seconds) 2009/03/31 09:07:04 BNO: completed:2 running:1 left:0 errors:0 failures:0 2009/03/31 09:07:04 Succeeded: (bno) beowulf => nereida (2 seconds) 2009/03/31 09:07:04 BNO: completed:3 running:0 left:0 errors:0 failures:0 2009/03/31 09:07:04 Job completed in 3 seconds 2009/03/31 09:07:04 Cumulative tansfer time of all jobs: 4 seconds 2009/03/31 09:07:04 Approximate Time Saved: 1 seconds
Para que funcione es necesario que todas las máquinas implicadas se reconozcan mediante autentificación automática. La siguiente transferencia falla por que la máquina europa no reconoce a la máquina beowulf
pp2@europa:~/src/perl/perltesting$ ccp -s -f ShellUI.tex -g bno:beowulf,nereida,europa 2009/03/31 08:51:19 Adding group: bno: beowulf, nereida, europa 2009/03/31 08:51:19 Starting: (bno) localhost => beowulf 2009/03/31 08:51:20 Succeeded: (bno) localhost => beowulf (1 seconds) 2009/03/31 08:51:20 BNO: completed:1 running:0 left:2 errors:0 failures:0 2009/03/31 08:51:20 Starting: (bno) beowulf => nereida 2009/03/31 08:51:21 Starting: (bno) beowulf => europa Permission denied, please try again. Permission denied, please try again. Permission denied (publickey,password). lost connection 2009/03/31 08:51:21 child exit status: 1 2009/03/31 08:51:22 Failed: (bno) beowulf => europa (1 seconds) 2009/03/31 08:51:22 BNO: completed:1 running:1 left:1 errors:1 failures:0 2009/03/31 08:51:22 Succeeded: (bno) beowulf => nereida (2 seconds) 2009/03/31 08:51:22 BNO: completed:2 running:0 left:1 errors:1 failures:0 2009/03/31 08:51:22 Starting: (bno) nereida => europa Host key verification failed. lost connection 2009/03/31 08:51:22 child exit status: 1 2009/03/31 08:51:23 Failed: (bno) nereida => europa (1 seconds) 2009/03/31 08:51:23 BNO: completed:2 running:0 left:1 errors:2 failures:0 2009/03/31 08:51:23 Starting: (bno) nereida => europa Host key verification failed. lost connection 2009/03/31 08:51:23 child exit status: 1 2009/03/31 08:51:24 Failed: (bno) nereida => europa (1 seconds) 2009/03/31 08:51:24 Error: giving up on (bno) europa 2009/03/31 08:51:24 BNO: completed:2 running:0 left:0 errors:3 failures:1 2009/03/31 08:51:24 Job completed in 5 seconds 2009/03/31 08:51:24 Cumulative tansfer time of all jobs: 6 seconds 2009/03/31 08:51:24 Approximate Time Saved: 1 seconds
Casiano Rodríguez León