Copia Segura en Cascada

El módulo Net::CascadeCopy de Alex White provee la utilidad ccp que permite la copia paralela de ficheros. La paralelización sigue una estructura arbórea. El siguiente extracto de la documentación explica el método:

A frequent solution to distributing a file or directory to a large number of servers is to copy it from a central file server to all other servers. To speed this up, multiple file servers may be used, or files may be copied in parallel until the inevitable bottleneck in network/disk/cpu is reached. These approaches run in O(n) time.

This module and the included script, ccp, take a much more efficient approach that is O(log n). Once the file(s) are been copied to a remote server, that server will be promoted to be used as source server for copying to remaining servers. Thus, the rate of transfer increases exponentially rather than linearly. Needless to say, when transferring files to a large number of remote servers (e.g. over 40), this can make a ginormous difference.

Servers can be specified in groups (e.g. datacenter) to prevent copying across groups. This maximizes the number of transfers done over a local high-speed connection (LAN) while minimizing the number of transfers over the WAN.

Sigue un ejemplo de ejecución:

pp2@europa:~/src/perl/perltesting$ ccp -s -f ShellUI.tex -g bno:beowulf,nereida,orion
2009/03/31 09:07:01 Adding group: bno: beowulf, nereida, orion
2009/03/31 09:07:01 Starting: (bno) localhost => beowulf
2009/03/31 09:07:02 Succeeded: (bno) localhost => beowulf (1 seconds)
2009/03/31 09:07:02 BNO: completed:1 running:0 left:2 errors:0 failures:0
2009/03/31 09:07:02 Starting: (bno) beowulf => nereida
2009/03/31 09:07:03 Starting: (bno) beowulf => orion
2009/03/31 09:07:04 Succeeded: (bno) beowulf => orion (1 seconds)
2009/03/31 09:07:04 BNO: completed:2 running:1 left:0 errors:0 failures:0
2009/03/31 09:07:04 Succeeded: (bno) beowulf => nereida (2 seconds)
2009/03/31 09:07:04 BNO: completed:3 running:0 left:0 errors:0 failures:0
2009/03/31 09:07:04 Job completed in 3 seconds
2009/03/31 09:07:04 Cumulative tansfer time of all jobs: 4 seconds
2009/03/31 09:07:04 Approximate Time Saved: 1 seconds

Para que funcione es necesario que todas las máquinas implicadas se reconozcan mediante autentificación automática. La siguiente transferencia falla por que la máquina europa no reconoce a la máquina beowulf

pp2@europa:~/src/perl/perltesting$ ccp -s -f ShellUI.tex -g bno:beowulf,nereida,europa
2009/03/31 08:51:19 Adding group: bno: beowulf, nereida, europa
2009/03/31 08:51:19 Starting: (bno) localhost => beowulf
2009/03/31 08:51:20 Succeeded: (bno) localhost => beowulf (1 seconds)
2009/03/31 08:51:20 BNO: completed:1 running:0 left:2 errors:0 failures:0
2009/03/31 08:51:20 Starting: (bno) beowulf => nereida
2009/03/31 08:51:21 Starting: (bno) beowulf => europa
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,password).
lost connection
2009/03/31 08:51:21 child exit status: 1
2009/03/31 08:51:22 Failed: (bno) beowulf => europa (1 seconds)
2009/03/31 08:51:22 BNO: completed:1 running:1 left:1 errors:1 failures:0
2009/03/31 08:51:22 Succeeded: (bno) beowulf => nereida (2 seconds)
2009/03/31 08:51:22 BNO: completed:2 running:0 left:1 errors:1 failures:0
2009/03/31 08:51:22 Starting: (bno) nereida => europa
Host key verification failed.
lost connection
2009/03/31 08:51:22 child exit status: 1
2009/03/31 08:51:23 Failed: (bno) nereida => europa (1 seconds)
2009/03/31 08:51:23 BNO: completed:2 running:0 left:1 errors:2 failures:0
2009/03/31 08:51:23 Starting: (bno) nereida => europa
Host key verification failed.
lost connection
2009/03/31 08:51:23 child exit status: 1
2009/03/31 08:51:24 Failed: (bno) nereida => europa (1 seconds)
2009/03/31 08:51:24 Error: giving up on (bno) europa
2009/03/31 08:51:24 BNO: completed:2 running:0 left:0 errors:3 failures:1
2009/03/31 08:51:24 Job completed in 5 seconds
2009/03/31 08:51:24 Cumulative tansfer time of all jobs: 6 seconds
2009/03/31 08:51:24 Approximate Time Saved: 1 seconds