Uploaded image for project: 'StoRM'
  1. StoRM
  2. STOR-1102

Investigate errors on transferred files through GridFTP that leave empty files with an adler32 checksum for a non-empty file

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Minor Minor
    • 1.11.18
    • 1.11.15
    • gridftp-plugin
    • Security Level: Public (Visbile by non-authn users.)
    • None

      We see a situation whereby a file is empty but has the adler32 attribute with a value different from 00000001 (which is the checksum for an empty file).

      The hypthosis is that the following happens:

      • FTS executes a PtP on StoRM and starts a transfer on one GridFTP server
      • the GridFTP server, for some reason, gets stuck
      • FTS times out the transfer, srmRm the file and re-issues the PtP, landing on another GridFTP server
      • this time the transfer succeeds, the file is not empty and the checksum is correctly computed and recorded in the xattr
      • the first GridFTP server wakes up and starts from where it left the first transfer. In particular it opens the file with O_TRUNC, making it empty. Soon after it stops because the transfer was aborted, without touching the xattr with the checksum

      More details in a mail thread on storm-devel.

      The tentative fix (rather, a mitigation) consists in checking that the file is empty before opening it in the StoRM GridFTP DSI module. This assumes that the GridFTP server was stuck before the open; of course if the server gets stuck after the check the problem would persist.

            giaco Francesco Giacomini
            vianello Enrico Vianello
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:
              Resolved: