Uploaded image for project: 'StoRM'
  1. StoRM
  2. STOR-1220

Enabling GPFS preallocation causes gfal-copy failures

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Fixed
    • Icon: Major Major
    • None
    • 1.11.18
    • Security Level: Public (Visbile by non-authn users.)
    • None

      On our CentOS 7 testbed we faced goal-copy failures with the following message: "file already exists and is not empty”.

      This error was a combination of GPFS preallocation enabled on Backend and the latest updates to StoRM GridFTP plugin.

      After some investigation, in fact, we discovered that within a gfal-copy an expectedFileSize is specified on the prepare-to-put. If Backend is configured with STORM_INFO_USE_GPFS_PREALLOCATION=True, the class it.grid.storm.filesystem.GPFSSpaceSystem is used as space-system driver instead of it.grid.storm.filesystem.MockSpaceSystem. With this GPFS driver, during the ptp a “reserve space” is done through a gpfs_prealloc. The consequence is that the created file is empty but has also a size > 0. This causes gridftp failures with the following message: "file already exists and is not empty”.

      This error message is a consequence of the fix done for:
      https://issues.infn.it/jira/browse/STOR-1102 
      The involved code is the following:
      https://github.com/italiangrid/storm-gridftp-dsi/commit/0ba2e838159eb8ca7d7fbcf3244b9c218df7f4a4

      I started an investigation on the several StoRM sites in order to understand if this driver is really used/necessary.
      Thread: https://lists.infn.it/sympa/arc/storm-users/2020-07/msg00000.html

            vianello Enrico Vianello
            vianello Enrico Vianello
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: