Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

Table of Contents
minLevel1
maxLevel2
outlinefalse
typelist
printablefalse

The defined SwarmFS exports allow users to access the named objects in Swarm storage. The Swarm namespace is flat below the bucket level. Users need to see and interact with these objects in a familiar way. Content Gateway includes a listing service translating that translates the object store in to into virtual file hierarchies. (v3.0)

Gateway achieves a file-system-like hierarchical structure on those objects by splitting them in to into “files” and “folders,” where

...

  • Synthetic folders do not exist in Swarm: they are client-side creations from the names of objects in Swarm.

  • Pseudo folders are stored in Swarm as a an object with no content, whose name ends in the delimiter (“/”). These objects define any placeholder folders as created by users (reserve/this/folder/).

Drawio
zoom1
simple0
inComment0
custContentId1252262567
pageId1253049103custContentId1252262567
lbox0
diagramDisplayNamefolder-listing.drawio
contentVer4
revision2
baseUrlhttps://caringo.atlassian.net/wiki
diagramNamefolder-listing.drawio
pCenter0
width613
links
tbstyle
height282.5

Listing Behaviors

The listing service translates any prefix+delimiter it finds in an object name in to into a folder listing. It renders data so it exactly matches the object stored in Swarm, other than a few milliseconds to acquire it. Listings sort are sorted by name, as is the S3 standard. 

Delimiters — : It uses a globally configured delimiter (defaults to slash “/”) in Swarm (the storage setting search.pathDelimiter); different characters can be used, but performance may be slower (see Settings Reference). SwarmFS converts slashes to backslashes inline during listing transfers if a client expects backslashes to be the directory delimiter. SwarmFS always converts backslashes to forward slashes when communicating with Swarm.

Info

Important

Object and bucket names must not start with a delimiter (/) to be valid. They do not appear in SwarmFS listings if existing objects are named like this. Rename and upload objects with valid names if this occurs.

Metadata Metadata: Listing returns objects with all metadata. It stores and renders pseudo folders as objects, which may hold usable folder and POSIX metadata. See Metadata Headers.

Listing delayDelay: New objects may be missed, or existing objects show up in the results twice, temporarily if native Swarm objects change between fetches. Objects created in Swarm natively (not via SwarmFS) can take up to 5 minutes plus the Search Feed's Feeds Batch Timeout to appear in SwarmFS listings, because they must be indexed by Elasticsearch. For best listing performance, lower the search feed's Batch Timeout to 1 or 0 (recommended). See Search Managing Feeds.

Versions : For versioned buckets, SwarmFS provides all object versions in the listings, or, on client request, returns the current versions and delete markers for the deleted objects. See Implementing Versioning.

Encoding : SwarmFS supports UTF-8 encoded names, and the REST interface returns UTF-8-encoded results. 

Exclusive opens Opens: SwarmFS supports exclusive opens of a file (O_EXCL and O_CREATE) but does not support exclusive reopens (EXCLUSIVE4).

...

Note

The listing service does not currently include unnamed objects, caching, folder locking/leasing, or client notification of namespace changes.

How Listings Appear to Users

SwarmFS creates a simulation: it translates each forward slash (/) in any object name in to a traditional directory delimiter to present named object listings as if they are in a traditional file system. SwarmFS then presents a view of objects at the simulated directory level.

...

  • By default, when a user requests a listing of the bucket contents, SwarmFS provides a simulated view of the first level of contents within the bucket.

  • SwarmFS returns a new listing for a directory each time a user opens a specific directory level within the bucket (such as Sales/Leads/campaigns/).

  • A single file is returned if a user requests a listing for a file (such as Sales/Leads/campaigns/addword.xls). 

  • SwarmFS creates a new empty object with Directory attribute metadata, which marks it as a pseudo directory if a user creates a new directory (such as AcmeBucket/Sales/Leads/campaigns/2019/Mar/).

  • SwarmFS removes it and all files it contains if a user deletes a directory.

  • A user cannot move or rename a directory. Instead, they need to create the directory they want and move the files into it. (NFS-607)

Simulated directory objects exist to support the expectations of end users and applications: objects can be written to a new virtual directory by including it in the pathname.

...