The defined SwarmFS exports allow users to access the named objects in Swarm storage. The Swarm namespace is flat below the bucket level. Users need to see and interact with these objects in a familiar way. Content Gateway includes a listing service that translates the object store into virtual file hierarchies. (v3.0)

Gateway achieves a file-system-like hierarchical structure on those objects by splitting them into “files” and “folders,” where

• files are content objects (such as videos) 

• folders are file system directories Gateway renders from object names with a prefix with one or more delimiters

Folders come in two types:

• Synthetic folders do not exist in Swarm: they are client-side creations from the names of objects in Swarm.

• Pseudo folders are stored in Swarm as an object with no content, whose name ends in the delimiter (“/”). These objects define any placeholder folders as created by users (reserve/this/folder/).

Listing Behaviors

The listing service translates any prefix+delimiter it finds in an object name into a folder listing. It renders data so it exactly matches the object stored in Swarm, other than a few milliseconds to acquire it. Listings are sorted by name, as is the S3 standard. 

Delimiters: It uses a globally configured delimiter (defaults to slash “/”) in Swarm (the storage setting search.pathDelimiter); different characters can be used, but performance may be slower (see https://perifery.atlassian.net/wiki/spaces/public/pages/2443811510). SwarmFS converts slashes to backslashes inline during listing transfers if a client expects backslashes to be the directory delimiter. SwarmFS always converts backslashes to forward slashes when communicating with Swarm.

Metadata: Listing returns objects with all metadata. It stores and renders pseudo folders as objects, which may hold usable folder and POSIX metadata. See https://perifery.atlassian.net/wiki/spaces/public/pages/2443821956

Listing Delay: New objects may be missed, or existing objects show up in the results twice, temporarily if native Swarm objects change between fetches. Objects created in Swarm natively (not via SwarmFS) can take up to 5 minutes plus the https://perifery.atlassian.net/wiki/spaces/public/pages/2443814403 Batch Timeout to appear in SwarmFS listings, because they must be indexed by Elasticsearch. For best listing performance, lower the search feed's Batch Timeout to 1 or 0 (recommended). See 

Versions: For versioned buckets, SwarmFS provides all object versions in the listings, or, on client request, returns the current versions and delete markers for the deleted objects. See 

Encoding: SwarmFS supports UTF-8 encoded names, and the REST interface returns UTF-8-encoded results. 

Exclusive Opens: SwarmFS supports exclusive opens of a file (O_EXCL and O_CREATE) but does not support exclusive reopens (EXCLUSIVE4).

How Listings Appear to Users

SwarmFS creates a simulation: it translates each forward slash (/) in any object name in to a traditional directory delimiter to present named object listings as if they are in a traditional file system. SwarmFS then presents a view of objects at the simulated directory level.

Suppose these named objects exist in the bucket “AcmeBucket”:

• By default, when a user requests a listing of the bucket contents, SwarmFS provides a simulated view of the first level of contents within the bucket.

• SwarmFS returns a new listing for a directory each time a user opens a specific directory level within the bucket (such as Sales/Leads/campaigns/).

• A single file is returned if a user requests a listing for a file (such as Sales/Leads/campaigns/addword.xls). 

• SwarmFS creates a new empty object with Directory attribute metadata, which marks it as a pseudo directory if a user creates a new directory (such as AcmeBucket/Sales/Leads/campaigns/2019/Mar/).

• SwarmFS removes it and all files it contains if a user deletes a directory.

• A user cannot move or rename a directory. Instead, they need to create the directory they want and move the files into it.

Simulated directory objects exist to support the expectations of end users and applications: objects can be written to a new virtual directory by including it in the pathname.