Filesystem

This document presents how files are stored in NightOS.

• Partitions
• Identifiers and limitations
  • Filesystem unique identifier
  • Element unique identifier
  • Temporary FEID
  • Paths' size limit
• Filenames
• Permissions
• Symbolic links
  • Concept
  • Cyclic symlinks
• Flows
  • Concept
  • Creating a flow
  • Connecting to a flow
• Structure
  • Notes

Partitions

NightOS uses the Btrfs filesystem for the main storage due to its robustness, performance, and features (e.g. snapshots).

Three partitions are used to store the data:

• One FAT32 partition for BOOT1 ;
• One FAT32 partition for BOOT2 (slot 1) ;
• One FAT32 partition for BOOT2 (slot 2) ;
• One Btrfs partition for users' data (/etc except /etc/sys, /apps and /home)

Identifiers and limitations

Filesystem unique identifier

Each existing filesystem gets a unique identifier called the Filesystem Iddentifier (FSID).

It is unique across all filesystems, and consistent across reboots.

Element unique identifier

Each filesystem item has an 8-byte identifier, called the Filesystem Element Identifier (FEID).

It is only unique in the filesystem the item is stored in.

Note that some filesystems may not support this feature. For those, the FEID will be derived from the item's split path, which means it will still be unique in the filesystem but will not be resilient to file moving or another file replacing the previous one.

Temporary FEID

A temporary FEID can be created to grant access to a resource to an application without giving actual access to the original item itself. A temporary FEID only lives in memory, and refers an existing filesystem item. If the original item is deleted, the temporary FEID is deleted too.

Paths' size limit

Paths' length is encoded on 2 bytes, allowing up to 65 534 characters plus the NULL character.

This limit exists to avoid too costly string copies on path manipulation, while being long enough for the very large majority of use cases.

Filenames

For naming conventions, see the related specification.

Permissions

Permissions on individual items is achieved through storage permissions maps.

Symbolic links

Symbolic links, abbreviated symlinks, are files that point to another location.

Concept

A symlink points to a specific item: file, folder, device, anything. It's just not a shortcut, though, as the symlink will still work if its target is moved.

When a symlink is accessed, the system will transparently access its target item instead.

When a symlink is removed, it does not affect the original target. Also, any number of symlinks can target the same item, and symlinks can target other symlinks to. When accessing a symlink, if its target item is a symlink itself, the latter's target will be accessed instead, and so on, until we do not encounter a symlink anymore.

This can be explicitly disabled when interacting with the filesystem, or limited to a specific number of children.

Also, symbolic links may point to a location on another storage.

Cyclic symlinks

Given the following situation:

1. We create a symlink A which points to a random file
2. We create a symlink B which points to A
3. We update the target of A to be B

When we will try to access A, the system will access B, then A, then B, and so on. This is called a cyclic symlink chain. In such case, the chain is reduced to the minimum (for instance, if we had C pointing to A, the minimum chain would not be C A B but just A B), and marked as erroneous. The process that tried to access the symlink will receive a specific error code to indicate a cyclic symlink chain was encountered.

Flows

Flows are a simple and efficient way for shell scripts to allow manipulating asynchronous streams of data.

Concept

A flow is a file without extension, located in the /fl directory, that can either send data to reader processes (read-only) or receive data from writer processes (write-only).

To understand the concept better, here is the list of native flow files that are always available:

Processes are based on pipes.

Creating a flow

When a process wants to create a flow, it follows the following procedure:

1. The process asks the sys::fs service to create a flow
2. The service creates the related flow file in /fl
3. When a process reads from the (readable) flow file, all data is continuely retrieved from the creator's SC (until the flow is closed)
4. When a process writes to the (writable) flow file, all data is continuely written to the creator's RC (the flow is not closed after that though)
5. When the creator closes its SC/RC, the IPC channels duo is closed and the flow file is removed

Connecting to a flow

When a process wants to read from or write to a file, it first asks the sys::fs service to connect to this file. If accepted, it receives a SC or RC to interact with the flow.

Structure

Below is the file tree indicating how elements are organized in NightOS.

NOTE: <F> indicates the item is a file.

/
├── app                            Interactables available to all users
│   └── <appname>                  An application's folder (NOTE: one sub-folder per version for libraries)
│       ├── content                Application's program (executables, static resources, ...)
│       ├── crashsaves             Application's crash saves
│       ├── data                   Application's data (e.g. database)
│       ├── packages               Application's packages (original package + update packages)
│       └── sandboxes              Application's sandboxes
├── dev                            Connected devices
│   ├── cam                        Cameras
│   ├── bst                        Basic storage devices (SD cards, USB keys, ...)
│   ├── etc                        Uncategorized devices
│   ├── mic                        Microphones
│   ├── net                        Network adapters (Ethernet adapter, WiFi card, ...)
│   ├── snd                        Sound-related output devices (Sound card, DAC, ...)
│   ├── sst                        Sensitive storage devices (Hard drives, SSDs, ...)
│   └── wrl                        Other supported wireless devices (Bluetooth adapter, ...)
├── etc                            Mutable data folder
│   ├── env   <F>                  Environment variables
│   ├── hosts <F>                  Hosts overriding (e.g. 'localhost')
│   ├── logs                       Log files
│   |   └── upe <F>                History of UPE requests (1)
│   ├── public                     Public data, readable and writable by everyone
│   └── sys (4)                    System's mutable data - available to system only
│       ├── registry    <F>        System's registry
│       ├── awake       <F>        System's shutdown indicator to detect if there was an error during last shutdown
│       ├── integrity              Integrity data used during the boot process (2)
|       |   ├── signkey <F>        Key used for signing system files
|       |   └── signs   <F>        System file signatures
│       ├── enckey      <F>        Global storage's encryption key (3)
│       └── users       <F>        User profiles and groups
├── fl                             Flow files
├── home                           Users' data
│   └── <user>                     A specific user's data
│       ├── apps                   User's applications (same structure as for `/apps`)
│       ├── appdata                User's applications persistent data (not removed when the application is uninstalled)
│       ├── desktop                User's files appearing on the desktop
│       ├── documents              User's documents
│       ├── downloads              User's downloads
│       ├── music                  User's music files
│       ├── pictures               User's pictures
│       ├── videos                 User's videos
│       └── trash                  User's trash
├── mnt                            Mounted storages
│   └── root                       Soft link to `/`
├── sys (4)                        System - immutable outside of installation, repair processes and updates
│   ├── apps                       System applications
│   ├── boot                   (5) System's boot program (BOOT2)
│   ├── langs                      Translation files
│   ├── old                        Old versions of the system, used during the repair process (compressed archives)
│   ├── backup                     Copy of the last system version (compressed archive)
│   ├── kernel                     Custom micro-kernel
│   └── valid   <F>                A file that just contains "ValidMasterKey" to test if the provided master key is valid at startup
├── tmp                            Temporary folder (cleaned during shutdown)
│   └── <user>                     Temporary folder for a specific user

Links:

• (1) UPE requests
• (2) Used for integrity checking
• (3) Global storage's encryption key
• (4) Not stored in the main data partition but in BOOT2's partition
• (5) Raw boot program (without header), represented by BOOT2

Notes

Globally installed applications (located in /apps) can store user-specific data in /home/[user]/appdata/[appname], which will only be made of the data, crashsaves and sandboxes folder.