Using ASMFS with NFS and fstab

One of the most interesting use cases for ASMFS seems to be the ability to list/read archivelogs on a remote (physical standby?) server as if they were normal local files.

A quick mental image of how this can be done is represented by the following image:

This tutorial doesn't only focus on said functionality; it also describes how to set all this up so that everything mounts automatically on boot.

ASMFS supports two modes of operation: one (the default) is by read-only access to raw ASM disks , while the other (which is now considered kind of obsolete due to various quirks) uses dbms_diskgroup. We'll be using raw access in this post.

Primary Site

First, let's focus on what needs to be done on the primary site - that is, the machine where the Oracle ASM instance is running.

We'll be mounting to /mnt/asm. I probably shouldn't need to say that this folder must exist and have the correct ownership (see setuid= option later on).

Prerequisites (allow_other)

Even though ASMFS will be reading raw ASM disks, it still needs access to metadata, mainly available through performance views such as v$asm_file and v$asm_alias. It could theoretically run as any user: either as a member of the OSASM group (e.g. grid) and thus use OS-based authentication (think of sqlplus / as sysasm type of login) or e.g. under root, which would normally require specifying a username and password - it works, but tends to be a bit more tedious. In either case, that user also needs read access to the raw disk devices.

So, I believe it makes the most sense to simply run as the grid user (or whichever user your Oracle ASM is running under). But that creates a tiny problem; by default the ASMFS mountpoint is accessible only by the user who mounted it - grid. Here's an example of trying to access such a mountpoint as root:

[root@us-db01 mnt]# ls -l
ls: cannot access 'asm': Permission denied
total 8
d????????? ? ?    ?           ?            ? asm
...

And later, the NFS server (which generally runs as root, not as grid) will need to access it. It's easy to fix this, simply add -o allow_other to the mount options (or ASMFS cmdline):

[grid@us-db01 mnt]$ /opt/asmfs/asmfs /mnt/asm/ -o allow_other

By default, FUSE only allows using this allow_other if you're running ASMFS as root. Which we're not doing. So, we need to either fiddle with a username/password and run as root or allow a non-root user to specify allow_other, which is done by uncommenting the following line in /etc/fuse.conf:

user_allow_other

Also note that allow_other does not change file ownership or permission bits; it merely removes FUSE’s mount-owner-only restriction. Keep in mind that user_allow_other is a system-wide policy: it permits any non-root user to specify allow_other for a FUSE filesystem they mount.

ASMFS (fstab)

Now, we're ready to mount asmfs. But since we want this to be done on boot, we'll add the following line to /etc/fstab:

asmfs  /mnt/asm  fuse3.asmfs ro,nofail,setuid=grid,allow_other,x-systemd.requires=asmfs-wait-for-asm.service,x-systemd.mount-timeout=300  0 0

I know, there's a lot going on. Let's go step-by-step.

The first field is simply a constant asmfs. In most filesystems, this field tells the source (e.g. /dev/sdb or my-server:/my/folder) from where we're mounting. But, since only one Oracle ASM instance is supported on a given machine and since the format requires this field, we simply use a constant (which is actually ignored in the ASMFS source).

Second field is where we want to mount. No surprises there.

The third field actually determines that we want to use ASMFS. ASMFS is distributed as .rpm package which installs to /opt/asmfs. In an automatic post-install phase, it also creates /usr/bin/fuse3.asmfs symlink, which points to /opt/asmfs/asmfs. So, fuse3.asmfs actually refers to this particular binary.

Next, a set of options is presented. Let’s go through them, loosely ordered by the role they play in the overall setup.

• ro - this explicitly mounts ASMFS as read-only. ASMFS does not support writes anyway, but being explicit both documents the intent and provides an additional kernel-enforced safeguard.

• nofail - tells the boot system that failure to mount this entry must not make the mount a required part of boot. The mount is still attempted, but boot can continue without waiting for it and won’t fail merely because ASMFS could not be mounted. This is important because the ASM instance may take a significant amount of time to become ready. This does not mean that nothing waits for the mount: later, we’ll explicitly make NFS wait for it.

• setuid=grid - This option is understood by FUSE 3 mount helper, which is instructed to switch to this user (grid) before launching the FUSE filesystem process (that being /usr/bin/fuse3.asmfs -> /opt/asmfs/asmfs).

• allow_other - we've already talked about this one in the previous section.

• x-systemd.requires=asmfs-wait-for-asm.service - this one is the key; this service will wait up to 30 minutes and exit successfully only when the Oracle ASM instance is ready to accept connections (otherwise, it fails after the timeout). Because this option adds both Requires= and After=, systemd starts the mount only after the service has completed successfully. It's implemented as a bash script that tries to run a simple select using sqlplus as the correct user. This service is installed automatically when you install asmfs .rpm package.

• x-systemd.mount-timeout=300 - Once the mount operation actually starts, it has 5 minutes (300 seconds) to finish, otherwise, mount is considered to be failed. This timing starts only when the previously mentioned service finishes successfully.

Finally, 0 0 at the end of this fstab line excludes ASMFS from dump and boot-time fsck; neither applies to this FUSE file system.

When things go wrong, you can inspect the wait service and generated mount unit:

systemctl status mnt-asm.mount
systemctl status asmfs-wait-for-asm.service

NFS

In /etc/exports we can export it like this:

/mnt/asm  172.19.1.12(ro,fsid=101,mountpoint)

Regarding the options,

• ro - makes the NFS export read-only.

• fsid=101 - gives this exported filesystem a stable NFS identifier. 101 must be unique among exports on this server.

• mountpoint - makes sure that /mnt/asm is exported only when that path is an actual mount point, preventing accidental export of the underlying directory. It does not wait for /mnt/asm to become mounted; it simply omits the export if the path is not a mount point when the exports table is processed. Such a wait can be achieved by editing nfs-server.service like this:

# systemctl edit nfs-server.service

[Unit]
RequiresMountsFor=/mnt/asm

RequiresMountsFor= adds both Requires= and After= dependencies on mnt-asm.mount, so NFS waits for ASMFS even though the mount itself uses nofail.

Note that if the 30-minute wait expires and NFS startup fails, mounting ASMFS successfully later on does not automatically retry nfs-server.service; it must be started again.

Standby Site

Here, we just need to mount NFS:

172.19.1.10:/mnt/asm /mnt/asm nfs ro,softerr,timeo=100,retrans=2,nofail,x-systemd.automount,x-systemd.mount-timeout=30s,x-systemd.idle-timeout=15min 0 0

• ro - keeps the client-side mount read-only as well.

• x-systemd.automount - it means that on boot, rather than mounting the fs, the kernel will set a trigger that will start the mount on first access to this mountpoint. So, until first access, this fs is not mounted.

• x-systemd.mount-timeout=30s - this controls how long to wait until mount command finishes. If not mounted in 30s, then the process accessing such mount receives an error.

• x-systemd.idle-timeout=15min - if mountpoint is idle for 15 minutes, systemd automatically attempts to umount it. But next access mounts it again due to automount trigger.

• nofail - makes that entry non-fatal to boot. Even though, as explained, boot does not even attempt to mount: it just creates an automount trigger which is triggered on first access.

• softerr,timeo=100,retrans=2 - controls NFS requests after the filesystem has mounted. timeo=100 sets the initial RPC timeout to 10 seconds, while retrans=2 allows two retries. If those attempts fail, softerr returns ETIMEDOUT instead of retrying indefinitely.

So, events follow like this:

First access (by any tool, e.g. ls, cp, rsync, anything) ↓ Attempt to mount NFS (for up to 30 seconds) ↓ If mounted, attempt to unmount after 15 minutes without activity ↓ If unmounted, the next access mounts it again

Finally, just a quick note regarding given fstab line. If you just do:

systemctl daemon-reload

instead of reboot-ing, the generated mnt-asm.automount will be inactive (not started). So, you need either reboot or start it explicitly:

systemctl start mnt-asm.automount

Final Thoughts

And that’s it: ASMFS starts only after Oracle ASM is ready, NFS waits for it on the primary, and the standby mounts the share only when it actually needs it.

This setup requires ASMFS ver 2.1.3 or later and was tested with Oracle AI Database 26ai on OCI Base Database Service. Feel free to grab .rpm in GitHub Releases.