# Using ASMFS with NFS and fstab One of the most interesting use cases for [ASMFS](https://github.com/usrecnik/asmfs/) seems to be the ability to list/read archivelogs on a remote (physical standby?) server as if they were normal local files. A quick mental image of how this can be done is represented by the following image: ![asmfs-nfs.png](https://cdn.hashnode.com/uploads/covers/66cc6fade8f3259687b3680a/9840acad-c79d-4966-8c2a-b49fccc72400.png) This tutorial doesn't only focus on said functionality; it also describes how to set all this up so that everything mounts automatically on boot. ASMFS supports two modes of operation: one (the default) is by read-only access to [raw ASM disks](https://blog.srecnik.info/reading-oracle-asm-files-directly-using-xkffxp) , while the other (which is now considered kind of obsolete due to [various quirks](https://blog.srecnik.info/asmfs-and-dbmsdiskgroupread)) uses `dbms_diskgroup`. We'll be using raw access in this post. ## Primary Site First, let's focus on what needs to be done on the *primary* site - that is, the machine where the Oracle ASM instance is running. We'll be mounting to `/mnt/asm`. I probably shouldn't need to say that this folder must exist and have the correct ownership (see `setuid=` option later on). ### Prerequisites (allow\_other) Even though ASMFS will be reading raw ASM disks, it still needs access to metadata, mainly available through performance views such as `v$asm_file` and `v$asm_alias`. It could theoretically run as any user: either as a member of the `OSASM` group (e.g. `grid`) and thus use OS-based authentication (think of `sqlplus / as sysasm` type of login) or e.g. under `root`, which would normally require specifying a username and password - it works, but tends to be a bit more tedious. In either case, that user also needs read access to the raw disk devices. So, I believe it makes the most sense to simply run as the `grid` user (or whichever user your Oracle ASM is running under). But that creates a tiny problem; by default the ASMFS mountpoint is accessible only by the user who mounted it - `grid`. Here's an example of trying to access such a mountpoint as `root`: ```plaintext [root@us-db01 mnt]# ls -l ls: cannot access 'asm': Permission denied total 8 d????????? ? ? ? ? ? asm ... ``` And later, the NFS server (which generally runs as `root`, not as `grid`) will need to access it. It's easy to fix this, simply add `-o allow_other` to the mount options (or ASMFS cmdline): ```plaintext [grid@us-db01 mnt]$ /opt/asmfs/asmfs /mnt/asm/ -o allow_other ``` By default, FUSE only allows using this `allow_other` if you're running ASMFS as `root`. Which we're not doing. So, we need to either fiddle with a username/password and run as root or allow a non-root user to specify `allow_other`, which is done by uncommenting the following line in `/etc/fuse.conf`: ```plaintext user_allow_other ``` Also note that `allow_other` does not change file ownership or permission bits; it merely removes FUSE’s mount-owner-only restriction. Keep in mind that `user_allow_other` is a system-wide policy: it permits any non-root user to specify `allow_other` for a FUSE filesystem they mount. ### ASMFS (fstab) Now, we're ready to mount `asmfs`. But since we want this to be done on boot, we'll add the following line to `/etc/fstab`: ```plaintext asmfs /mnt/asm fuse3.asmfs ro,nofail,setuid=grid,allow_other,x-systemd.requires=asmfs-wait-for-asm.service,x-systemd.mount-timeout=300 0 0 ``` I know, there's a lot going on. Let's go step-by-step. The first field is simply a constant `asmfs`. In most filesystems, this field tells the source (e.g. `/dev/sdb` or `my-server:/my/folder`) from where we're mounting. But, since only one Oracle ASM instance is supported on a given machine and since the format requires this field, we simply use a constant (which is actually ignored in the ASMFS source). Second field is where we want to mount. No surprises there. The third field actually determines that we want to use ASMFS. ASMFS is distributed as `.rpm` package which installs to `/opt/asmfs`. In an automatic post-install phase, it also creates `/usr/bin/fuse3.asmfs` symlink, which points to `/opt/asmfs/asmfs`. So, `fuse3.asmfs` actually refers to this particular binary. Next, a set of options is presented. Let’s go through them, loosely ordered by the role they play in the overall setup. * `ro` - this explicitly mounts ASMFS as read-only. ASMFS does not support writes anyway, but being explicit both documents the intent and provides an additional kernel-enforced safeguard. * `nofail` - tells the boot system that failure to mount this entry must ***not*** make the mount a required part of boot. The mount is still attempted, but boot can continue without waiting for it and won’t fail merely because ASMFS could not be mounted. This is important because the ASM instance may take a significant amount of time to become ready. This does not mean that nothing waits for the mount: later, we’ll explicitly make NFS wait for it. * `setuid=grid` - This option is understood by FUSE 3 mount helper, which is instructed to switch to this user (`grid`) ***before*** launching the FUSE filesystem process (that being `/usr/bin/fuse3.asmfs -> /opt/asmfs/asmfs`). * `allow_other` - we've already talked about this one in the previous section. * `x-systemd.requires=asmfs-wait-for-asm.service` - this one is the key; this service will wait up to 30 minutes and exit successfully only when the Oracle ASM instance is ready to accept connections (otherwise, it fails after the timeout). Because this option adds both `Requires=` and `After=`, systemd starts the mount only after the service has completed successfully. It's implemented as a bash script that tries to run a simple select using `sqlplus` as the correct user. This service is installed automatically when you install asmfs `.rpm` package. * `x-systemd.mount-timeout=300` - Once the mount operation actually starts, it has 5 minutes (300 seconds) to finish, otherwise, mount is considered to be failed. This timing starts only when the previously mentioned service finishes successfully. Finally, `0 0` at the end of this fstab line excludes ASMFS from dump and boot-time fsck; neither applies to this FUSE file system. When things go wrong, you can inspect the wait service and generated mount unit: ```plaintext systemctl status mnt-asm.mount systemctl status asmfs-wait-for-asm.service ``` ### NFS In `/etc/exports` we can export it like this: ```plaintext /mnt/asm 172.19.1.12(ro,fsid=101,mountpoint) ``` Regarding the options, * `ro` - makes the NFS export read-only. * `fsid=101` - gives this exported filesystem a stable NFS identifier. `101` must be unique among exports on this server. * `mountpoint` - makes sure that `/mnt/asm` is exported only when that path is an actual mount point, preventing accidental export of the underlying directory. It does not wait for `/mnt/asm` to become mounted; it simply omits the export if the path is not a mount point when the exports table is processed. Such a wait can be achieved by editing `nfs-server.service` like this: ```plaintext # systemctl edit nfs-server.service ``` ```plaintext [Unit] RequiresMountsFor=/mnt/asm ``` `RequiresMountsFor=` adds both `Requires=` and `After=` dependencies on `mnt-asm.mount`, so NFS waits for ASMFS even though the mount itself uses `nofail`. Note that if the 30-minute wait expires and NFS startup fails, mounting ASMFS successfully later on does not automatically retry `nfs-server.service`; it must be started again. ## Standby Site Here, we just need to mount NFS: ```plaintext 172.19.1.10:/mnt/asm /mnt/asm nfs ro,softerr,timeo=100,retrans=2,nofail,x-systemd.automount,x-systemd.mount-timeout=30s,x-systemd.idle-timeout=15min 0 0 ``` * `ro` - keeps the client-side mount read-only as well. * `x-systemd.automount` - it means that on boot, rather than mounting the fs, the kernel will set a trigger that will start the mount on first access to this mountpoint. So, until first access, this fs is not mounted. * `x-systemd.mount-timeout=30s` - this controls how long to wait until mount command finishes. If not mounted in 30s, then the process accessing such mount receives an error. * `x-systemd.idle-timeout=15min` - if mountpoint is idle for 15 minutes, systemd automatically attempts to umount it. But next access mounts it again due to automount trigger. * `nofail` - makes that entry non-fatal to boot. Even though, as explained, boot does not even attempt to mount: it just creates an automount trigger which is triggered on first access. * `softerr,timeo=100,retrans=2` \- controls NFS requests after the filesystem has mounted. `timeo=100` sets the initial RPC timeout to 10 seconds, while `retrans=2` allows two retries. If those attempts fail, `softerr` returns `ETIMEDOUT` instead of retrying indefinitely. So, events follow like this: First access (by any tool, e.g. ls, cp, rsync, anything) ↓ Attempt to mount NFS (for up to 30 seconds) ↓ If mounted, attempt to unmount after 15 minutes without activity ↓ If unmounted, the next access mounts it again Finally, just a quick note regarding given fstab line. If you just do: ```plaintext systemctl daemon-reload ``` instead of reboot-ing, the generated `mnt-asm.automount` will be inactive (not started). So, you need either reboot or start it explicitly: ```plaintext systemctl start mnt-asm.automount ``` ## Final Thoughts And that’s it: ASMFS starts only after Oracle ASM is ready, NFS waits for it on the primary, and the standby mounts the share only when it actually needs it. This setup requires ASMFS ver `2.1.3` or later and was tested with Oracle AI Database 26ai on OCI Base Database Service. Feel free to grab .rpm in [GitHub Releases](https://github.com/usrecnik/asmfs/releases).