📢 [Public Preview] Accelerating BlobNFS throughput & scale with FUSE for superior efficiency

Azure Blob Storage might be mounted and accessed like an area file system utilizing BlobFuse, which is a FUSE-based driver for the Blob REST API. Clients select BlobFuse for AI/ML, HPC, analytics and backup workloads. It supplies exceptionally excessive throughput together with advantages like native caching and safety integration by way of Azure Entra ID.

For patrons requiring NFS 3.0 protocol assist or POSIX compliance, Azure Blob Storage additionally natively helps NFSv3 (aka BlobNFS). It permits Azure Blob storage entry for purchasers’ legacy functions with out requiring modifications. BlobNFS is accessed by way of the Linux NFS consumer mixed with our AZNFS mount helper package deal, which streamlines mounting and reliably connecting to Blob Storage’s NFS endpoints. Please word that BlobNFS solely helps entry over a digital community since Azure Entra ID primarily based auth isn’t but out there on NFS 3.0. 

Immediately, we’re excited to announce an replace to AZNFS (3.0) for BlobNFS, which now makes use of the identical libfuse3 library that powers BlobFuse bringing important enhancements in efficiency and scale. The up to date AZNFS for BlobNFS delivers considerably increased throughput, bigger file assist, higher metadata efficiency, and removes consumer group limits, enhancing efficiency for demanding workloads.

• Maximize Digital Machine throughput: AZNFS now helps as much as 256 TCP connections (up from16 in native NFS consumer) permitting throughput to succeed in VM NIC bandwidth (the utmost information switch charge of digital machine’s community interface card) or storage account limits. This advantages HPC workloads by guaranteeing excessive throughput for giant dataset operations. Moreover, a small quantity (4 or fewer) of parallel file reads/writes can now totally saturate the VM NIC bandwidth even for bigger VM sizes.
• Enhanced learn/write velocity: The up to date AZNFS consumer outperforms native NFS consumer for learn and write eventualities. For instance, single file learn/write efficiency is improved by an element of 5x and 3x respectively, which might be helpful for giant database backup duties requiring excessive single file throughput for writing and studying backup recordsdata. Seek advice from the hyperlink for an in depth efficiency comparability.
• Elimination of the consumer’s group restrict: Linux NFS shoppers with an area Id server can move entry permissions for as much as 16 teams of a consumer, limiting useful resource entry for customers belonging to greater than 16 teams. This replace permits FUSE to deal with permission checks, eradicating the 16-group limitation.
• Improved metadata question efficiency: READDIR can question extra listing entries in a single name. The Linux consumer has a restrict of 1MB, whereas the up to date AZNFS can now attain as much as 3 MB. Clients with quite a few recordsdata will expertise faster itemizing and metadata operations with decreased latency. This will likely be useful for workloads like EDA (Digital Design Automation) and HPC (Excessive Efficiency Computing) which regularly contain studying metadata for appreciable variety of recordsdata earlier than deciding on a subset for processing.
• Help for giant file sizes (as much as 5TB): The brand new launch can assist bigger file sizes for sequential write patterns. On account of bigger block sizes doable with AZNFS, customers can create bigger recordsdata as much as the 5TB restrict. With Linux shoppers, beneath finest situations, the max. file sizes have been restricted to ~3TB. CAD instruments producing simulation and checkpoint information recordsdata over 3TB will profit from this enchancment.

The next charts evaluate efficiency between up to date AZNFS and the Native Linux consumer. Please refer the detailed benchmarks for extra particulars.

[Test parameters – VM: Standard D96ds v5, File size: 100GB, Linux NFS is with nconnect =16, Linux kernel 5.x.x ; Test used: DD test]

Notice: The VM helps increased learn throughput than write throughput. For up to date AZNFS, throughput ranging from 4 parallel file learn/write operations is constrained by VM NIC bandwidth, or it could possibly scale increased.

Please register for the preview utilizing this kind.

Please refer the hyperlink for directions on easy methods to set up and use newest model of AZNFS. For any queries or suggestions, please contact us at aznfs@microsoft.com.

References:

What’s BlobFuse? – BlobFuse2 – Azure Storage | Microsoft Be taught

Community File System (NFS) 3.0 protocol assist for Azure Blob Storage

Mount Blob Storage through the use of the Community File System (NFS) 3.0 protocol on Linux

Directions to put in and use newest model of AZNFS · Azure/AZNFS-mount Wiki