Preferences

Mounting WebDAV -- if you are in a situation, where you have to do it (e.g. own^W^W^Wnextcloud) is such an adventure. Everything - mac, win, linux - supports WebDAV. You mount and it works! Then you notice HOW it works: files are downloaded in full before program can access them, some operations are super slow, some fail or time out, plaintext credentials in mysterious places...

I heard DeltaV is very advanced, and Subversion supported it. I'm afraid to ask.


I'm using the nextcloud app on my android, and for my Linux systems I mount WebDAV using rclone, with VFS cache mode set to FULL. This way I can: 1. Have the file structure etc synced to local without downloading the files 2. Have it fetch files automatically when I try to read them. Also supports range requests, so if I want to play a video, it sort of streams it, no need to wait for download. 3. If a file has been accessed locally, it's going to be cached for a while, so even if I'm offline, I can still access the cached version without having to verify that it's the latest. If I'm online, then it will verify if it's the latest version.

Overall, this has worked great for me, but it did take me a while before I set it up correctly. Now I have a cache of files I use, and the rest of the stuff that I just keep there for backup or hogging purposes doesn't take disk space and stays in the cloud until I sync it.

Sine you are mounting and not syncing the files, what happens when you edit a file offline? And what if on another offline device the file is also edited?
Fair question. Conflicts happen, which I'm fine with.

Realistically speaking, most files I have in my cloud are read-only. The most common file that I read-write on multiple devices is my keepass file, which supports conflict resolution (by merging changes) in clients.

Also used to happen when I tried editing some markdown notes using obsidian on PC, and then using text editor (or maybe obsidian again?) on android, but I eventually sort of gave up on that use-case. Editing my notes from my phone is sort of inconvenient anyway, so I mostly just create new short notes that I can later edit into some larger note, but honestly can't remember the last time this happened.

But yes, if not careful, you could run into your laptop overwriting the file when it comes online. In my case, it doesn't really happen, and when it does, Nextcloud will have the "overwritten version" saved, so I can always check what was overwritten and manually merge the changes.

P.S. If anyone wants to set this up, here's my nixos config for the service, feel free to comment on it:

  # don't forget to run `rclone config` beforehand
  # to create the "nextcloud:" remote
  # some day I may do this declaratively, but not today
  systemd.services.rclone-nextcloud-mount = {
    # Ensure the service starts after the network is up
    wantedBy = [ "multi-user.target" ];
    after = [ "network-online.target" ];
    requires = [ "network-online.target" ];

    # Service configuration
    serviceConfig = let
      ncDir = "/home/username/nextcloud";
      mountOptions = "--vfs-cache-mode full --dir-cache-time 1w --vfs-cache-max-age 1w";
    in {
      Type = "simple";
      ExecStartPre = "/run/current-system/sw/bin/mkdir -p ${ncDir}"; # Creates folder if didn't exist
      ExecStart = "${pkgs.rclone}/bin/rclone mount ${mountOptions} nextcloud: ${ncDir}"; # Mounts
      ExecStop = "/run/current-system/sw/bin/fusermount -u ${ncDir}"; # Dismounts
      Restart = "on-failure";
      RestartSec = "10s";
      User = "username";
      Group = "users";
      Environment = [ "PATH=/run/wrappers/bin/:$PATH" ];
    };
  };
Windows officialy removed support for WebDAV. It still works, but nothing is guaranteed. It has stupid limitation on file size of 10MB, it can be lifted to 2GB (max signed 32 bit number) in Registry, but it is still not very much in modern world (I wanted to share my medial library via WebDAV and failed due to this limitation). It lose credentials on regular basis, errors are too vague («Wrong credentials» means both mistyped password AND expired server certificate), etc.
It’s also a bit of a disaster from a security perspective.

https://www.thehacker.recipes/ad/movement/mitm-and-coerced-a...

> own^W^W^Wnextcloud

own^H^H^Hnextcloud

or

own^Wnextcloud

You might wanna look into OpenCloud (formerly known as nextcloud-go) [1]. I still use Nextcloud for the uploading of files and the calendar (though I may switch the latter), but I now sync the dir with Immich. Performance-wise a relief. I also swapped Airsonic Advanced (Java) with Navidrome (Go). Same story.

[1] https://github.com/opencloud-eu/opencloud

> but I now sync the dir with Immich

Do you use this for anything other than photos and videos?

Good point, I have to migrate that away and clean it up, but the main user is my wife, not me. I might completely get rid of Nextcloud. I mean, the name is tarnished for my wife.
WebDAV is like the 12V plug in cars.

Its certainly not the optimal design, but it exists in pretty much all cars, so we use it because it's there, and because of it's universal presence, its also hard to replace.

The sad part is, in a world that is increasingly mobile first, and computing devices move in and out of network coverage, HTTP based protocols actually handle frequent disconnects/reconnects much better than something like SMB.

For my personal backup needs, running from my phone, WebDAV is king. S3 would probably be better, protocol wise, but i can't have that in a simple "wrapper" that simply exposes existing files, and WebDAV works perfectly fine for LAN anyway.

Subversion works ok over webdav, it has done it for decades.

Mounting a directory through nfs, smb or ssh and files are downloaded in full before program access them. What you mean? Listing a directory or accessing file properties, like size for example do not need full download.

I am confused, what do you mean? What OS forces you to download whole file over NFS or SMB before serving read()? Even SFTP does support reading and writing at an offset.
If I open a nfs doc with, let's say Libreoffice, will I not download whole file?

On a second thought, I think you are looking at webdav as sysadmins not as developers. Webdav was designed for document authoring, and you cannot author a document, version it, merge other authors changes, track changes without fully controlling resources. Conceptually is much like git needs a local copy.

I can't imagine how to have an editor editing a file and file is changed at any offset at any time by any unknown agent whitouth any type of orchestration.

If you open a file with LibreOffice will read the whole thing regardless of whether or not the file is on NFS or not.

The parent comment was stating that if you use the open(2) system call on a WebDAV mounted filesystem, which doesn't perform any read operation, the entire file will be downloaded locally before that system call completes. This is not true for NFS which has more granular access patterns using the READ operation (e.g., READ3) and file locking operations.

It may be the case that you're using an application that isn't LibreOffice on files that aren't as small as documents -- for example if you wanted to watch a video via a remote filesystem. If that filesystem is WebDAV (davfs2) then before the first piece of metadata can be displayed the entire file would be downloaded locally, versus if it was NFS each 4KiB (or whatever your block size is) chunk would be fetched independently.

Libreoffice will likely download the whole file.

But many others clients won't. In particular, any video player will _not_ download entire file before accessing it. And for images, many viewers start showing image before whole thing is downloaded. And to look at zip files, you don't need the whole thing - just index at the end. And for music, you stream data...

Requiring that file is "downloaded in full before program access them" is a pretty bad degradation in a lot of cases. I've used smb and nfs and sshfs and they all let you read any range of file, and start giving the data immediately, even before the full download.

NFS infamously proxies reads and writes. Obviously there is some caching but that just makes the behavior funner.
Are you saying WebDAV doesn't support range requests?
That's the beauty of working with WebDAV, also captured vividly in the above article -- any particular server/client combination feels no obligation to try and act like some "standards" prescribe, or make use of facilities available.

I might be wrong, but when I last mounted webdav from windows, it did the same dumb thing too.

WebDAV as standard? Supports. This particular combination of client and server? Who knows, good luck.
> Subversion works ok over webdav, it has done it for decades.

Thank you!!!!

Regarding Linux, WebDAV has been partially working/broken in Dolphin/kio since Plasma 5 on KDE. I've found the davfs2 FUSE module to be more reliable.
Sibling comment mentioned rclone, which is enabling piece of software and much better at webdaw than davfs2
Rclone works but is still not very good, either it doesn't allow a lot of the things you'd expect from a filesystem or it'll just consume tremendous amount of disk space while it slowly transfers the files to the remote host (even on a really fast link).
As it happens, I've been deep in the with rclone, webdav, and Nextcloud over the last few days.

It works. It's slow when synching a ton of small files but besides that I haven't run into any unexpected problems.

If I did it again I'd probably use the Nextcloud client in API mode. But this was originally supposed to be a small project... Oh well.

I just imagined implementing webdav as a kernel module and I think I just broke my brain
Here's some prior art for your cursed journey https://github.com/sysprog21/khttpd
Actually - I believe - within Windows 11 - the "WebClient" service is now deprecated (which is what - IIRC, actually implements the WebDAV client protocol so that it works with Windows File Explorer, drive mappings, etc.)...

Played around with WebDAV alot... a long time ago... (Exchange Webstore/Webstorage System, STS/SharePoint early editions)...

This item has no comments currently.

Keyboard Shortcuts

Story Lists

j
Next story
k
Previous story
Shift+j
Last story
Shift+k
First story
o Enter
Go to story URL
c
Go to comments
u
Go to author

Navigation

Shift+t
Go to top stories
Shift+n
Go to new stories
Shift+b
Go to best stories
Shift+a
Go to Ask HN
Shift+s
Go to Show HN

Miscellaneous

?
Show this modal