File system can be unmounted before all data will be saved to ZIP file.

[Former user]

Originally reported on Google Code with ID 7

Commands log:

$ mkdir test
$ ./fuse-zip test.zip test
$ tar xjf /var/tmp/emerge-webrsync/portage-20080627.tar.bz2 -C test
$ top -p 7053
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND      
 7053 daraku    20   0  534m 520m  812 S  0.0 51.9   0:44.86 fuse-zip
$ fusermount -u testumount:
/home/daraku/Programming/c++/fuse-zip-0.2.3/test: device is busy
umount: /home/daraku/Programming/c++/fuse-zip-0.2.3/test: device is busy

//after a few seconds
$ fusermount -u test
$ top -p 7053
  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND      
 7053 daraku    20   0  534m 521m  824 R 94.6 52.0   1:15.36 fuse-zip
$ ls -l|grep zip
102400 -rwxr-xr-x 1 daraku users 101626 Июл  4 19:14 fuse-zip
  4096 -rw-r--r-- 1 daraku users   2120 Июн 26 16:12 fuse-zip.1
 16384 -rw-r--r-- 1 daraku users  14850 Июн 26 16:12 fuse-zip.cpp
 81920 -rw-r--r-- 1 daraku users  81904 Июл  4 19:14 fuse-zip.o
$ uname -a
Linux localhost 2.6.24-gentoo-r3 #4 PREEMPT Thu Apr 10 19:11:19 EEST 2008
x86_64 AMD Athlon(tm) 64 Processor 3000+ AuthenticAMD GNU/Linux

So, I'm unmount file system, but test.zip file wasn't been created.

Used libraries:
libzip 0.8
fuse 2.7.3
fuse-zip 0.2.3

P.S. I don't have /var/log/user.log file.

Reported by daraku.aka.datenshi on 2008-07-04 16:45:38

[Former user]

``` Be patient and wait for fuse-zip process finished after calling fusermount -u.

It is a problem in FUSE because fusermount does not support "synchronized" unmounting. ```

Reported by `alexander.galanin` on 2008-07-05 01:03:27

[Former user]

``` Does this a known problem in fusermount? When this issue planed to be fixed? (I can't fide the issuer tracker for fusermount)

The second problem of this issue is a memory usage. Because I'm trying to save about 600MB of data to fuse-zip file system and file system doesn't save data on-line, so all 600MB stored into memory :( Shall I open a new issue for fuse-zip, or this is issue of libzip/fuse? ```

Reported by `daraku.aka.datenshi` on 2008-07-05 04:20:21

[Former user]

```

Does this a known problem in fusermount? When this issue planed to be fixed? (I can't

fide the issuer tracker for fusermount)

I will report this issue to fuse mailing list in

The second problem of this issue is a memory usage.

I know about this issue. ```

Reported by `alexander.galanin` on 2008-07-05 12:02:00

[Former user]

``` Issue 19 has been merged into this issue. ```

Reported by `alexander.galanin` on 2009-09-07 18:26:42

[Alexander Galanin]

More info:

• discussion in fuse-devel mailing list: umount(8) returning before destroy completes

• kernel patch by Maxim Patlasov

[Alexander Galanin]

Issue in libfuse's tracker: https://github.com/libfuse/libfuse/issues/1

[Chris Graham]

I haven’t checked the code behavior, but do you use a write lock on the zip path? Seems to me you could create an empty file, lock it, until the actual .zip data is written in. That makes things better for a number of reasons.

[Alexander Galanin]

Separate lock file has no effect on end-user, since they may ignore it and copy a zip file before unmounting is finished. Mandatory file locking could protect a zip file from reading but this feature marked as unreliable (see fcntl(2) section BUGS) and scheduled for deletion.

[Chris Graham]

That’s interesting about mandatory locks having race condition issues on Linux, I didn’t know that. But Linux is really going to remove the feature? Surely not as it is a part of POSIX (I would think) and even if it’s not perfect, it’s better than no lock.

[Sobel-Sorenson, Chandler - (scar)]

Anything more that could be added to fixing this?  Or workarounds?  I’m needing to implement this on a system with low free disk, and low free memory.  I didn’t see it mentioned, but can this issue can cause data corruption?  One problem is solved but at least 2 others created, it seems.

This looks like a disaster waiting to happen, with regards to memory usage.  Currently testing and 93% memory used, 72% swap used, and the main apps the system needs to run are stuck swapping already.  Performance of the system is significantly reduced.  Will oom-killer start getting involved?  Since fuse-zip is running in userspace, will it get killed?  Does this create an avenue for potential data corruption as well?

[Alexander Galanin]

Hi Chandler,

You may start fuse-zip process with -f to run it in foreground and wait until it exit after unmounting. Once the filesystem process is finished, the archive file are finally ready for use.

Yes, zuse-zip can be killed by OOM like any another userspace application. But the risk of data corruption is very small because of nature of libzip that actually implements ZIP file operations: on unmount the library creates a new file, copies all the data ot it and then replaces the old archive with the new one. So the only place for race is the rename at the end of file saving process.

[Sobel-Sorenson, Chandler - (scar)]

I put them into systemd user mount units and just need to wait for “File system unmounted” message.  ℰ.ℊ.:

Jan 16 11:55:41 Hae systemd[2155]: Unmounting home-user-Archives.mount - Mount Archives.zip...
Jan 16 11:55:41 Hae systemd[2155]: home-user-Archives.mount: Unit process 199811 (fuse-zip) remains running after unit stopped.
Jan 16 11:55:41 Hae systemd[2155]: Unmounted home-user-Archives.mount - Mount Archives.zip.
Jan 16 11:55:41 Hae systemd[2155]: home-user-Archives.mount: Consumed 17min 55.342s CPU time.
Jan 16 12:03:18 Hae fuse-zip[199811]: File system unmounted

Trick to getting systemd to work was Type=fuse.fuse-zip in the [Mount] section!

However,

I was more concerned with fuse-zip getting killed while it’s running!  Its memory usage increases as more data is written, so what happens when amount of data written exceeds free memory+swap??

[Alexander Galanin]

Hi Chandler,

Could you share your systemd unit file? I want to try it and confirm that it really wait until the file system process finish.

I was more concerned with fuse-zip getting killed while it’s running!  Its memory usage increases as more data is written, so what happens when amount of data written exceeds free memory+swap??

It will be killed by OOM killer. See also #62 and #98.