Jamie McCracken (jamiemcc) wrote,
Jamie McCracken

File system notifications without watches

One of the most eagerly anticipated kernel features that desktop search and VFS authors have been crying out for is full recursive watching of the entire file system - something which is not practical with inotify especially with filesystems containing many thousands of directories.

I read on KDE planet that this has been communicated to kernel devs but hey why reinvent the wheel as a technology called fschange has been around for a while now which pretty much meets the bill as far as the kernel side of things goes. The only snag is it exists only as a patch to the kernel and so needs constant updating as new kernels are released

I would love it if some kernel hacker can get this into the mainline kernel and I am prepared to write a user space root daemon which will use the dbus system bus to send out recursive notifications for it (only root can read the output at /proc/fschange and the daemon must make sure any client has necessary read permission to preserve confidentiality).

The daemon I plan to write for it will also log all changes at the directory level to an sqlite database with the timestamp as primary key for fast date based lookups and delete any data older than 30 days to prevent db growing large. That way an indexer merely has to ask for all file change events from a certain timestamp as well as subscribe to a simple dbus based file notification and tell it what directories it wants to watch recursively.

The benefits of this will be virtually instant startup times for indexers and no grinding of the disk recursively seeking out folders to watch and checking their mtime values for changes (not to mention the memory and resource usage of watches in the kernel). Also it will make it more viable to index the entire filesystem and keep it up to date (and do away with the likes of periodic runs of update-db too).
  • Post a new comment


    default userpic

    Your reply will be screened

    Your IP address will be recorded 

    When you submit the form an invisible reCAPTCHA check will be performed.
    You must follow the Privacy Policy and Google Terms of use.