1. Lior Amar Avatar Lior Amar
  2. Untitled project
  3. fswalker

Wiki

Clone wiki

fswalker / Wiki_Run_fswalk

View History

Running fswalk and generating the files metadata DB

fswalk is the scanner part of the FSWalk project. This program scan a given filesystem tree and produces an sqlite3 database with information about each file in the scanned filesystem tree. This database can be later analyzed using the fsquery tool or directly by accessing the database from any programming language that supports sqlite (there are many).

First, run fswalk --help to obtain the help screen. Below is a list of the major flags of fswalk:

  • --dir DIR - The directory to scan (recursively of course)
  • --output DIR - The output directory to use for storing the resulting database.
  • --dbprefix PREFIX - A prefix to prepend to the database file
  • --delay-milli MILLI - The number of milliseconds to sleep after scanning a predefined number of files.
  • --delay-files NUM - The number of files to scan before performing a delay (see the --delay-mili option)
  • -p - Print a one line progress bar during the scan

Note: fswalk should run with root permissions in order to be able to access all files

If fswalk is to run without root permissions use the --noroot. This will signal fswalk to avoid generating a lock file which requires root permissions.

The following line is an example of using the fswalk: 

> fswalk --dir /usr --output /opt/dbs/fswalk/ --dbprefix USR-SCAN -p

The above example will scan the /usr directory and use /opt/dbs/fswalk as the output dir. The database output file will start with USR-SCAN and a one line progress bar will be displayed during the scan.

Since the scan may be long (sometimes very long) it is better to run fswalk in the background without a controlling terminal, so in case you close the terminal you started the command from fswalk will not exit.

It is also recommended to inject delays when scanning a busy filesystem since the scan may incure additional load on the server. To do that set the --delay-files and the --delay-milli arguments to control the frequency and the length of the delay.

Progress Bar Description

Below is an example of the progress bar produced by fswalk. 

Active : [top-dirs: 1/8] [f: 24000 d: 3765] [depth: 5:12]  [err: p 0 d 0]
Lets cover the information provided, by going over the fields one by one: * First we can see the status which during the scan will be Active and at the end will be Done. * The top-dirs field displays number of directories at the top level of the scan dir. In our example there are 8 such directories. From those 8 directories 1 directory was allready scanned. * Files and directories encountered during the scan. In our example files=24000 and directories=3765. * Depth of filesystem tree. In our example the current depth is 5 and the maximal depth reached so far was 12. * Errors enountered during the scan: the p stands for permissions error (0 in our example) and the d stands for deleted files errors (0 in our example).

Updated