FATAL: failed to lock /var/piler/sphinx/tag1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option

[Uwe]

Hi, piler works fine, but i get a lot of error mails

FATAL: failed to open /var/piler/sphinx/delta1.tmp.spl: Permission denied, will not index. Try --rotate option.

or

FATAL: failed to lock /var/piler/sphinx/tag1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option.

my crontab entry /5 * * root LC_ALL=C mpstat | tail -1 | awk '{print $11}' >/var/piler/stat/cpu.stat /5 * * root /usr/bin/indexer --quiet delta1 --rotate && sleep 2 && /usr/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate /15 * * * root /usr/bin/indexer --quiet tag1 --rotate

debian 7

any idea ?

[Uwe]

i changed user root to user piler. i think this was the problem

[Uwe]

Still have a lot of E-Mails with this Message.

FATAL: failed to lock /var/piler/sphinx/tag1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option

i also tried to ignore messages by

/5 * * * piler /usr/bin/indexer --quiet delta1 --rotate && sleep 2 && /usr/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate > /dev/null 2>&1

but i have a lot of these e-mails the last Days... piler collects e-mails and the Message ist not permanent. but often

any idea ?

i also tried to make a script

sudo -u piler /usr/bin/indexer --quiet delta1 --rotate > /dev/null 2>&1 sleep 10 sudo -u piler /usr/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate > /dev/null 2>&1

and changed crontab to execute the script the files in /var/piler/sphinx/ are all owned by piler:piler

[Uwe]

still error mails

[Janos SUTO]

The tag1 index is updated by another cron job. Make sure that you have followed the installation procedure found on the website, because I have doubts since you don't use indexer.*sh scripts. Also check if piler's crontab has an entry for tag1 (and for note1 for the matter).

[Uwe]

Hi, sorry i forgott to tell... i have made the installation and crontab-Entries like i have found on your Manual piler-docs-0.1.20.pdf i have messages by both entries

FATAL: failed to lock /var/piler/sphinx/delta1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option.

FATAL: failed to lock /var/piler/sphinx/tag1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option.

/5 * * * piler LC_ALL=C mpstat | tail -1 | awk '{print $11}' >/var/piler/stat/cpu.stat

/5 * * * piler /usr/bin/indexer --quiet delta1 --rotate && sleep 2 && /usr/bin/indexer --quiet --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate

/10 * * * piler /usr/bin/indexer --quiet tag1 --rotate

30 7 * * * piler php /usr/local/libexec/piler/daily-report.php --webui /var/www/piler

i have testet the commands manual by hand as user piler and had no errors.

[Janos SUTO]

Dear Lord, you have followed 0.1.20 installation manual, despite we are at 1.2.0? This is a sure way for trouble. Show me piler -V output, and please, please, please follow the installation documentation of the used version.

The first task is to figure out the indexer problem. I asked for checking the permission, yet you didn't confirm that the permissions are fine.

[Uwe]

Ooops... embarrassing... where can i get the right PDF ? i search and cannot find...

piler 1.2.0, build 952, Janos SUTO [e-mail]

Build Date: Wed Dec 21 11:55:15 CET 2016 ldd version: ldd (Debian EGLIBC 2.13-38+deb7u10) 2.13 gcc version: gcc version 4.7.2 (Debian 4.7.2-5) OS: Linux catweazle 3.2.0-4-686-pae #1 SMP Debian 3.2.78-1 i686 GNU/Linux Configure command: ./configure --localstatedir=/var --with-database=mysql --enable-starttls --enable-tcpwrappers MySQL client library version: 5.5.47 Extractors: /usr/bin/pdftotext /usr/bin/catdoc /usr/bin/catppt /usr/bin/xls2csv /usr/bin/unrtf /usr/bin/tnef

my new crontab...

5,35 * * piler /usr/local/libexec/piler/indexer.delta.sh 30 2 * * piler /usr/local/libexec/piler/indexer.main.sh /15 * * piler /usr/bin/indexer --quiet tag1 --rotate /15 * * piler /usr/bin/indexer --quiet note1 --rotate /5 * * piler /usr/bin/find /var/www/piler/tmp -type f -name i.* -exec rm -f {} \;

optional: the same report you can see on the health page

30 7 * * * piler /usr/bin/php /usr/local/libexec/piler/daily-report.php --webui /var/www/piler

optional: populate accouting data

30 6 * * * piler /usr/bin/php /usr/local/libexec/piler/generate_stats.php --webui /var/www/piler

optional: purge aged emails

2 0 * * * piler /usr/local/bin/pilerpurge

[Janos SUTO]

Check out the piler website (http://www.mailpiler.org/wiki/current:index), it contains every possible information about piler. Also it would be great if you could do a proof of concept installation following the install docs using a virtual machine or even docker, and see if there are any sphinx issues or not. I assure you it will work as expected - provided that you have followed the installation docs.

[Uwe]

I tried everithing. i checked everithing. sphinxsearch 2.2.11 piler 1.2.0 Debian 7 permission = ok owner = piler:piler i read the install howto. i still have these mails. i tried MAILTO="" in crontab no chance.

/15 * * * piler /usr/bin/indexer --quiet tag1 --rotate > /dev/null 2>&1

/15 * * * piler /usr/bin/indexer --quiet note1 --rotate > /dev/null 2>&1

FATAL: failed to lock /var/piler/sphinx/note1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option.

FATAL: failed to lock /var/piler/sphinx/tag1.tmp.spl: Resource temporarily unavailable, will not index. Try --rotate option.

i have no more idea

[Janos SUTO]

Show me ls -lR /var/piler/sphinx output. Make sure you use bitbucket's formatting feature.

[Janos SUTO]

You still have some deviation from the stock installation. Get rid of the tag1 and note1 cron jobs, and use the jobs that piler ships. Then you'll be fine.

[Uwe]

i dont understand what is wrong ???

i used the crontab that was described in the piler archive

"piler-1.2.0.tar\piler-1.2.0\etc\cron.jobs"

i searched the location of indexer which is in /usr/bin/ on my system

So i used the piler example and changed the location of idexer.

This is my cron entry (/etc/crontab)

• /15 * * * piler /usr/bin/indexer --quiet tag1 --rotate
• /15 * * * piler /usr/bin/indexer --quiet note1 --rotate

This is the Example

• /15 * * * /usr/local/bin/indexer --quiet tag1 --rotate
• /15 * * * /usr/local/bin/indexer --quiet note1 --rotate

[Janos SUTO]

OK, by looking your ls output more carefully, you don't have the missing file. Run "indexer tag1" as user piler, and see if it's created. Make sure you use the proper sphinx.conf file, check indexer.delta.sh script for the sphinx.conf path.

[Uwe]

Which File you ar missing in my listing ?

sudo -u piler indexer tag1 Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'... indexing index 'tag1'... FATAL: failed to lock /var/piler/sphinx/tag1.spl: Resource temporarily unavailable, will not index. Try --rotate option.

sudo -u piler indexer tag1 --rotate Sphinx 2.2.11-id64-release (95ae9a6) Copyright (c) 2001-2016, Andrew Aksyonoff Copyright (c) 2008-2016, Sphinx Technologies Inc (http://sphinxsearch.com)

using config file '/etc/sphinxsearch/sphinx.conf'... indexing index 'tag1'... collected 204 docs, 0.0 MB sorted 0.0 Mhits, 100.0% done total 204 docs, 1212 bytes total 0.005 sec, 214551 bytes/sec, 36112.58 docs/sec total 4 reads, 0.000 sec, 1.1 kb/call avg, 0.0 msec/call avg total 12 writes, 0.000 sec, 0.7 kb/call avg, 0.0 msec/call avg rotating indices: successfully sent SIGHUP to searchd (pid=11081).

• i dont know, why it searches
• using config file '/etc/sphinxsearch/sphinx.conf'...
• but i have always a copy of the original file in there
• in my indexer.delta.sh
• INDEXER="indexer --config /usr/local/etc/piler/sphinx.conf"

[Janos SUTO]

Check if you have the /var/piler/sphinx/tag1.tmp.spl file by now, and if you are still getting the error emails.

[Uwe]

Still the same...

I made a test with "touch /var/piler/sphinx/tag1.tmp.spl" and changed owner to piler:piler

File is there...

Then i made a

sudo -u piler indexer tag1 --rotate

and the File is deleted by the script !

Thats the Reason why we cannot find these File(s) .tmp.spl

For me it looks like the error-Mails are not permanently send but often. Piler collects Mails... but lot of error Mails too.

Is there a chance to send these Mails to /dev/null or send no mails ?

in my crontab the > /dev/null 2>&1 haves no effekt

I will try out pieler on Debian 8 in any weeks.

[Janos SUTO]

Sigh.... You know what? Comment out the tag1 and note1 indexer jobs in piler's crontab, and then you are fine.

The root cause of the problem is that you didn't follow the installation manual. You did some sort of steps to get piler up and running, but at some point you must have deviated from the manual, and now you are facing some issue. I encourage you to setup a test installation, even in docker is fine, and see for yourself that it works properly.

[Uwe]

• You dont believe it.
• I found the Problem by a google search.
• An other User had the same Problem. So i'm not the only one with this Problem.
• i used /etc/crontab - This is the wrong one - The jobs running twice and block himsef
• sudo -u piler crontab -e
• ist the right crontab

[Janos SUTO]

Yes, I was quite confident that you had something wrong at your side. And you are right there are some other users having trouble following the installation procedure. Anyway, I'm glad that it's finally solved.