Upgrade to sphinx 2.2 -- syslog shows lots of "reindex ... cannot open()"

[Robert Chavers]

I just upgraded from 0.1.25 to 1.1.1 (I'm using Ubuntu), and I also upgraded sphinx to 2.2.

I upgraded the Database by modifying the 0.1.24 to 1.1.1 script. I fixed the sphinx.conf file as recommended.

I ran the following:

screen
cd /tmp
su piler
reindex -a

I can still use the web GUI and see emails and attachments, but my /var/log/syslog is full of errors like this:

May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/ef/07/40000000502bef3a0abdbf8c50276addef07.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/d2/5c/40000000502bef4c2d0b9fb45fbc9ebdd25c.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/17/a2/40000000502bef59051ea17c50eeb01817a2.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/d7/54/40000000502bef7b062aae942f9888e2d754.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/02/c0/40000000502befb60ee53d74c15c294d02c0.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/0b/3e/40000000502beff50edb758ce1599b790b3e.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/cb/d5/40000000502beffc2e5f8b3c21818530cbd5.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/2c/51/40000000502bf00608ab17e40cccb84a2c51.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/a7/cd/40000000502bf00c2c684b34c3b5a80ba7cd.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/88/ef/40000000502bf01106cbd47c1b8af88688ef.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/c5/36/40000000502bf0701bc922e4a049b7cec536.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/1f/38/40000000502bf07b38bb1c0492b4feb41f38.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/24/b0/40000000502bf081261b786489091a4224b0.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/4c/94/40000000502bf0e71b43c6445639d8c74c94.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/7b/95/40000000502bf0f51a775654c84061bc7b95.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/e1/88/40000000502bf0fa102280343a7ae196e188.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/46/13/40000000502bf10010429edcc686f1184613.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/4c/c0/40000000502bf1362c9742a4c9e461204cc0.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/d2/2d/40000000502bf1553107a0ccc79c8ca2d22d.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/c1/a7/40000000502bf18700fe7694c584aca2c1a7.m: cannot open()
May 16 22:00:37 piler reindex[3706]: /var/piler/store/00/502/78/63/40000000502bf190066a0d3cf00246197863.m: cannot open()

...

May 17 09:07:33 piler reindex[3706]: /var/piler/store/00/506/b6/a6/400000005064adb213c512c4b67e9edcb6a6.a2: cannot open()
May 17 09:07:34 piler reindex[3706]: /var/piler/store/00/502/b3/0a/400000005026425911cbcb34788524ecb30a.a1: cannot open()
May 17 09:07:34 piler reindex[3706]: /var/piler/store/00/502/1c/f9/40000000502a9b6f1fd8c36c4a5aef111cf9.a1: cannot open()
May 17 09:07:36 piler reindex[3706]: /var/piler/store/00/502/1a/fd/4000000050293296366c079c0fd19e1b1afd.a1: cannot open()
May 17 09:07:37 piler reindex[3706]: /var/piler/store/00/507/a1/c2/40000000507e8f6e0cb3f5f483ae23cba1c2.a1: cannot open()
May 17 09:07:39 piler reindex[3706]: /var/piler/store/00/502/59/de/40000000502bb1df37c9da4c8439b5c559de.a1: cannot open()
May 17 09:07:49 piler reindex[3706]: /var/piler/store/00/507/b3/34/40000000507f20771be5f5cccf825fe5b334.a1: cannot open()
May 17 09:07:52 piler reindex[3706]: /var/piler/store/00/505/eb/13/40000000505cb694118ef4fcb44e69b9eb13.a1: cannot open()
May 17 09:08:02 piler reindex[3706]: /var/piler/store/00/502/1c/f9/40000000502a9b6f1fd8c36c4a5aef111cf9.a1: cannot open()
May 17 09:08:09 piler reindex[3706]: /var/piler/store/00/507/a0/c3/40000000507612050b5be00435c5b7efa0c3.a1: cannot open()
May 17 09:08:17 piler reindex[3706]: /var/piler/store/00/504/ed/52/40000000504e24b639d79ff411f5ef1fed52.a1: cannot open()

Did I do something wrong during the upgrade? I notice that other emails are reindexing fine, because I can see reinder creating files in /tmp/xxxxx.eml that do have content.

Thanks, Rob

[Janos SUTO]

You must have started with the older path format. Make sure you add --enable-compat-layout to the configure options to enable support for old style format as well.

[Robert Chavers]

Good to know. I will try it. My reindex is still running; almost 3 years and 190GB of emails!

Is it safe to stop the reindex now and reconfigure then run reindex again, or should I just wait until the current reindex is finished?

[Janos SUTO]

Yes, it's safe to stop it. You can start over later.

[Robert Chavers]

OK, so I stopped piler and sphinx. Then I recompiled, and now I'm reindexing. So far, so good. There are no more cannot open messages.

Thanks again for the help!