Folder restrictions not work. Empty search.

[Mateusz Szymborski]

I have imported lot of emails with -m name.surname.mbox -F name.surname flag. If I have $config['ENABLE_FOLDER_RESTRICTIONS'] = 0; In auditor account i see e-mails. If i have $config['ENABLE_FOLDER_RESTRICTIONS'] = 1; And i add access to this folder in user config i don't see any e-mail.

[Janos SUTO]

Try creating a non auditor user, and give him permissions on the given folder.

[Mateusz Szymborski]

Nope. Not working. Without effect.

[Mateusz Szymborski]

And i see were wrong. User see emails which should not.

If i check folder > search > click on email = i see good email.

If i uncheck folder > search > CLICK ON EMAIL = i see bad email

We don't open properly email.

[Mateusz Szymborski]

regular@local have access to 51 folder.

In 51 folder is 2648 emails.

In 52 folder is 37 emails.

If I uncheck all folders he see 2685 emails. I select first email and check it.

mysql> select * from main1,dailydelta1,delta1 where id=237123;
+--------+------------+-------+--------+-------------+
| id     | sent       | size  | folder | attachments |
+--------+------------+-------+--------+-------------+
| 237123 | 1399369066 | 13034 |     52 |           0 |
+--------+------------+-------+--------+-------------+
1 row in set (0.00 sec)

[Janos SUTO]

If you check a folder on the left pane, it means that you want to narrow the search to that given folder. If unchecked (all) then the query searches in all visible (to the user) folders.

If you mean that user X has no right to folder 52, then make sure you logout, then login to reassign folder permissions.

[Mateusz Szymborski]

Now i do "su - piler" > "indexer --all --rotate", and i have back one problem. Without folders i see all in auditor. With folder search is empty.

In /etc/sphinxsearch/sphinx.conf i have sql_attr_uint = folder

Jan 30 16:20:29 e577ed228937 piler-webui[2727]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE       folder IN (1,51) AND  MATCH('') ORDER BY `sent` DESC LIMIT 0,1000 OPTION max_matches=1000' in 0.00 s, 0 hits, 0 total found

[Mateusz Szymborski]

So I have problem. I don't see any email. reindex is not helpful. If we can today connect per teamviewer and try to repair that and accesses to folders i will be happy.

[Janos SUTO]

Please don't just run any command. indexer --all is for install time only to create/reset the sphinx database. After that you have an empty sphinx db, so no wonder that you can't find anything. Now you should run reindex -f 1 -t 1000, to reindex the first 1000 messages, then check the gui.

[Mateusz Szymborski]

I have been reindex all messages, but without sql_attr_uint = folder in sphinx.conf. Now i have it. How to index/reindex to folders?

[Janos SUTO]

That's a problem, because pilerimport writes the folder info to sph_index table only, and it's gone. However perhaps you may fix it by adding -F option to reindex, just as with pilerimport to specify folder info. However you have to know which message belongs to which folder, eg. reindex -f 1 -t 1234 -F aaa, reindex -f 1235 -t 7645 -F bbb, etc.

[Mateusz Szymborski]

I import with pilerimport -m "$var/$(ls -1 $var | grep .mbox)" -F $var; I need to index with folders.

[Janos SUTO]

OK, but I'm talking about reindexing. It also supports the -F option.

[Mateusz Szymborski]

So the easiest way is to reimport all?

[Mateusz Szymborski]

When I import, and after import i want to immediately see these emails, how to index it?

[Janos SUTO]

Look at the piler crontabs.

[Mateusz Szymborski]

Now i have 244754 (24 hours) Processed emails. I import with import.sh:

#!/bin/bash

if [ "$#" -eq 0 ]; 
then 
    echo "Usage: ./`basename $0` d_name.surname d_name.surname2 d_name.surname3 ..."; 
else
    for var in "$@"; do
        echo $var
        zip=$var/$(ls -1 $var | grep .zip)
        mbox=T$(unzip -l $zip | grep .mbox | cut -d'T' -f2-) # Because google.zip have "Takeout" folder first
        #echo $mbox
        nice -n 10 unzip -j "$zip" "${mbox}" -d $var/
        nice -n 10 pilerimport -m "$var/$(ls -1 $var | grep .mbox)" -F $var;
        nice -n 10 rm -i $var/*.mbox
    done
    su - piler -c "nice -n 10 indexer delta1 --rotate" 
fi
exit

or import.mbox.sh (when i have unzipped mbox)

#!/bin/bash

if [ "$#" -eq 0 ]; 
then 
    echo "Usage: ./`basename $0` d_name.surname d_name.surname2 d_name.surname3 ..."; 
else
    for var in "$@"; do
        echo $var
        nice -n 10 pilerimport -m "$var/$(ls -1 $var | grep .mbox)" -F $var | grep -Ev 'a4'
    done 
    su - piler -c "nice -n 10 indexer delta1 --rotate"
fi
exit

and i have

Feb  4 08:43:01 e577ed228937 piler-webui[282]: sphinx query: 'SELECT id FROM main1,dailydelta1,delta1 WHERE       folder IN (3,1) AND  MATCH('') ORDER BY `sent` DESC LIMIT 0,1000 OPTION max_matches=1000' in 0.00 s, 0 hits, 0 total found

mysql -h 127.0.0.1 -P 9306

mysql> select * from main1,dailydelta1,delta1;
Empty set (0.01 sec)

When data is writing to this mysql? How to fix it?

[Mateusz Szymborski]

Fixed.

Source of error: su - piler -c "nice -n 10 indexer delta1 --rotate"

When you index second package with it, it will replace delta1.

Fix: su - piler -c "nice -n 10 indexer delta1 --rotate && sleep 2 && nice -n 10 indexer --merge main1 delta1 --merge-dst-range deleted 0 0 --rotate && sleep 1 && nice -n 10 indexer tag1 --rotate && sleep 1 && nice -n 10 indexer note1 --rotate"

[Mateusz Szymborski]

All works. Thanks! Waiting for LDAP Folders feature. (Login with ldap data, connect to database, if user not exist then create account, else get data about accesses) This Issue can be closed.

[Janos SUTO]

I am glad that it finally works.