cleanup script does not working.

[IT Expert]

Hi, Olivier.

Looks like clean-up script does not working. Have config file option:

# Clean up ##################################
# Number of days after which job directory
# can be deleted by cleanup script
# Default: 30
clean_old: 14

but there are jobs dirs from April

drwxr-xr-x  12 root root  12 Apr 23 17:03 79/
drwxr-xr-x  12 root root  12 Apr 24 17:58 88/
...

drwxr-xr-x  12 root root  12 Apr 26 10:23 99/

Is it scheduler feature, or should I have any external cleaning script ?

[Olivier Sallou]

Cleanup is done by archive service and cleanup script (via cron)

See Archiver section if readme.md

[IT Expert]

Thank you, will see.

[IT Expert]

One more question.

How should it work properly if we use containers ? Maybe better idea to move this feature to god-scheduler ?

[IT Expert]

In case of containers, am I right if I say, to get it work:

should setup all environments on some host, have mounted storage, expose port from mongodb, have additional go-d.ini (because of different settings to db, mountpoint, etc ?), and then container --link option could not work...

or have installed and configured crontab on one of this docker containers (god-scheduler, god-watcher), but there will be issues with containers update.

or make additional container "cron" link it to mongo, and go-d.ini ?

[Olivier Sallou]

cron should just execute something like (where godarchiver container runs)

docker exec containername python3 go-d-clean.py [options]

regarding setup, all services should run on same host , or use dockerswarm and use a docker network between containers. With docker-compose, a network is automatically created and containers can talk each other with the defined container name, no need to open ports (except for godocker web of course which should be the only one open on internet) . shared storage should be mounted on all godocker containers (using same path than on host ( -v /somepath:/somepath with somepath being the go-d.ini configured shared dir)

you have an example docker-compose in source at go-d-sched/docker-files/compose/docker-compose.yml (which may be customized to match your config/setup)

go-d.ini in this example is mounted in container to override defaults.

[IT Expert]

Oh, I see. so you propose add to cron something like this:

30 * * docker exec -it \ --link god-mongo:god-mongo \ --link god-redis:god-redis \ -v /opt/godshared:/opt/godshared \ -v path_to/go-d.ini:/opt/go-docker/go-d.ini \ --rm \ osallou/go-docker \ /usr/bin/python3 go-d-clean.py

in this case we have all environments and connections to db and storage.

[Olivier Sallou]

nope, a docker exec is to be run on an existing container

docker exec your_godocker_archiver_running_container_id /usr/bin/python3 go-d-clean.py

in your case, it could also work with a

docker run --link ....

[IT Expert]

yeap, my mistake

every 30 minutes

docker run --rm --link god-mongo:god-mongo --link god-redis:god-redis -v /opt/godshared:/opt/godshared -v path_to/go-d.ini:/opt/go-docker/go-d.ini osallou/go-docker /usr/bin/python3 go-d-clean.py

can't find information about godocker archiver container, if I understood right this is the same image osallou/go-docker container but running with /usr/bin/python3 go-d-clean.py script ?

[Olivier Sallou]

archiver uses the same container image, but uses:

/usr/bin/python3 go-d-archive.py run

see docker-compose example file https://bitbucket.org/osallou/go-docker/src/master/docker-files/compose/docker-compose.yml

[Olivier Sallou]

cleanup script looks at "old" jobs and send an archive request

the archive service will then pull all archive commands and archive each related job

[IT Expert]

Got it.

Thank you very much!