Get logs when job running

[IT Expert]

Hi. Is it possible to get logs of job through god-web api or mesos web interface when the job is still running ?

When the job is running there is only way to get it logs is open god.log from shared folder from server like:

tail -f /data/godshared/tasks/pairtree_root/80/task/god.log

But users have not access to servers shell.

When I try to get it through api: http://host:6543/api/1.0/task/80/files/god.log

I got error

404 Not Found
The resource could not be found.

Job does not exist 

until job will not finish. When the job will finish this url will work.

[Olivier Sallou]

It is possible, but you need to run and configure go-d-live process This gives you live monitoring of god.log via web sockets. https:://bitbucket.org/osallou/go-docker-live

[Olivier Sallou]

I will have a look anyway, file should be accessible as soon as job is running.

[IT Expert]

Oh, thank you! I saw it, but didn't understood what it for.

[Olivier Sallou]

fix access to running jobs in osallou/go-docker-web: https://bitbucket.org/osallou/go-docker-web/commits/f77ca7ed8d47dcd98c63bb099257f47621c1e6ce

[Olivier Sallou]

web proxy need to forward go-d-live requests via web proxy, as described in doc.go-d.ini should specify use of go-d-live:

live_events: False
#  Go-D-Live websocket URL
live_events_url: 'http://localhost:8001'

best, to avoid CORS issues and remote port access (for ips and firewall), is to forward go-docker-web AND go-d-live to same URL, a web proxy, then forward /socket.io web socket requests to go-d-live and others to go-d-web

[Olivier Sallou]

in go-d-live config , secret passphrase should match go-d.ini secret_passphrase field

[Olivier Sallou]

I have updated go-d-web in develop branch (and :dev container) to access god.log via http://host:6543/api/1.0/task/80/files/god.log for running jobs. Will be in next release. There is however no link in UI to access it. In web ui, log access for running jobs goes through go-d.live (but files will remain accessible anyway via direct http link)

[IT Expert]

Thank you, I will try it.

[Olivier Sallou]

tell me if you have issues setting up go-d-live (doc may need to be updated to be more concise)

[IT Expert]

Just launched it. You should add to documentation something like this:

1. Prepare config files:

default.json
{
 "app": {
     "port": 8001,
     "secret_passphrase": "passs"
 },
 "redis": {
    "host": "god-redis",
    "port": 6379,
    "prefix": "god",
    "password": null
  }

}

go-d.ini

live_events: True
#  Go-D-Live websocket URL
live_events_url: 'http://localhost:8001'

secret_passphrase: 'passs'
shared_secret_passphrase: 'passs'

2. Run go-docker-live link redis & shared folder

docker run -d --name god-live  --link god-redis:god-redis -v /data/godshared:/data/godshared -v /opt/go-docker/default.json:/root/config/default.json -p 8001:8001 osallou/go-docker-live

Then from web interface click on Job id then click on Live log button (http://host:6543/app/#/job/149/live)

If everything ok you will see logs, god-live logs should be like this:

2. Check god-live logs
docker logs god-live

info: no auth
info: Listen on: 8001
info: Redis: god-redis:6379
info: Subscription channel: god:jobs:pubsub
info: Redis connected
debug: Subscribed
debug: new connection
debug: Received message from client: authenticate
debug: Received message from client: authenticate
debug: new connection
debug: Received message from client: authenticate
debug: new connection
debug: Received message from client: authenticate
debug: Received message from client: tail
debug: Received message from client: untail
debug: Received message from client: tail
debug: Received message from client: untail
debug: Received message from client: tail
debug: new connection
debug: Received message from client: authenticate
debug: Received message from client: tail
debug: Received message from client: untail

[Olivier Sallou]

I have updated the README of go-docker-live, thanks