go-docker: mem_limit has been moved to host_config in API version 1.19

[Ruben]

Hi,

We are trying to launch a batch with go-docker and we have this error:

"mem_limit has been moved to host_config in API version 1.19"

{code} Client: Version: 1.12.0 API version: 1.24 Go version: go1.6.3 Git commit: 8eab29e Built: Thu Jul 28 22:05:53 2016 OS/Arch: linux/amd64

Thxs for helping.

[Olivier Sallou]

Is it with swarm plugin executor?

[Olivier Sallou]

docker 1.12 may have broken older APIs. Code (swarm.py) specifies the docker api version used with docker-py with go-d.ini param docker_api_version ('1.17' by default) and 1.17 is ok with mem_limit.

If you use mesos, mesos docker executor may not be compatible with docker 1.12.

I plan for swarm to switch to the host_config change, but if related to mesos, issue must be moved to mesos.

[Olivier Sallou]

Test swarm executor against Docker 1.12 mem_limit needs to move to host_config in create_container. Check if other fields are impacted. May need a docker-py upgrade

[Ruben]

Hi!

First of all, thanks for your quick response.

In our case, we are using the swarm plugin executor. We are using the docker-py version 1.9 (docker_py-1.9.0.dist-info), isn't it the latest version?

Sorry, but how can we move the mem_limit to host_config in create container? We have to change some python script or something like that?

p.d: attacked go-d.ini

Thanks!!!

[Olivier Sallou]

go-docker uses docker-py 1.7 in requirements.txt, as it is the version used in testing. Upgrade is planned but not yet done. But docker-py version is not the issue, I could reproduce the issue after an upgrade to docker 1.12. Software has been tested with docker 1.11 and lower.

I need additional tests as the upgrade produced docker crashes.

In plugins/swarm.py host config must be added to create_container:

# Create host_config
container_host_config= self.docker_client.create_host_config(mem_limit=str(job['requirements']['ram'])+'g')
container = self.docker_client.create_container(host_config=container_host_config, image=....)

and mem_limit=str(job['requirements']['ram'])+'g' must be removed from create_container.

Not sure however it will be enough, I will try to add host_config management in next few days.

[Olivier Sallou]

Closes #39, use docker host_config to match recent docker API

→ <<cset 1710bb6852b8>>

[Ruben]

Thank for your help Olivier.

I update my local version of Go-Docker with this changes and I'm getting this error:

2016-08-02 18:27:26,439 ERROR [godocker-scheduler][MainThread] Execute:Job:3:('Connection aborted.', BadStatusLine('\x15\x03\x01\x00\x02\x02\x16',))

I don't know if it could be an error with my docker swarm configuration (i'm new with that too :) ) or something with the docker_py version, or other kind of things.

My go-d.ini looks like:

docker_url: 'tcp://192.168.99.100:3376'

docker info
Containers: 12
 Running: 1
 Paused: 0
 Stopped: 11
Images: 2
Server Version: 1.12.0
Storage Driver: aufs
 Root Dir: /mnt/sda1/var/lib/docker/aufs
 Backing Filesystem: extfs
 Dirs: 28
 Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge null overlay host
Swarm: active
 NodeID: 4wbdner4v2y60d35w4ck794vd
 Is Manager: true
 ClusterID: 27gzrg2wuqeymz80mw1znr7r4
 Managers: 1
 Nodes: 1
 Orchestration:
  Task History Retention Limit: 5
 Raft:
  Snapshot interval: 10000
  Heartbeat tick: 1
  Election tick: 3
 Dispatcher:
  Heartbeat period: 5 seconds
 CA configuration:
  Expiry duration: 3 months
 Node Address: 192.168.99.100
Runtimes: runc
Default Runtime: runc
Security Options: seccomp
Kernel Version: 4.4.16-boot2docker
Operating System: Boot2Docker 1.12.0 (TCL 7.2); HEAD : e030bab - Fri Jul 29 00:29:14 UTC 2016
OSType: linux
Architecture: x86_64
CPUs: 1
Total Memory: 995.9 MiB
Name: manager
ID: 5GB2:CENI:FCJB:5N2G:SUOU:GE5D:I2NS:SE7V:ED33:5IAL:SKYF:RNS6
Docker Root Dir: /mnt/sda1/var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 33
 Goroutines: 119
 System Time: 2016-08-02T16:32:51.427254648Z
 EventsListeners: 0
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox
Insecure Registries:
 127.0.0.0/8

and

docker -H 192.168.99.100:3376 info
Containers: 17
 Running: 2
 Paused: 0
 Stopped: 15
Images: 4
Server Version: swarm/1.2.4
Role: primary
Strategy: spread
Filters: health, port, containerslots, dependency, affinity, constraint
Nodes: 2
 agent1: 192.168.99.101:2376
  └ ID: 2HTQ:4BAI:XJ7U:FTOI:EANZ:GRA4:4KPG:ASMH:ZI7M:GPC6:24WE:XKW3
  └ Status: Healthy
  └ Containers: 9 (1 Running, 0 Paused, 8 Stopped)
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: kernelversion=4.4.16-boot2docker, operatingsystem=Boot2Docker 1.12.0 (TCL 7.2); HEAD : e030bab - Fri Jul 29 00:29:14 UTC 2016, provider=virtualbox, storagedriver=aufs
  └ UpdatedAt: 2016-08-02T16:35:11Z
  └ ServerVersion: 1.12.0
 agent2: 192.168.99.102:2376
  └ ID: LQJP:2IH5:A6GA:SBJW:NDZD:LXNH:OZ4F:2BPH:ARKM:4ZXA:VG26:BEUL
  └ Status: Healthy
  └ Containers: 8 (1 Running, 0 Paused, 7 Stopped)
  └ Reserved CPUs: 0 / 1
  └ Reserved Memory: 0 B / 1.021 GiB
  └ Labels: kernelversion=4.4.16-boot2docker, operatingsystem=Boot2Docker 1.12.0 (TCL 7.2); HEAD : e030bab - Fri Jul 29 00:29:14 UTC 2016, provider=virtualbox, storagedriver=aufs
  └ UpdatedAt: 2016-08-02T16:35:23Z
  └ ServerVersion: 1.12.0
Plugins:
 Volume: 
 Network: 
Swarm: 
 NodeID: 
 Is Manager: false
 Node Address: 
Security Options:
Kernel Version: 4.4.16-boot2docker
Operating System: linux
Architecture: amd64
CPUs: 2
Total Memory: 2.042 GiB
Name: 689c244f822d
Docker Root Dir: 
Debug Mode (client): false
Debug Mode (server): false
WARNING: No kernel memory limit support

Have you ever seen this error before?

Thanks!!!

[Olivier Sallou]

First time i see this. Did you use swam standalone above docker or the new docker swam embedded in docker?

I tested fix against docker directly, using docker tcp url tcp:// 127.0.0.1:2375, i did not tested fix against swarm on my computer.

Do you use python 2 or 3?

Could you open a new issue to track this to avoid mixing pb?

Thanks

[Olivier Sallou]

could you also try:

docker -H 192.168.99.100:3376 info

I made a very quick test with the new swarm / docker 1.12 and faced same issue:

osallou@tifenn~ $ docker -H 127.0.0.1:2377 info
Get http://127.0.0.1:2377/v1.24/info: malformed HTTP response "\x15\x03\x01\x00\x02\x02".
* Are you trying to connect to a TLS-enabled daemon without TLS?

So it looks like a swarm setup issue vs tls

[Olivier Sallou]

I can't test with swarm, I face on my computer some Docker crashes from a known bug that should be fixed in next docker release 1.12.1 (https://github.com/docker/docker/pull/25053). But if you have same error when connecting to swarm directly, this is definitly a docker/swarm setup issue.