1. Atlassian Avatar Atlassian
  2. Project: Atlassian
  3. localstack
  4. Issues

Stack boot fails when using specifying DATA_DIR

Issue #38 resolved
Narinder Chandi created an issue

When trying to boot the stack with the DATA_DIR env var specified like this:

docker run -it -p 4567-4578:4567-4578 -p 8080:8080 -e DATA_DIR='/tmp/localstack_data' atlassianlabs/localstack

Localstack fails to boot. This is the output:

/usr/lib/python2.7/site-packages/supervisor/options.py:296: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
2017-06-27 09:54:03,289 CRIT Supervisor running as root (no user in config file)
2017-06-27 09:54:03,292 INFO supervisord started with pid 1
2017-06-27 09:54:04,296 INFO spawned: 'dashboard' with pid 9
2017-06-27 09:54:04,299 INFO spawned: 'infra' with pid 10
(. .venv/bin/activate; bin/localstack web --port=8080)
(. .venv/bin/activate; bin/localstack start)
Traceback (most recent call last):
  File "bin/localstack", line 27, in <module>
    from localstack import config, constants
  File "/opt/code/localstack/localstack/config.py", line 48, in <module>
    os.makedirs(folder)
  File "/opt/code/localstack/.venv/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 17] File exists: '/tmp/localstack_data'
make: *** [Makefile:74: web] Error 1
2017-06-27 09:54:04,384 INFO exited: dashboard (exit status 2; not expected)
Starting local dev environment. CTRL-C to quit.
Starting local Elasticsearch (port 4571)...
2017-06-27 09:54:05,355 INFO success: infra entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
Starting mock ES service (port 4578)...
Starting mock S3 (port 4572)...
Starting mock SNS (port 4575)...
Starting mock SQS (port 4576)...
Starting mock API Gateway (port 4567)...
Starting mock DynamoDB (port 4569)...
Starting mock DynamoDB Streams service (port 4570)...
Starting mock Firehose service (port 4573)...
Starting mock Lambda service (port 4574)...
Starting mock Kinesis (port 4568)...
Starting mock Redshift (port 4577)...
Starting mock Route53 (port 4580)...
2017-06-27 09:54:05,388 INFO spawned: 'dashboard' with pid 85
Starting mock SES (port 4579)...
Starting mock CloudFormation (port 4581)...
Starting mock CloudWatch (port 4582)...
(. .venv/bin/activate; bin/localstack web --port=8080)
2017-06-27 09:54:08,740 INFO success: dashboard entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 487-245-508
ERROR:localstack.mock.generic_proxy:Error forwarding request: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d855a50>: Failed to establish a new connection: [Errno 111] Connection refused',))
Traceback (most recent call last):
  File "/opt/code/localstack/localstack/mock/generic_proxy.py", line 134, in forward
    headers=forward_headers)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 471, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 581, in send
    r = adapter.send(request, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/adapters.py", line 481, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d855a50>: Failed to establish a new connection: [Errno 111] Connection refused',))
[2017-06-27T09:54:14,960][INFO ][o.e.n.Node               ] [] initializing ...
[2017-06-27T09:54:15,145][INFO ][o.e.e.NodeEnvironment    ] [dB8Q3RR] using [1] data paths, mounts [[/ (none)]], net usable_space [176.7gb], net total_space [192.6gb], spins? [possibly], types [aufs]
[2017-06-27T09:54:15,146][INFO ][o.e.e.NodeEnvironment    ] [dB8Q3RR] heap size [483.3mb], compressed ordinary object pointers [true]
[2017-06-27T09:54:15,149][INFO ][o.e.n.Node               ] node name [dB8Q3RR] derived from node ID [dB8Q3RRhTJS4pdeiCGfjyQ]; set [node.name] to override
[2017-06-27T09:54:15,150][INFO ][o.e.n.Node               ] version[5.3.0], pid[56], build[3adb13b/2017-03-23T03:31:50.652Z], OS[Linux/4.9.27-moby/amd64], JVM[Oracle Corporation/OpenJDK 64-Bit Server VM/1.8.0_131/25.131-b11]
[2017-06-27T09:54:16,690][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [aggs-matrix-stats]
[2017-06-27T09:54:16,690][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [ingest-common]
[2017-06-27T09:54:16,691][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [lang-expression]
[2017-06-27T09:54:16,691][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [lang-groovy]
[2017-06-27T09:54:16,692][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [lang-mustache]
[2017-06-27T09:54:16,692][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [lang-painless]
[2017-06-27T09:54:16,693][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [percolator]
[2017-06-27T09:54:16,695][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [reindex]
[2017-06-27T09:54:16,695][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [transport-netty3]
[2017-06-27T09:54:16,695][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] loaded module [transport-netty4]
[2017-06-27T09:54:16,697][INFO ][o.e.p.PluginsService     ] [dB8Q3RR] no plugins loaded
[2017-06-27T09:54:19,589][INFO ][o.e.n.Node               ] initialized
[2017-06-27T09:54:19,590][INFO ][o.e.n.Node               ] [dB8Q3RR] starting ...
[2017-06-27T09:54:19,801][WARN ][i.n.u.i.MacAddressUtil   ] Failed to find a usable hardware address from the network interfaces; using random bytes: a8:e6:9b:cc:9f:53:e7:b8
[2017-06-27T09:54:19,957][INFO ][o.e.t.TransportService   ] [dB8Q3RR] publish_address {127.0.0.1:9300}, bound_addresses {[::1]:9300}, {127.0.0.1:9300}
[2017-06-27T09:54:19,969][WARN ][o.e.b.BootstrapChecks    ] [dB8Q3RR] initial heap size [209715200] not equal to maximum heap size [524288000]; this can cause resize pauses and prevents mlockall from locking the entire heap
[2017-06-27T09:54:23,128][INFO ][o.e.c.s.ClusterService   ] [dB8Q3RR] new_master {dB8Q3RR}{dB8Q3RRhTJS4pdeiCGfjyQ}{6IxMos-6RKSK8dDt9hNXgw}{127.0.0.1}{127.0.0.1:9300}, reason: zen-disco-elected-as-master ([0] nodes joined)
[2017-06-27T09:54:23,180][INFO ][o.e.h.n.Netty4HttpServerTransport] [dB8Q3RR] publish_address {127.0.0.1:4588}, bound_addresses {[::1]:4588}, {127.0.0.1:4588}
[2017-06-27T09:54:23,192][INFO ][o.e.n.Node               ] [dB8Q3RR] started
[2017-06-27T09:54:23,194][INFO ][o.e.g.GatewayService     ] [dB8Q3RR] recovered [0] indices into cluster_state
ERROR:localstack.mock.generic_proxy:Error forwarding request: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d558c90>: Failed to establish a new connection: [Errno 111] Connection refused',))
Traceback (most recent call last):
  File "/opt/code/localstack/localstack/mock/generic_proxy.py", line 134, in forward
    headers=forward_headers)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 471, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 581, in send
    r = adapter.send(request, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/adapters.py", line 481, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d558c90>: Failed to establish a new connection: [Errno 111] Connection refused',))
ERROR:localstack.mock.generic_proxy:Error forwarding request: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d558e90>: Failed to establish a new connection: [Errno 111] Connection refused',))
Traceback (most recent call last):
  File "/opt/code/localstack/localstack/mock/generic_proxy.py", line 134, in forward
    headers=forward_headers)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 110, in post
    return request('post', url, data=data, json=json, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/api.py", line 56, in request
    return session.request(method=method, url=url, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 471, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/sessions.py", line 581, in send
    r = adapter.send(request, **kwargs)
  File "/opt/code/localstack/.venv/lib/python2.7/site-packages/requests/adapters.py", line 481, in send
    raise ConnectionError(e, request=request)
ConnectionError: HTTPConnectionPool(host='127.0.0.1', port=4584): Max retries exceeded with url: / (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f791d558e90>: Failed to establish a new connection: [Errno 111] Connection refused',))

I also tried changing the DATA_DIR location to /data/localstack without success. Any thoughts? We are really keen to get this working for us!

I am running on Mac OS X 10.12.5 with Docker version 17.03.1-ce, build c6d412e and Python 2.7.10 (as reported by docker --version and python --version respectively).

Comments (7)

  1. Narinder Chandi reporter

    Just an update that by modiying the previous docker run comand to include the -v option we have the stack working now: docker run -it -p 4567-4578:4567-4578 -p 8080:8080 -e DATA_DIR='/tmp/localstack/data' -v '/data/localstack:/tmp/localstack/data' atlassianlabs/localstack

    However, the errors are still reported as above and the net effect is that the web front-end running on port 8080 is unable to load active resources. It's not a show stopper but it still seems worth reporting as an issue.

  3. Narinder Chandi reporter

    This is much improved thanks! Now, the front-end is functioning and reflecting entities that I create. I have tested with DymanoDB, SQS and S3.

    However, I am now seeing an odd thing - when I create and delete S3 buckets the bucket is persisted between docker runs regardless of the fact that I deleted it. But with SQS, the behaviour is the exact opposite - the queue is not persisted after creation and container exit. DynamoDB is behaving exactly as expected regarding persistence.

    Fyi, here is the docker run command I am using:

    docker run -it -p 4567-4578:4567-4578 -p 8080:8080 -e DATA_DIR='/tmp/localstack/data' -v '/data/localstack:/tmp/localstack/data' atlassianlabs/localstack

  4. Waldemar Hummer Account Deactivated

    Thanks for the follow-up. Please note the remark in the README (in fact it should also mention S3):

    saving persistent data (currently only supported for these services: Kinesis, DynamoDB, Elasticsearch)

    Regarding the point about deletion of buckets - that's a bug. Root cause is here: https://bitbucket.org/atlassian/localstack/src/master/localstack/utils/persistence.py?at=master&fileviewer=file-view-default#persistence.py-27 This will be fixed shortly.

  5. Narinder Chandi reporter

    Ah ok, thanks for the reminder about persistence support limitations. We should be able to workaround the lack of support for SQS persistence for the actual queues we need but is there any short-term prospect of full persistence support for it as we are currently using SQS and DynamoDB in our application architecture? Thanks.

  8. Log in to comment
Assignee
–
Type
bug
Priority
major
Status
resolved
Votes
0
Watchers
1