Use MySql docker image for unit tests in Pipelines

Issue #12901 duplicate
Daniel Vu
created an issue

I want to have a database running so that my build can run tests which actually query from a live database (integration tests).

Conceptually, I'm trying to do something like:

image: mysql

pipelines: default: - step: script: # Modify the commands below to build your repository. - "mysqld" - "mysql -u root -e 'CREATE DATABASE databaseTestInstance'"

That should just pull the official mysql image down from dockerhub, start the database process, and create a default database that my test suite can connect to.

This, however, doesn't work. Officially, it looks like the MySql team just built this image such that you can run it from your local box, via:

docker run --name app-container-name --link my-container-name:mysql -d app-that-uses-mysql

https://hub.docker.com/_/mysql/

I'm not sure how to port that logic to something Pipelines can understand. To me, it seems like Pipelines just uses "image:mysql" to construct the docker run command. So what I'd like to know is what to put in my .yml file such that I can pull a mysql server docker image down, start the mysql process, and connect to it such that I can create a default database.

Comments (14)

  1. Sten Pittet

    Hi Daniel,

    We want to solve this issue with #12757 - Ability to run multiple docker containers in the same build which will allow you to start multiple containers for the step in progress. In other words it will allow you to have both your application code and a database service in your pipeline.

    If you only specify image: mysql we will indeed start only one container that is optimised for MySQL and that probably cannot be used for your application due to missing dependencies.

    A workaround would be to create a custom Docker image that has both MySQL and your dependencies installed.

    I'm going to close this issue as a duplicate of #12757 - feel free to add your feedback there.

    Thanks,

    Sten

  2. Daniel Vu reporter

    Hi Sten,

    I'm not sure if I feel that this issue is a duplicate of #12757.

    "A workaround would be to create a custom Docker image that has both MySQL and your dependencies installed."

    I have attempted this. MySQL does not appear to start. It starts if I run the docker image locally, however, in Pipelines it does not. You guys probably have more insight into why this might be, as I can only guess at what Pipelines is doing. From my perspective, I have been unable to get MySql to run during a Pipelines build.

    Can you guys run MySql? Can you provide me with a .yml configuration where MySQL is actually running? If so, perhaps it's just a documentation improvement. I would consider it working if I could specify that particular mysql image in the yml file and also get a valid response from "mysql -u root -e 'CREATE DATABASE databaseTestInstance'".

    Whenever I try and use a MySql docker image, the process is not running, so a command such as:

    "mysql -u root -e 'CREATE DATABASE databaseTestInstance'"

    just fails to work. The stock mysql image relies on a specific startup script to execute and bootstrap the image. From my perspective, it looks like Pipelines is circumnavigating that process, and so no daemon is available to respond after startup.

    Thanks!

    daniel

  3. Jonathan Higgs

    Hi Daniel,

    Did you ever find a solution to this? I've got the exact same problem where I'm trying to run some integration tests with mysql, but I just can't get the mysql instance to start up when running in pipelines. Please let me know if you found a work around?

  4. Barnabás Gema

    Hi Daniel and Jonathan,

    I was facing a pretty similar task yesterday, I had to run integration tests for a node.js application with a MySQL database. I solved it by pulling the official node.js docker image, installing MySQL onto it, and uploading it to docker hub. My bitbucket-pipelines.yml uses this image and contains these two lines to make MySQL work:

    - service mysql start
    - mysql -u root --password=$MYSQL_PW mysql < pipelines_init.sql
    

    So basically I start the MySQL server and execute the pipelines_init.sql file with the root user and the deafult database. This file contains the creation of a new database, the creation of a database user for my application, and some other stuff that my application needs.

    I hope this helps you finding a workaround.

    Barnabas

  5. Jonathan Higgs

    Thanks Barnabas, That did help! I had installed mysql on the official ubuntu image and I could get mysql running locally but not in pipelines. (I had been using /etc/init.d/mysql start to start the service which worked locally but not in pipelines).. Changing that to the use the command - service mysql start fixed my issue. Thanks!

  6. Barnabás Gema

    On a side note, since then I tried starting MySQL with /etc/init.d/mysql start and it worked for me just fine, my integration tests came out the same way as they did with service mysql start. (I'm using Ubuntu 14.04.) So I'm not sure what the cause of your issue has been, but I'm glad that I could help.

  7. Daniel Vu reporter

    Just an FYI, I had not tried installing MySql on a custom image. I have been trying to use the official MySql docker images, which apparently work a bit differently. I can't get them to work, even with "service mysql start". I'll try to manually install MySql and see if that takes care of it...unless you guys have already made a docker image with MySql public that I can just point to.

  8. Sumit Gupta

    Hey, I am having trouble with custom docker and mysql. As reported by other, my Docker works fine on local machine, I can get

    service mysql start 
    

    working, but through yml, it just says "fail" and doesn't execute next command.

  9. Sumit Gupta

    Hey, if anyone is looking for solution that works for Custom Images

    usermod -d /var/lib/mysql/ mysql
    service mysql start
    

    i.e. all I did is set home directory of MySQL user, which was not set by default in docker image. now it starts fine. I will try to set it in my docker image if that works.

  10. Sten Pittet

    Hi,

    I just created a Node.js image with MySQL bundled to use with Bitbucket Pipelines. I have an example in a repository that you can copy and adapt for your own needs and I'm preparing an equivalent example for Ruby and Python.

    Feedback is welcomed, don't hesitate if you have any question.

    Thanks,

    Sten

  11. David D Lowe

    Here's how I managed to get it to work for Python and PostgreSQL. The Bitbucket docs recommend this Docker image, but don't explain how to use it.

    image: phodder/python-postgresql
    
    pipelines:
      default:
        - step:
            script: # Modify the commands below to build your repository.
              - pip install -r requirements.txt
              - service postgresql start
              - sudo -u postgres sh -c "psql -c \"CREATE USER test WITH PASSWORD 'test'; ALTER USER test CREATEDB;\" && createdb test"
              - python manage.py test
    
  12. Log in to comment