The below script works fine for me. Just tested in the new infrastructure:
#!/bin/bash
##---Fill-up the user account details, instance url and timezone-----
USERNAME=USER_NAME_HERE
PASSWORD=PASSWORD_HERE
INSTANCE=INSTANCE.atlassian.net
LOCATION="/path/to/download/folder"
# Set this to your Atlassian instance's timezone.
# See this for a list of possible values:
# https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
TIMEZONE=Europe/Amsterdam
##----START-----###
echo "Starting the script..."
# Grabs cookies and generates the backup on the UI. ########
### PLEASE NOTICE THAT THE SESSION IS CREATED BY CALLING THE JIRA SESSION ENDPOINT!!! ######
#### THE SCRIPT DOES NOT WORK IF JIRA IS NOT INSTALLED !!! #######
TODAY=$(TZ=$TIMEZONE date +%Y%m%d)
COOKIE_FILE_LOCATION=jiracookie
curl --silent --cookie-jar $COOKIE_FILE_LOCATION -X POST "https://${INSTANCE}/rest/auth/1/session" -d "{\"username\": \"$USERNAME\", \"password\": \"$PASSWORD\"}" -H 'Content-Type: application/json' --output /dev/null
## The $BKPMSG variable will print the error message, you can use it if you're planning on sending an email
BKPMSG=$(curl -s --cookie $COOKIE_FILE_LOCATION --header "X-Atlassian-Token: no-check" -H "X-Requested-With: XMLHttpRequest" -H "Content-Type: application/json" -X POST https://${INSTANCE}/wiki/rest/obm/1.0/runbackup -d '{"cbAttachments":"true" }' )
## Checks if the backup procedure has failed
if [ "$(echo "$BKPMSG" | grep -ic backup)" -ne 0 ]; then
rm $COOKIE_FILE_LOCATION
echo "FAILED, IT RETURNED $BKPMSG"
exit
fi
## Checks if the backup exists every 10 seconds, 2000 times. If you have a bigger instance with a larger backup file you'll probably want to increase that.
for (( c=1; c<=2000; c++ ))
do
PROGRESS_JSON=$(curl -s --cookie $COOKIE_FILE_LOCATION https://${INSTANCE}/wiki/rest/obm/1.0/getprogress.json)
FILE_NAME=$(echo "$PROGRESS_JSON" | sed -n 's/.*"fileName"[ ]*:[ ]*"\([^"]*\).*/\1/p')
##ADDED: PRINT BACKUP STATUS INFO ##
echo "$PROGRESS_JSON"
if [[ $PROGRESS_JSON == *"error"* ]]; then
break
fi
if [ ! -z "$FILE_NAME" ]; then
break
fi
sleep 10
done
#If after 2000 attempts it still fails it ends the script.
if [ -z "$FILE_NAME" ];
then
rm $COOKIE_FILE_LOCATION
exit
else
## PRINT THE FILE TO DOWNLOAD ##
echo "File to download: $FILE_NAME"
curl -s -L --cookie $COOKIE_FILE_LOCATION "https://${INSTANCE}/wiki/download/$FILE_NAME" -o "$LOCATION/CONF-backup-${TODAY}.zip"
fi
rm $COOKIE_FILE_LOCATION
Cookie-based auth for REST APIs is deprecated. API tokens worked for me:
PROGRESS_JSON=$(curl -s --user "${USERNAME}:${PASSWORD}" https://${INSTANCE}/wiki/rest/obm/1.0/getprogress.json)
API token is available at https://id.atlassian.com/manage/api-tokens.
Props to you Kenta!! Dropping by to 7 month old post to PROACTIVELY give us a heads up about that change.
THANK YOU!!
OK, so we don't need to use username and password? What would replace the lines using username and password then? I understand what API tokens are and generated one (even though the page you do so states they are only used when two factor is turned on), but I have no idea how to call it in regards to the script.
I don't understand your question. Can you try to be a bit more specific and point to the lines you are referring to?
Also, did you see the comment from Kenta Yamamoto?
I did see Kenta's response, I did finally get it fixed based on his example after doing research and testing not knowing what I needed to remove from your curl commands (still new to working with APIs from command line). Everything is good now for Confluence, but my Jira script seems to no longer work for some reasons. I just keep getting back:
{"fileName":"temp/filestore/e0613346-ed9d-45de-84f1-e4439878a99d","size":406365173,"currentStatus":"Archiving attachments.","alternativePercentage":"100%","concurrentBackupInProgress":false}
I contact Atlassian Support and basically got back "Oh I only use UI, I don't know anything about the API, here is a list of feature requests to add automated backups in the UI, none of them are being worked on though." Yes, seriously.
Basically when I run the Jira script (which is pretty much the same except for slightly different endpoints) I get back that error message over and over again, I've been poking at this all week. Also the script worked perfectly fine 2 weeks ago when they still allowed cookie auth, not quite sure why the Confluence works just fine after I switched to token but the Jira one won't...
Hi Devin,
Automatic backups are indeed not officially supported as you can read in: * https://confluence.atlassian.com/jirakb/how-to-automate-backups-for-jira-cloud-applications-779160659.html
Now, from what I can see your script is telling you that the backup is completed (100%) and it has been saved in 'temp/filestore/e0613346-ed9d-45de-84f1-e4439878a99d'.
Can you kindly confirm you are using the below version of the script? * https://bitbucket.org/atlassianlabs/automatic-cloud-backup/issues/16/endpoint-changed-for-jira-again-new-script
In case you are, most likely the below line will have to be changed to reflect the new response:
FILE_NAME=$(echo "$PROGRESS_JSON" | sed -n 's/.*"result"[ ]*:[ ]*"\([^"]*\).*/\1/p')
If I have some time I will try to test and fix the script this week.
Cheers, Dario
I'm done testing the Jira Scripts. Soon I will add the one for CONF as well.
Hello Dario
First of all, thanks a lot for your script.
And one more request or report.
We want to organise notifying (via email for example) that script sucsecfuly done. But when we check result and code of status, we find file with backup and that your script had "Last run result" is 0x1 (incorrect function call).
Could you fix it please? Thanks a lot, Dmytro
Can you kindly confirm whether you are using the script at the top of this page and or the powershell one?
Script from the top. We launch Git Bash for Windows from Windows task scheduler.
.png)
.png)
I would happy to use powershell but i cant find any solutions for this goal.
PS I don't know why but images aren't showed. Pls try to check https://snag.gy/fgR01d.jpg https://snag.gy/v7pGga.jpg
Sorry for late reply. Tomorrow I am going to test the script and try to update this issue :)
I have just tested the script and it works fine for me.
Can you kindly paste the last lines of what is logged in the console when you run the script?
E.g. In my case I have:
...
....
{"size":0,"currentStatus":"Archiving attachments.","alternativePercentage":"Progress: 26% longer than last time","concurrentBackupInProgress":false}
{"size":0,"currentStatus":"Archiving attachments.","alternativePercentage":"Progress: 26% longer than last time","concurrentBackupInProgress":false}
{"size":0,"currentStatus":"Archiving attachments.","alternativePercentage":"Progress: 27% longer than last time","concurrentBackupInProgress":false}
{"size":0,"currentStatus":"Archiving attachments.","alternativePercentage":"Progress: 28% longer than last time","concurrentBackupInProgress":false}
{"fileName":"temp/filestore/0dc57fde-b470-44f2-b959-c0511caf75c3","size":4561838724,"currentStatus":"Archiving attachments.","alternativePercentage":"100%","concurrentBackupInProgress":false}
File to download: temp/filestore/0dc57fde-b470-44f2-b959-c0511caf75c3
And as you can see the backup was correctly downloaded and it is about 4.5GB (the size matches the one returned by the script):
$ ls -la
total 4454944
drwxr-xr-x 2 dbonotto dbonotto 4096 Feb 14 20:01 .
drwxr-xr-x 4 dbonotto dbonotto 4096 Dec 18 18:52 ..
-rw-rw-r-- 1 dbonotto dbonotto 4561838724 Feb 14 20:01 CONF-backup-20190214.zip
-rwxr-x--x 1 dbonotto dbonotto 2429 Feb 14 19:20 Confluence-backup.sh
Hi, I've been trying to integrate your script, I've entered all details required, but for some reason the script cannot authenticate with the Confluence Cloud instance. It keep responding fir 401 error. The instance I am trying to backup is on Confluence Cloud.
I tried various username/password combinations: - email + password - email + token - username + password - username + token
I am the admin of the instance.
Any advise would be appreciated.
edit: I can confirm that the jiracookie file is being created. I have remove the -s from the curl command line and this is the output
The attached script only works with Cookie Authentication (that is going to be deprecated soon).
I will try to update the script to use API Tokens as soon as I have time. In the meanwhile please notice that you are getting 401 Unauthorized.
Now, the combination email/password to be used in the script are the same credentials that you can use to log-into id.atlassian.com
Can you confirm you are able to log-into id.atlassian.com using the same credentials?
Also, it happened to me in the past that I had to reset the password in id.atlassian.com to make this work. Can you try if the same procedure works for you?
Please let me know
I had to reset my password for the script to work. It Works! Having API token authentication would be fantastic.
Thank you, Dario.
YOOO!!! :)
Super happy to hear so.
Next week I should have some time to test a new script using API Tokens that I have already started writing. I don't want to set expectations but finger crossed and let's hope that everything will be fine ;)
Enjoy the weekend!
Dario
Hi Dario.
Thank you very much for your script, it is very useful for us. With impatience, we are waiting for the next version with support for the Tokens API.
Best wishes.
Hi Dario,
Thank you very much for the script. but for some reason, the script cannot authenticate with the Confluence Cloud instance. It keeps responding fir 401 error. The instance I am trying to backup is on Confluence Cloud.
Please let me know in case i am missing something in Authentication part.
@5ae8a8ced5c9632c3013bb70 ,
The above script has some requirements: 1. You must have Jira as well in order to be able to call the session endpoint 2. You need to provide email address and the password you use to access id.atlassian.com and not the API Token 3. In some cases, you need to reset the password in https://id.atlassian.com/manage-profile/security in order to be able to correctly authenticate
Can you confirm all the above requirements are matched?
Thanks, Dario
@"gitadmin@goodstream.eu" Thanks a lot, it's this kind of feedback that keep me motivated to continue maintaining these scripts! :)
I need some time to test the script and I will let you know how it goes asap!
Hi Dario,
Thanks a lot for your quick response yes i am using user name password in a script. after reset password, i am getting below changed error.
Devs-MacBook-Air:Downloads devpandey$ ./Confluence-backup.sh Starting the script... FAILED, IT RETURNED User may not create a backup.
Thanks, Dev
Hi Dev,
From the error message it looks like the user does not have rights to create a backup. Can you kindly make sure you have the proper rights?
Otherwise, just try to create a backup from the UI using the same credentials to log-in (it may be a good idea to try in an incognito/private browser window). It can be the case that either you don't have right or you have to wait for the backup timer to be cleared.
Also, everyone, I have just uploaded a slightly new version of this script: * https://bitbucket.org/atlassianlabs/automatic-cloud-backup/src/master/backup-confluence.sh
Can you kindly test it and confirm it works as expected (the API Token version will follow)?
I am closing this issue since I have just uploaded the new backup script for Confluence using API Tokens: https://bitbucket.org/atlassianlabs/automatic-cloud-backup/src/master/backup-confluence-api-token.sh
Please use that script instead and open a new issue in case of troubles with it.
Hi Dario,
I am using scripts backup-jira-api-token.sh and backup-confluence-api-token.sh to automate backups of my Atlassian cloud environment. Backup of JIRA is successful, and restore was successful as well. Backup of Confluence seems to be OK, but when I want to restore to my other cloud site, I get the error:
Import error
There was an error importing file CONF-backup-15-04-2019.zip: Validation failed. Following issues were reported:
Error importing data: org.xml.sax.SAXException: The XML document does not contain the <entity-engine-xml> root element or it was closed too early.
Do you have any suggestions or recommendation what to could be potential problem? Could it be that scripts for Confluence backup is not working properly?
Thanks, Elmin.
Please notice that this issue is closed and it is related to a different script than the one you are using.
As a quick answer I can tell you to check if the zip file has been created correctly since from the error message it looks like the importer is failing to find the entities.xml file (it is not supposed to be in any folder)
Also, Confluence restore is known to be broken if you have both, Jira and Confluence:
If this is not the case then please open a new issue and provide more details.
Thank you very much Dario for this answer, now I have a clear picture. I have both on the same subscription so that is the reason why my Confluence restore fails. Is there a way/script to automate backup of spaces one by one?
Best regards, Elmin.
Please open a new issue, this one is closed.
Thank you very much for your reply. New issue is open (Issue #21).
Why would you need a web browser to run a bash script?
Also, why would you add a question to an issue already marked as resolved?
Cookie-based auth for REST APIs is deprecated. API tokens worked for me:
API token is available at https://id.atlassian.com/manage/api-tokens.