Implement a URL checker as a CI job
I recently implemented URL checkers into my repositories as CI jobs that validate URLs at each commit. I am using the urlchecker package that produces outputs like this:
https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/jobs/52536
Maybe you also want to consider this for the EnMAP Box.
Comments (18)
-
-
reporter Well, you just run something like
urlchecker check . --file-types .py,.rst,.md,.json
in the root directory of a package and the tool automatically finds all URLs in these file types in your package and checks if each one if it is still working or if is a dead link. In case there are dead links in your package, the above command returns the exitstatus 1 which can for example be interpreted as a failed CI job. Of course, it is also possible to define a white-list to exclude specific URLs.This way, I can automatically discover dead links shortly after they became dead.
Here is a screenshot of the output in case you cannot access the above link:
-
Alright, now I get it
-
added unittest to check urls (to be completed). addresses
#609Signed-off-by: Benjamin Jakimow benjamin.jakimow@geo.hu-berlin.de benjamin.jakimow@geo.hu-berlin.de
→ <<cset ce089f309735>>
-
addresses
#609→ <<cset 11faf08305cf>>
-
addresses
#609→ <<cset f0b8da7321bb>>
-
Can we close this?
-
-
assigned issue to
- changed version to 3.9
Benjamin, please decide if we can close this.
-
assigned issue to
-
so far not implemented as CI job_._ Feel free to implement it
-
- changed status to on hold
-
@Benjamin Jakimow , for now, I would recommend that you include your offline URL check into the create_plugin.py script.
-
I don’t think it’s a good idea. We don’t run other tests either. Creation and Testing should be separated (and in the best case the first is done automatically after the second has been completed without errors)
-
- removed responsible
-
We don’t run other tests either.
The poor one that builds a new release already has to do a lot of extra care work:
In that sense, the URL check is just another point on our release checklist.
But it’s not a crucial one, so skipping it is fine for me.
-
- removed version
-
- changed milestone to Future Release - Nice to have
-
- changed status to open
-
- changed status to wontfix
URL checking is now part of the release checklist.
- Log in to comment
Hi Daniel, I have no experience with URL checkers. Never needed those. What are they good for? Where would we use this inside the EnMAP-Box?
Maybe @Benjamin Jakimow knows more about this.