Implement a URL checker as a CI job

[Daniel Scheffler]

I recently implemented URL checkers into my repositories as CI jobs that validate URLs at each commit. I am using the urlchecker package that produces outputs like this:

https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/jobs/52536

Maybe you also want to consider this for the EnMAP Box.

[Andreas Janz]

Hi Daniel, I have no experience with URL checkers. Never needed those. What are they good for? Where would we use this inside the EnMAP-Box?
Maybe @Benjamin Jakimow knows more about this.

[Daniel Scheffler]

Well, you just run something like urlchecker check . --file-types .py,.rst,.md,.json in the root directory of a package and the tool automatically finds all URLs in these file types in your package and checks if each one if it is still working or if is a dead link. In case there are dead links in your package, the above command returns the exitstatus 1 which can for example be interpreted as a failed CI job. Of course, it is also possible to define a white-list to exclude specific URLs.

This way, I can automatically discover dead links shortly after they became dead.

Here is a screenshot of the output in case you cannot access the above link:

‌

[Andreas Janz]

Alright, now I get it

[Benjamin Jakimow]

added unittest to check urls (to be completed). addresses #609

Signed-off-by: Benjamin Jakimow [e-mail] [e-mail]

→ <<cset ce089f309735>>

[Benjamin Jakimow]

addresses #609

→ <<cset 11faf08305cf>>

[Benjamin Jakimow]

addresses #609

→ <<cset f0b8da7321bb>>

[Andreas Janz]

Can we close this?

[Andreas Janz]

Benjamin, please decide if we can close this.

[Benjamin Jakimow]

so far not implemented as CI job_._ Feel free to implement it

[Andreas Janz]

@Benjamin Jakimow , for now, I would recommend that you include your offline URL check into the create_plugin.py script.

[Benjamin Jakimow]

I don’t think it’s a good idea. We don’t run other tests either. Creation and Testing should be separated (and in the best case the first is done automatically after the second has been completed without errors)

[Andreas Janz]

We don’t run other tests either.

The poor one that builds a new release already has to do a lot of extra care work:

In that sense, the URL check is just another point on our release checklist.

But it’s not a crucial one, so skipping it is fine for me.

‌

[Andreas Janz]

URL checking is now part of the release checklist.