Implement a URL checker as a CI job

Issue #609 wontfix
Daniel Scheffler created an issue

I recently implemented URL checkers into my repositories as CI jobs that validate URLs at each commit. I am using the urlchecker package that produces outputs like this:

https://git.gfz-potsdam.de/EnMAP/GFZ_Tools_EnMAP_BOX/EnPT/-/jobs/52536

Maybe you also want to consider this for the EnMAP Box.

Comments (18)

  1. Andreas Janz

    Hi Daniel, I have no experience with URL checkers. Never needed those. What are they good for? Where would we use this inside the EnMAP-Box?
    Maybe @Benjamin Jakimow knows more about this.

  2. Daniel Scheffler reporter

    Well, you just run something like urlchecker check . --file-types .py,.rst,.md,.json in the root directory of a package and the tool automatically finds all URLs in these file types in your package and checks if each one if it is still working or if is a dead link. In case there are dead links in your package, the above command returns the exitstatus 1 which can for example be interpreted as a failed CI job. Of course, it is also possible to define a white-list to exclude specific URLs.

    This way, I can automatically discover dead links shortly after they became dead.

    Here is a screenshot of the output in case you cannot access the above link:

  3. Andreas Janz

    @Benjamin Jakimow , for now, I would recommend that you include your offline URL check into the create_plugin.py script.

  4. Benjamin Jakimow

    I don’t think it’s a good idea. We don’t run other tests either. Creation and Testing should be separated (and in the best case the first is done automatically after the second has been completed without errors)

  5. Andreas Janz

    We don’t run other tests either.

    The poor one that builds a new release already has to do a lot of extra care work:

    In that sense, the URL check is just another point on our release checklist.

    But it’s not a crucial one, so skipping it is fine for me.

  6. Log in to comment