Link testing

[BlueHatbRit]

Link decay is often a problem with large lists like these, what do we think about a daily build with travis which checks all the URL's return with a 200 OK? Should be quite easy to write by parsing the readme into html and then cycling through all links making a request to each.

A down side is it would probably be quite slow. To combat that we could configure it to only run daily and not be a requirement for PR's. Although that would make it easier for people to add urls which don't work in the first place.

[PandelisZ]

htmlproofer does this https://github.com/gjtorikian/html-proofer

Could be setup with travis. We do this for our jekyll site

[PandelisZ]

Looks like this npm script does it specifically for markdowns. https://www.npmjs.com/package/markdown-link-check

If we want to start some automation on the repo then I guess node would be the way to go. Peter has some tools for auto generating the contents also.

[BlueHatbRit]

@PandelisZ that second library is really good, we could easily make a little node script with mocha which runs markdown-link-check on the main file. I'll get it done this afternoon and I'll try to hide it away in a directory so it doesn't clutter up the root. As you said we can then add a travis file and someone (maybe @joenash) can hook it up when they get a spare few moments.

[joenash]

Not a problem, I'll get this sorted! Good idea

[BlueHatbRit]

I'll see about whipping up a quick test tomorrow morning and adding a PR

[PandelisZ]

Cause I'm evil and I haven't quite found a package that does it out on the web, I constructed a regex match for bullet points that fit the schema of

- [Name](http url) - description

This is useful because as you saw with #10 sometimes mistakes can be made and also the format for links in markdown can be forgotten and entered in reverse. To avoid that we can check for the user.

Here is the regex its self. I don't know why I went ahead with this but I did.

(- )+([[])+([a-z])\w+]+\(+https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}\b([-a-zA-Z0-9@:%_\+.~#?&//=]*)+\)+( - )+([A-Za-z\-,\);.\(]+[\s])*

We could employ that to check that links submitted are in the correct format and provide valid urls and descriptions

[BlueHatbRit]

@PandelisZ I just shoved that into a regex tester to check if it works with sub-lists and got an error around ?&//=]*) I believe one of those slashes is meant to be a backslash resulting in:

(- )+([[])+([a-z])\w+]+\(+https?:\/\/(www\.)?[-a-zA-Z0-9@:%._\+~#=]{2,256}\.[a-z]{2,6}\b([-a-zA-Z0-9@:%_\+.~#?&\/=]*)+\)+( - )+([A-Za-z\-,\);.\(]+[\s])*