Enhancement : SEO of the documentation by google/yahoo/bing indexer...

[mcarbonneaux]

adding minimal robot.txt and generating sitemap.xml based on url list like in the start page.
in that way all search about haproxy configuration in google/bing point correctly to this pages... because the documentation while be indexed...

• https://www.sitemaps.org/fr/
• http://robots-txt.com/
• http://robots-txt.com/ressources/robots-txt-sitemap-xml/

Minimal robot.txt:

User-agent: *
Disallow:
Sitemap: https://cbonte.github.io/haproxy-dconv/sitemap.xml

sample sitemap (generate it at the same time of head page : https://cbonte.github.io/haproxy-dconv/):

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.6/intro.html</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.6/configuration.html</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.6/management.html</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.5/intro.html</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.5/configuration.html</loc>
   </url>
   <url>
      <loc>https://cbonte.github.io/haproxy-dconv/2.5/management.html</loc>
   </url>

....


</urlset> 

use https github in place on http (google prefere https...) every where in the documentation and in about.
https://cbonte.github.io/haproxy-dconv/ in place of http://cbonte.github.io/haproxy-dconv/

submit the sitemap url (https://cbonte.github.io/haproxy-dconv/sitemap.xml) to google/bing search console:

• https://search.google.com/search-console
• https://www.bing.com/webmasters/about