Add workflow to automatically generate and commit site HTML

[nchammas]

As proposed here, I believe it would be best to overhaul the workflow for this project by splitting the source into a master branch, and having asf-site be just for the generated HTML. We could even have per-branch staging sites generated and published automatically for us by ASF.

However, this PR takes a more conservative approach as suggested by @cloud-fan and reworks the GitHub Actions workflows as follows:

• html-build.yml: Added a new workflow that runs on PRs and simply generates the docs. This confirms the Jekyll build is not obviously broken.
• html-push.yml: Added a new workflow that runs on new commits to asf-site. It generates the docs and pushes the generated HTML automatically as a new commit to asf-site.
• doc_gen.yml: Deleted since it is subsumed by the new workflows.

This PR also removes the content/ symlink to site/, which doesn't appear to be needed. ASF understands Jekyll configs and looks here to know where to serve the site from:

spark-website/_config.yml

Line 6 in fd110d9

destination: site

TODO:

• Remove current branch from push workflow.
• Confirm ASF destination config with committer.

[nchammas]

I need a maintainer to approve the workflow runs so I can test my changes.

[nchammas]

Thank you to whoever approved the previous workflows. I need another approval.

Could I perhaps be added to some workflow whitelist so maintainers don't have to manually approve my workflows every time?

[nchammas]

Looks like everything is working. New [html] commits are not being pushed anymore because there are no changes to the HTML output. If I make a change to the source that changes the output HTML, we will see a new [html] commit pushed to my branch.

This is just test behavior. Before merging, I will remove it so that [html] commits are only pushed to asf-site when necessary.

[cloud-fan]

so PR only need to modify md files, once PR is merged, a post-merge job will regenerate html and push? how do we handle concurrency? like 2 PRs merged almost at the same time?

[nchammas]

so PR only need to modify md files, once PR is merged, a post-merge job will regenerate html and push?

Correct.

how do we handle concurrency? like 2 PRs merged almost at the same time?

There should be no problem.

1. PR A is merged and pushes commit C1 to asf-site.
2. Immediately after, PR B is merged and pushes commit C2 to asf-site.
3. The HTML workflow for commit C1 fails when it tries to push an HTML commit because its checkout is stale. C2 has already been pushed.
4. The HTML workflow for commit C2 succeeds and includes the HTML for both commits C1 and C2.

To be a bit cleaner and avoid the spurious workflow failure in this scenario, I've added a concurrency group so that the workflow for commit C1 would be canceled immediately once the workflow for commit C2 starts.

[nchammas]

Could I perhaps be added to some workflow whitelist so maintainers don't have to manually approve my workflows every time?

@cloud-fan - Checking in again about this. Not absolutely required, obviously, but it would make it easier for me to contribute here.

I understand there is a repo setting to require workflow approval only for first-time contributors to a repo. You can also add me specifically as a read-only repo collaborator.