Tracking issue to upgrade to the latest Hugo when markdown rendering issues have been addressed

[justinvp]

Recent versions of Hugo have bugs in the markdown renderer (Blackfriday) that prevents fenced code from rendering correctly in lists when a language is specified. Many of our tutorials are made up of ordered lists of steps, each step containing a code snippet.

• Regression: a list item is nested after a code block with info-string russross/blackfriday#484
• #v2 fenced code in list renders differently with language specified russross/blackfriday#526

Until these issues are addressed, we've pinned to Hugo v0.55.4, which is the latest release that does not contain the regression.

We should ensure these issues are addressed and adopted by Hugo so we can stay current with the latest versions of Hugo.

[leezen]

I had assigned myself #2380 and was looking into what was going on. Which led to me this issue.

I ended up trying the latest version of Hugo, which is using blackfriday 1.5.3. The nested code we're generating now certainly seems better with Hugo 0.65.3 vs. the pinned version from what I can tell.

Hugo 0.55.4

vs.

Hugo 0.65.3

Are there other pages we've seen in the past that suffered from this issue? It'd be nice to check them and if they seem fine, let's update so that our tutorials look better? I did notice thatinstall-language-runtime.md doesn't seem to build correctly anymore and would need to debug what's going on there.

[justinvp]

The issues were mostly in tutorial content, where we have lists and code fences within lists. When we ported the docs from Jekyll to Hugo, we had some tutorial content that I had to manually tweak to workaround similar rendering issues as in the screenshot above. This involved tweaking the content to not use code fences. One way I worked around it was by removing the code fence and just indenting the content, which does make it render as a code block (albeit, without a specified language for syntax highlighting). In more recent cases, I realized the {{< highlight >}} shortcode could be used as an alternative workaround that allows the code language to be specified. There were also problems around ordered lists, where the numbers would reset to 1 instead of continuing to increment.

In any case, these workarounds worked as of Hugo v0.55.4, but later versions of Hugo adopted newer versions of Blackfriday, which contained new bugs (linked to in the description of this issue above) which caused rendering regressions.

And since then, we're now generating tutorial content from our examples repo, and the markdown from those example READMEs does not have any workarounds in place, hence the poor rendering for #2380.

But it's been many months since I've tried more recent versions of Hugo. If the latest version fixes these issues, we should definitely move to it!

We've also had to deal with other Blackfriday quirks. It doesn't render markdown within HTML blocks, so we've had to workaround this by wrapping markdown content inside HTML blocks with a {{% md %}} ... {{% /md %}} shortcode. But this has downsides: the rendered markdown is always wrapped in a <p></p> -- which isn't always desirable.

I mentioned on Slack the other day that Hugo, as of v0.60.0, is now using Goldmark rather than Blackfriday as the default markdown renderer: https://gohugo.io/news/0.60.0-relnotes/

Goldmark is CommonMark compliant, so if we moved to that, we should be able to also move away from the {{% md % }} shortcode approach to rendering markdown inside HTML blocks, using the standard CommonMark approach noted at https://stackoverflow.com/a/29378250

It'd be interesting to try building the site with the latest version of Hugo with Goldmark enabled, to see how it fares. We'd have to go through the Config to make sure we're not losing anything significant switching to Goldmark.

It'd be worth building the site with v0.55.4, and then with the latest version with Goldmark (or still with Blackfriday, if that makes it easier as a first step to moving to the latest version of Hugo), and then diff the generated HTML to see what the changes look like.

[justinvp]

Note: If we do end up moving forward with a Hugo upgrade, in addition to pinning to the more recent version in this repo, we'll also need to update the version that Pulumify uses here: https://github.com/pulumi/actions-pulumify/blob/43f558cc4749b6444102d578d72f03981dedc6ed/Dockerfile#L19-L21

[praneetloke]

I brought this into M33 to investigate if upgrading Hugo was our only option to workaround an OOM issue we were hitting on Travis with new resource docs. So I did a quick POC of upgrading to 0.63.0 and things looked good mostly. There were a few issues, but moving this to M34 so that we can focus on this while not in the midst of rolling out the resource docs.