Ease reusability of the template

[dafyddj]

PR progress checklist (to be filled in by reviewers)

• Changes to documentation are appropriate (or tick if not required)
• Changes to tests are appropriate (or tick if not required)
• Reviews completed

---

What type of PR is this?

Primary type

• [build] Changes related to the build system
• [chore] Changes to the build process or auxiliary tools and libraries such as documentation generation
• [ci] Changes to the continuous integration configuration
• [feat] A new feature
• [fix] A bug fix
• [perf] A code change that improves performance
• [refactor] A code change that neither fixes a bug nor adds a feature
• [revert] A change used to revert a previous commit
• [style] Changes that do not affect the meaning of the code (white-space, formatting, missing semi-colons, etc.)

Secondary type

• [docs] Documentation changes
• [test] Adding missing or correcting existing tests

Does this PR introduce a BREAKING CHANGE?

Yes.
All state names and state ids are changed to the new unique token.

Related issues and/or pull requests

#98 slightly

Describe the changes you're proposing

While creating my first formula using this template, I found it wasn't as straightforward as I'd hoped.
Obviously the word template is used all over this formula and some need changing and some don't.

This is just an idea I had of uniquely marking out the replaceable template by using the token template__ - of course, it could be something else too.

Then a simple global replace can be performed while converting the template to a real formula.

Thoughts before I continue further?

Pillar / config required to test the proposed changes

Debug log showing how the proposed changes work

Documentation checklist

• Updated the README (e.g. Available states).
• Updated pillar.example.

Testing checklist

• Included in Kitchen (i.e. under state_top).
• Covered by new/existing tests (e.g. InSpec, Serverspec, etc.).
• Updated the relevant test pillar.

Additional context

[myii]

Thanks for starting this PR, @dafyddj. It's an interesting take on things, definitely worth a discussion. I'd like to start by pulling in other contributors who have experience with conversions using the template-formula (based on previous discussions). Hopefully, we can formulate an effective solution to make this process as efficient as possible.

CC: @saltstack-formulas/wg @xenophonf @GMAzrael.

Some alternatives to compare against:

1. Provide a script to convert this formula according to cookiecutter-style input after cloning.
2. Prepare detailed documentation to perform the conversion after cloning.
3. Have a separate repo that's effectively a "pure" template (not numerous things mixed together, as we have in this repo).

[myii]

@dafyddj Thanks for the ongoing effort here, this will definitely be useful in resolving this problem. I hope others can share their opinions, so that we select the best approach going forwards.

[daks]

I think that in the middle/long term it should be nice to have a tool/script to generate a formula from scratch like other config management tools have. Maybe cookiecutter can be used for it but I don't really know it so I can't judge if it'll work for our needs.
And maybe we already have a certain stability in this formula to work on it already.

In the meantime (until someone take this job or we consider this a priority, maybe we already should?) manual generation (like documented in my PR) can be nice, and if this PR help it, I'm ok with it.

[myii]

@dafyddj The script looks great, this is really taking shape. Some comments:

• Since our formulas are consumed across various OSes, can we make this as portable as possible (i.e. POSIX standard)?
  • sh rather than bash
  • Any alternative to perl, such as sed
• Apply the improvements suggested by https://www.shellcheck.net/
  • Of course, ignoring wherever necessary
  • As a side note, this is integrated by default into Travis CI, which I think would be a great addition to the lint stage -- thoughts?

[myii]

@dafyddj I've started #181, with the plan to propagate using shellcheck to all of the formulas. Ignores are simple enough where required.

[myii]

@dafyddj Thanks for moving over to sh (and for ex!). A quick question, not prescriptive: what are your thoughts on using uppercase TEMPLATE instead of template__? I haven't thought this through but it just occurred to me, so I thought I'd ask.

[dafyddj]

I'm not particularly attached to template__ and I think TEMPLATE could work because the search/replace is case-sensitive. Would we ever have a legitimate need to use TEMPLATE in the formula? Probably not.

Also, do you agree that this conversion process should be CI tested? I'm thinking, in Travis, clone the repo, run the script, then run at least one of the test suites in that cloned repo. Thoughts?

[myii]

Also, do you agree that this conversion process should be CI tested? I'm thinking, in Travis, clone the repo, run the script, then run at least one of the test suites in that cloned repo. Thoughts?

@dafyddj Definitely useful and worthwhile. A single job would be enough to test that.

[dafyddj]

@myii this is mostly done as far as I'm concerned.

It could probably do with some documentation. Can you suggest where best to place the docs - and should the script remove that part of the documentation when it's done?

[myii]

@myii this is mostly done as far as I'm concerned.

Great, I'm going to take this for a test drive soon, since I need to make a formula or two. That will help me finalise the review.

It could probably do with some documentation. Can you suggest where best to place the docs - and should the script remove that part of the documentation when it's done?

The README is a good place. A couple of places to consider:

• A sub-heading under https://github.com/saltstack-formulas/template-formula#special-notes
• Or since this repo is a template, we could use a section right at the very top, before the general notes

Since the end-user is going to be editing the README for their own purposes, I don't think the script needs to remove this. I'll confirm this after my tests.

[myii]

Suggestion:

  git commit --quiet --all \
    --message 'feat: convert `template-formula` to `'"${NEW_NAME}"'-formula`' \
    --message 'BREAKING CHANGE: changed all state names and ids'

[myii]

In my first local tests, the script doesn't get past this line. Since this is working in Travis, there's probably something in the way I've patched things together. The only way past it is to comment out line 3, i.e.:

# set -o errexit # If a command fails exit the whole script

[dafyddj]

You might get some clues by running as (if you haven't already done so):
DEBUG=true bin/convert-formula.sh ...

[myii]

Yes, I started with that and then tried all sorts, including removing all of the quiet or silent flags but to no avail. Even when I get it running with this line commented out, there are no warning given. Not sure what's happening with that ex command to cause an error status...

[myii]

@dafyddj This sounds a little strange but I found a solution (using -u NONE) based on this reference:

• https://unix.stackexchange.com/questions/14497/why-would-vim-return-non-zero-exit-code-if-i-exit-immediately-after-opening

So that's:

  grep --recursive --files-with-matches --exclude-dir=.git TEMPLATE . \
    | xargs -L 1 ex -u NONE -sc '%s/TEMPLATE/'"${NEW_NAME}"'/g|x'

• This actually worked, with set -o errexit re-enabled.
• Maybe the end user (like me) could have something strange in their configuration, affecting ex when it exits?

[myii]

Suggested change

	git rm --quiet bin/convert-formula.sh AUTHORS.md CHANGELOG.md
	git rm --quiet bin/convert-formula.sh AUTHORS.md CHANGELOG.md \
	docs/_static/css/custom.css docs/AUTHORS.rst docs/CHANGELOG.rst \
	docs/conf.py docs/CONTRIBUTING_DOCS.rst docs/index.rst

[myii]

Current status:

1. Discuss and finalise changes as suggested above.
2. Add notes to the README, including explaining that git push -f needs to be used and perhaps how to configure Travis to enable semantic-release (e.g. GH_TOKEN, etc.).

A wish-list collection for the long-run, doesn't prevent merging this PR:

1. README: Update the headings symbols to match the length of the heading itself.
2. .rubocop.yml: Remove the Metrics/BlockLength section, specific to the template-formula.
3. .travis.yml: Remove the env: 'Conversion' block.
4. FORMULA: reset the version to 1.0.0.

[myii]

@dafyddj Merged, the rest of the issues can be covered in a future PR. Thanks for this fantastic PR.

[saltstack-formulas-travis]

🎉 This PR is included in version 4.0.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀