Feature: Setting individual scenario state via Admin API

[noxfate]

Problem
I'm currently using static file mapping as the main source for development, integration-test and e2e testing with the same set of static file mapping. It has grow in size and number of files has become more difficult to manage, but the pros of it still appreciated. This have cause flakiness and harder to do mock in isolation. By forcing scenario state really help solve this problem.

I used to leverage stateful behaviour and manual setting individual state, not only for stability but for test data maintainability and make sure that the mock is in isolation as well. I do think this really help with mock data traceability and maintainability of the test.

Describe the solution you'd like
Java implementation of wiremock has this endpoint POST /__admin/scenarios/<scenario_name>/state available in admin api client. I was hoping this become available in this .Net version as well.
ref: https://wiremock.org/docs/stateful-behaviour/#setting-the-state-of-an-individual-scenario

Describe alternatives you've considered
We have tried to organise the static file by using "priority" but that become a nightmare to handle real quick.

Is your feature request supported by WireMock (java version)? Please provide details.
Yes. here is the clear doc https://wiremock.org/docs/stateful-behaviour/#setting-the-state-of-an-individual-scenario

Additional context
N/A

[StefH]

@noxfate
I think I've updated the code to support this.
Can you try preview version: WireMock.1.8.12-ci-110218

The API call is a PUT with body:

{
  "State": "the state"
}

(https://github.com/wiremock/WireMock.Net/wiki/MyGet-preview-versions)

[noxfate]

Thank you very much for a fast support! A couple notes

I cannot make it to work yet, seems like a problem not on the WireMock.Net implementation anymore but on the docker image that I was using. Im mainly testing on WireMock.Net.TestContainers package and the server that I tried to curl directly to is not that up-to-date. Any suggestions where I can take the correlate update of the image?

I tested with the image tag in the screenshot above sheyenrath/wiremock.net-alpine:1.8.10. This is the result from my debug directly postman to the testcontainers.

Meanwhile, I'll try to setup quickly on the WireMock standalone to help test your implementation.

Few thing that I hope its useful

1. C# docs seems to be misleading on scenario function
2. ResetScenariosAsync seems to call a wrong url. It should POST /__admin/scenarios/reset

[StefH]

There is no preview Docker image yet which contains this fix. I can try to make one.

1. Can you point me to that doc error? I think this was fixed in a newer preview (1.8.12-ci-110220)

2. There are 2 options to delete or reset a scenario: a POST and a DELETE.

[noxfate]

Thank you @StefH . ping me once you have that image, I'll help test it out

For

1. I see that all docs are "Delete (reset) all scenarios"....very minor.
   

2. I can see that, and i think its an upgrade to able to delete scenario. However, the ResetScenariosAsync is mapped to the wrong endpoint. I got some error when calling it and need a workaround like this.
   

[StefH]

@noxfate
I published Linux Docker version 1.8.13-preview-01, so to test this, you need to remove all WireMock.Net Docker images to be sure you get this one (the most recent).

[noxfate]

Okay, I just test with the new image. Looking good so far, I can successfully GET scenario mapping, PUT update it state and POST can reset it.

However, I noticed there're several difference between scenario setup from java wiremock version. Are these expected?

1. GET __admin/scenarios response is quite different than the java version (ref here). I was expecting to see $.scenarios[].possibleStates

2. There're no default Started state. I quickly setup my scenario like so

//  __admin/mappings/test-scenario-a.json
{
  "Scenario": "default-scenario-a",
  "WhenStateIs": "my expected state",
  ....
}

GET __admin/scenarios cannot pick it up. Therefore, cannot update it with new PUT __admin/scenarios/:name/state

3. I find this version we have auto completing the scenario. When I tried setup a full scenario, when reaching the final state the request. It becomes automatically completed. which is differ from what I recall with java version that will stuck at that state until reset or force into another state.

// response from GET __admin/scenarios
[
    {
        "Name": "default-scenario-A",
        "NextState": null,
        "Started": true,
        "Finished": true,
        "Counter": 0
    }
]

[StefH]

1

It's indeed different, I never build it to be 100% compatible.
However your remark sounds logical; can you create a new issue for this?

2

When a call have never been made to that mapping, the state is still null I think.
However I can almost understand what you try to do, but can you please explain in steps and more detail what you see and what you want?

3

As workaround you can set the number-of-requests to int.max
See https://github.com/wiremock/WireMock.Net/wiki/Scenarios-and-States#stay-in-the-same-state-for-a-number-of-requests

[noxfate]

I see thanks for taking your time to response. Actually, I don't have much concerns about (1). I rarely used those field and mostly for debug than actual use-case. What I found is much related to (2) that is what I actually wants as a feature.

for (2), I'd give you an example of my use case using this wiremock for automate UI test... I need a list of item for render the screen. Now the implementation on the UI requires to call GET /v1/items with no special parameters whatsoever.
But I would want to be able to control the data for many case and make my automate test cover more edge cases, like error state, delay loading, different dataset, empty dataset, etc. I would never able to achieve this without differentiate the stubbing with different userId, special query param, or hack with http headers in the request to make that happen. With this set state, allows me to create same requestMatching and easily controls the response on my test. Like so,

Feature: As a user, I can see list of items on item page
  Given  I open the page
  And my scenario is "default-scenario-A" with "my expected state" 
  When I open list items page
  Then I should see lists of item

Feature: As a user, I can see empty emoji when item is empty
  Given  I open the page
  And my scenario is "default-scenario-A" with "my empty state" 
  When I open list items page
  Then I should see empty list of items
  And I should see empty emojis

Now, the static file setup would actually never have only a requestMatcher for GET /v1/items, but also will have scenario setup annotate to it for each file separately. And got manually force state to update in the test as a part of test arrange. Static file would look like this.

//  __admin/mappings/v1-items_my expected state.json
{
  "Scenario": "default-scenario-A",
  "WhenStateIs": "my expected state",
  ....
}

//  __admin/mappings/v1-items_my empty state.json
{
  "Scenario": "default-scenario-A",
  "WhenStateIs": "my empty state",
  ....
}

[StefH]

OK. I think I understand your issue 2.

What you want to to define this (as static json file):

_server
  .Given(Request.Create()
    .WithPath("/todo/items")
    .UsingPost())
  .InScenario("To do list")
  .WhenStateIs("my expected state")
  .RespondWith(Response.Create()
    .WithBody("FILLED LIST"));


_server
  .Given(Request.Create()
    .WithPath("/todo/items")
    .UsingGet())
  .InScenario("To do list")
  .WhenStateIs("my empty state")
  .RespondWith(Response.Create()
    .WithBody("EMPTY LIST"));

So when you run the first UI Test

• set the state to "my expected state"
• run your test
• and you expect that WireMock.Net returns "FILLED LIST"

And when you run the second UI Test

• set the state to "my empty state"
• run your test
• and you expect that WireMock.Net returns "EMPTY LIST"

I hope, this should now be possible in 1.8.13-preview-02.

[noxfate]

tested, it works!...Although the adminApi.GetScenariosAsync(); returns empty scenarioName after I've called adminClient.PutScenarioStateAsync("To do list", new { State = "my expected state" });

[
    {
        "Name": null, // empty
        "NextState": "my expected state",
        "Started": false,
        "Finished": false,
        "Counter": 0
    }
]

[StefH]

@noxfate
Thanks for verifying.
I'll fix this last bug and then release a official NuGet + Docker.