[scala] client escapes terms which may be common in models, with no serialization fixes

[jimschubert]

Description

The Scala client codegen escapes terms which may be fairly common in response models (path, lazy, match, type). These are prefixed with an underscore with no serialization fix to account for the escaped value.

This is probably an issue in other generators as well.

Swagger-codegen version

2.2.3

Swagger declaration file content or url

{
    "swagger": "2.0",
    "info": {
        "version": "1.0.0",
        "title": "Reserved Words Issue",
        "description": "A sample of reserved words being escaped and breaking serialization",
        "termsOfService": "http://swagger.io/terms/",
        "contact": {
            "name": "Jim Schubert"
        },
        "license": {
            "name": "MIT"
        }
    },
    "host": "example.com",
    "basePath": "/api",
    "schemes": [
        "http"
    ],
    "consumes": [
        "application/json"
    ],
    "produces": [
        "application/json"
    ],
    "paths": {
        "/hi": {
            "get": {
                "description": "Returns all examples",
                "produces": [
                    "application/json"
                ],
                "responses": {
                    "200": {
                        "description": "A list of examples.",
                        "schema": {
                            "type": "array",
                            "items": {
                                "$ref": "#/definitions/Example"
                            }
                        }
                    }
                }
            }
        }
    },
    "definitions": {
        "Example": {
            "type": "object",
            "required": [
                "id",
                "name",
                "path"
            ],
            "properties": {
                "id": {
                    "type": "integer",
                    "format": "int64"
                },
                "name": {
                    "type": "string"
                },
                "path": {
                    "type": "string"
                },
                "lazy": {
                    "type": "boolean"
                },
                "match": {
                    "type": "integer",
                    "format": "int64"
                },
                "type": {
                    "type": "string"
                }
            }
        }
    }
}

Command line used for generation

swagger-codegen generate -l scala -i swagger.json -o example

Steps to reproduce

1. Save above spec as swagger.json
2. Run above command to generate Scala client
3. Inspect the model (cat example/src/main/scala/io/swagger/client/model/Example.scala):

package io.swagger.client.model


case class Example (
  id: Long,
  name: String,
  _path: String,
  _lazy: Option[Boolean],
  _match: Option[Long],
  _type: Option[String]
)

Related issues/PRs

Suggest a fix/enhancement

Scala allows these reserved words to be escaped in code. Model properties requiring escaping can easily be done like so:

case class Example (
  id: Long,
  name: String,
  `path`: String,
  `lazy`: Option[Boolean],
  `match`: Option[Long],
  `type`: Option[String]
)

Another option would be to annotate each escaped property with the unescaped property name, e.g. @JsonProperty("path"). However, this assumes only JSON serialization. This shouldn't be a huge issue, considering the ApiInvoker for the Scala generator assumes JSON serialization.

I'd imagine the escapes would be a better solution, especially for anyone who may only be generating models without API implementations.

[emtiazahmed]

Any fixes or workarounds for this. I am facing the same issues. All the fields in my json which match keyword are not deserialized because of missing annotations in the generated case classes

[wing328]

@emtiazahmed Would you have time to contribute the fix?

https://github.com/swagger-api/swagger-codegen/blob/master/modules/swagger-codegen/src/main/resources/scala/model.mustache#L15 is a good starting point.

[jimschubert]

@emtiazahmed

As a work around, you can clone this repo, git checkout v2.2.3 (or the branch for your version of swagger-codegen), and edit the file linked by @wing328
(modules/swagger-codegen/src/main/resources/scala/model.mustache to the following:

{{>licenseInfo}}
package {{package}}

import com.fasterxml.jackson.annotation.JsonProperty

{{#imports}}
import {{import}}
{{/imports}}

{{#models}}
{{#model}}
case class {{classname}} (
  {{#vars}}
  {{#description}}
  /* {{{description}}} */
  {{/description}}
  @JsonProperty("{{{baseName}}}") {{{name}}}: {{^required}}Option[{{/required}}{{datatype}}{{^required}}]{{/required}}{{#hasMore}},{{/hasMore}}
  {{/vars}}
)

{{/model}}
{{/models}}

Then, target the modified template's directory as the custom template directory:

swagger-codegen generate -t /path/to/_/src/main/resources/scala -l scala -i swagger.json -o example

This should give you the explicit @JsonProperty annotation I mentioned. I haven't yet opened a PR for this because I don't think it's an ideal solution. I'd like to add a property to the codegen model's vars object, such as isReservedWord which would allow us to change that relevant line to something like this:

{{#isReservedWord}}`{{/isReservedWord}}{{{name}}}{{#isReservedWord}}`{{/isReservedWord}}: {{^required}}Option[{{/required}}{{datatype}}{{^required}}]{{/required}}{{#hasMore}},{{/hasMore}}

The reason I think this would be helpful is because it could be used in templates across languages. For example, in C# server generators, it's not uncommon to have a property like class. In C# this is a reserved word but the language allows escaping as @class.

---

For reference, another "workaround" would be to do what my team has done. Generate the code with the incorrect property names, then fix them manually and add a comment to these manually adjusted files. Then add the manually modified files to .swagger-codegen-ignore with a HACK or TODO comment to remove after this bug is resolved.

[akkie]

If I'm not wrong, then every reserved word in Scala can be escaped on every position(val, var, class, def) by enclosing the word with grave accents. So the following code is completely valid Scala code:

case class `class` (
  id: Option[Long],
  category: Option[Category],
  name: String,
  photoUrls: Seq[String],
  tags: Option[Seq[Tag]],
/* pet status in the store */
  status: Option[String],
/* Test for enum values */
  color: Option[`class`.Color],
/* Test for reserved values */
  `type`: Option[String])

/**
 * The companion object.
 */
object `class` {
  import ai.x.play.json.Jsonx
  import play.api.libs.json.OFormat
  import com.mohiva.swagger.codegen.core.ApiJsonFormats.enumValueFormat

  /**
   * The `Color` enum.
   */
  type Color = Color.Value
  object Color extends Enumeration {
    val Black = Value("black")
    val White = Value("white")
    val Brown = Value("brown")
  }

  def `implicit`: `class` = ???

  def `def`: Int = 1

  /**
   * Converts a [[Test]] class into a JSON object.
   */
  implicit val `val`: OFormat[`class`] = Jsonx.formatCaseClass[`class`]
}

The DefaultCodegen.escapeReservedWord method allows us to override the escaping for reserved words:

@Override
public String escapeReservedWord(String name) {
    return "`" + name + "`";
}

But then the generated code looks like:

`type`: Option[String])

So the solution would be to override the Mustache escaper with a custom one:

/**
 * Overrides the default Mustache escaper with a custom one that allows to escape variables with grave accents.
*/
@Override
public Mustache.Compiler processCompiler(Mustache.Compiler compiler) {
    Mustache.Escaper SCALA = text -> {
        // The given text is a reserved word which is escaped by enclosing it with grave accents. If we would
        // escape that with the default Mustache `HTML` escaper, then the escaper would also escape our grave
        // accents. So we remove the grave accents before the escaping and add it back after the escaping.
        if (text.startsWith("`") && text.endsWith("`")) {
            String unescaped =  text.substring(1, text.length() - 1);
            return "`" + Escapers.HTML.escape(unescaped) + "`";
        }

        // All none reserved words will be escaped with the default Mustache `HTML` escaper
        return Escapers.HTML.escape(text);
    };

    return compiler.withEscaper(SCALA);
}

[ramzimaalej]

@jimschubert can you confirm that this has been fixed by this PR: mohiva/swagger-codegen-play-scala#12

cc @wing328

[jimschubert]

@ramzimaalej that change looks much more reasonable than my suggestion. It'll be a few days before I can evaluate generated outputs.

Any plan to open a PR with that change against this repo?

[ramzimaalej]

@jimschubert yes, the plan is to clear the scala issue list as soon as possible. It might not come before the end of the year though.