OutOfMemoryError

[robert-mygind]

Description

When running codegen I get an OutOfMemoryError

Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
at io.swagger.codegen.examples.ExampleGenerator.resolvePropertyToExample(ExampleGenerator.java:160)
at io.swagger.codegen.examples.ExampleGenerator.resolveModelToExample(ExampleGenerator.java:251)
at io.swagger.codegen.examples.ExampleGenerator.resolvePropertyToExample(ExampleGenerator.java:210)
at io.swagger.codegen.examples.ExampleGenerator.resolvePropertyToExample(ExampleGenerator.java:161)
at io.swagger.codegen.examples.ExampleGenerator.generate(ExampleGenerator.java:49)
at io.swagger.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:2195)
at io.swagger.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:864)
at io.swagger.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:791)
at io.swagger.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:418)
at io.swagger.codegen.DefaultGenerator.generate(DefaultGenerator.java:730)
at io.swagger.codegen.cmd.Generate.run(Generate.java:285)
at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)

Swagger-codegen version

Just checkout from github and build

Swagger declaration file content or url

https://api.reepay.com/swagger.json

Command line used for generation

java -jar modules/swagger-codegen-cli/target/swagger-codegen-cli.jar generate -i https://api.reepay.com/swagger.json -l php -o ~/Projects/reepay-php

Operation system and Java

OS X 10.11.6
java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

[wing328]

I was able to repeat the issue with similar errors:

[main] INFO io.swagger.codegen.AbstractGenerator - writing file /tmp/Projects/reepay-php/SwaggerClient-php/docs/Model/WebhookUpdateRequest.md
Exception in thread "main" java.lang.OutOfMemoryError: Requested array size exceeds VM limit
	at io.swagger.codegen.examples.ExampleGenerator.resolvePropertyToExample(ExampleGenerator.java:160)
	at io.swagger.codegen.examples.ExampleGenerator.resolveModelToExample(ExampleGenerator.java:251)
	at io.swagger.codegen.examples.ExampleGenerator.resolvePropertyToExample(ExampleGenerator.java:210)
	at io.swagger.codegen.examples.ExampleGenerator.generate(ExampleGenerator.java:49)
	at io.swagger.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:2188)
	at io.swagger.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:864)
	at io.swagger.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:791)
	at io.swagger.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:418)
	at io.swagger.codegen.DefaultGenerator.generate(DefaultGenerator.java:730)
	at io.swagger.codegen.cmd.Generate.run(Generate.java:285)
	at io.swagger.codegen.SwaggerCodegen.main(SwaggerCodegen.java:35)

I'll take a deeper look tomorrow by investigating the spec.

[dariota]

@wing328 The issue is in #/definitions/CardGatewayAgreement/properties/card_types/. It defines maxItems as MAX_INT, which causes that issue here for obvious reasons.

I suspect a fix may be to simply cap the number of items in an example array - there's probably not much value in a 2 billion item example.

[aiham]

I've hit this with the javascript language. Had to revert to v2.2.2 because v2.2.3 is broken because of #5845

[wing328]

@dariota thanks for pointing out the exact root cause of the issue. May I know if you've time to contribute a PR with the fix?

[dariota]

Not until mid January, but I can take it up then.

[xmeng1]

typescript-angular same issue

[atbe]

Python codegen same issue

Affected versions:

• 2.3.1
• 2.3.0

[xmeng1]

according to my test (MacOS 10.13.3)
if I assign more than 8G for VM (such as -Xmx9g, -Xmx8200m, less than 8200m may lead to Exception in thread "main" java.lang.OutOfMemoryError: Java heap space, strange), the max of the object array can be 2147483645 (Integer.MAX_VALUE - 2). but I think setting the size of the object in the example too high is not necessary. Add the code as follow before this line can solve the issue. However, I am not sure is it better to limit the Object array size here, or there will be other issues occur. @wing328 What do you think?

if(arrayLength>10000) {
    arrayLength=10000;
}

The simple code to test the Obejct array size limit:

public class ObjectArraySizeTestHighXmx {
    public static void main(String args[]) {
        Integer arrayLength = Integer.MAX_VALUE;
        Object[] objectProperties = null;

        // objectProperties = new Object[2396729];
        // System.out.print(objectProperties.length);
        // step = 1
        for (int i=arrayLength; i>1;i=i-1) {
            try {
                objectProperties = new Object[i];
            } catch (OutOfMemoryError e) {
                System.out.print("Out of memory: " + String.valueOf(i) + "\n");
                // System.out.print(e.toString());
            }
            if (objectProperties != null) {
                System.out.print("Find the limit by step 1: " + String.valueOf(i) + "\n");
                System.out.print(objectProperties.length + "\n");
                break;
            }
        }
    }
}

run

javac ObjectArraySizeTestHighXmx.java
java -Xmx9g ObjectArraySizeTestHighXmx  > ObjectArraySizeTestHighXmx-xmx9g.log

the log file will be

Out of memory: 2147483647
Out of memory: 2147483646
Find the limit by 1: 2147483645
2147483645

[dariota]

I don't see why we shouldn't limit the size of the example - what benefit would you get from looking at a 2 billion item example array (with just generated default data in it) that you wouldn't get from a 10,000 (or even 100) item array?

Of course, it would be nice to replace this in-memory creation of a massive array with a lazily generated array that's written as it's produced and massively reduces the memory footprint, but you'd still have an absolutely gigantic file for little to no value (bonus points when you try to view it and crash your browser).