Add support for AsyncAPI kind

[deliahu]

Description

An AsyncAPI would allow users to submit single-request "jobs", and would provide an API to check the status/result of the prediction request.

Design doc: https://docs.google.com/document/d/18bN4-wBH4wF2J-M2biEVPgm_GPU2dWqV05tpca7I8zY/edit?usp=sharing

Implementation

Would use SQS as the queue, and base autoscaling on queue length

[lminer]

This would be a great feature for us!

[ospillinger]

Thank you for letting us know, we'll try to prioritize it for one of our upcoming releases. Would you be open to sharing a little bit more about your use case? I'd be happy to schedule some time to chat if you're open to it.

[lminer]

Thanks @ospillinger. We'd be happy to give some more details on our use case. I'm including @adamwylie and @MyNameIsURL as they have the best understanding of the details.

[ospillinger]

Sounds great, thank you! Could you please email me at omer@cortexlabs.com so that we can set up a call?