API v2 design

[c8y3]

I am starting this discussion about the REST API v2 design.

Here are some rules the new REST API should adhere to :

• use verbs POST to create, GET to read, PUT to update and DELETE to remove
• resources which return collections should end with an 's'
• resources which return collections should be paginated
• resources which return collections should include fields total, data, last_page, current_page and next_page
• in case of success, GET and PUT endpoints should return code 200 and return the object
• in case of success, POST endpoints should return code 201 and the newly created object
• in case of success, DELETE endpoints should return code 204 and an empty body
• in case an object is not found, GET endpoints should return code 404
• in case of some other error, endpoints should return 400 and an object which contains at least a field message with the error message
• enpoints' urls should not contain verbs
• identifiers (in query parameters, body and results) should have type integer
• DELETE endpoints should not have any body
• given an url /api/v2/resources, the object returned with POST, should have the same type as the objects returned in GET /api/v2/resources and the object returned in PUT/GET /api/v2/resources/{identifier}

[c8y3]

TODO

Global tasks:

• GET /global/tasks/list => GET /global-tasks
• POST /global/tasks/add => POST /global-tasks
• GET /global/tasks/<int:cur_id> => GET /global-tasks/{identifier}
• POST /global/tasks/update/<int:cur_id> => PUT /global-tasks/{identifier}
• POST /global/tasks/delete/<int:cur_id> => DELETE /global-tasks/{identifier}

Alerts comments:

• GET /alerts/<int:alert_id>/comments/list => GET /api/v2/alerts/{alert_identifier}/comments
• POST /alerts/<int:alert_id>/comments/add => POST /api/v2/alerts/{alert_identifier}/comments
• GET /alerts/<int:alert_id>/comments/<int:com_id> => GET /api/v2/alerts/{alert_identifier}/comments/{identifier} or alternatively GET /api/v2/comments/{identifier}
• POST /alerts/<int:alert_id>/comments/<int:com_id>/edit => PUT /api/v2/alerts/{alert_identifier}/comments/{identifier} or alternatively PUT /api/v2/comments/{identifier}
• POST /alerts/<int:alert_id>/comments/<int:com_id>/delete => DELETE /api/v2/alerts/{alert_identifier}/comments/{identifier} or alternatively DELETE /api/v2/comments/{identifier}

Assets comments:

• GET /case/assets/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/assets/{asset_identifier}/comments
• POST /case/assets/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/assets/{asset_identifier}/comments
• GET /case/assets/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/assets/{asset_identifier}/comments/{identifier}
• POST /case/assets/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/assets/{asset_identifier}/comments/{identifier}
• POST /case/assets/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/assets/{asset_identifier}/comments/{identifier}

Events comments:

• GET /case/timeline/events/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/events/{event_identifier}/comments
• POST /case/timeline/events/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/events/{event_identifier}/comments
• GET /case/timeline/events/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/events/{event_identifier}/comments/{identifier}
• POST /case/timeline/events/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/events/{event_identifier}/comments/{identifier}
• POST /case/timeline/events/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/events/{event_identifier}/comments/{identifier}

Tasks comments:

• GET /case/tasks/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/tasks/{task_identifier}/comments
• POST /case/tasks/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/tasks/{task_identifier}/comments
• GET /case/tasks/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/tasks/{task_identifier}/comments/{identifier}
• POST /case/tasks/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/tasks/{task_identifier}/comments/{identifier}
• POST /case/tasks/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/tasks/{task_identifier}/comments/{identifier}

Notes comments:

• GET /case/notes/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/notes/{note_identifier}/comments
• POST /case/notes/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/notes/{note_identifier}/comments
• GET /case/notes/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/notes/{note_identifier}/comments/{identifier}
• POST /case/notes/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/notes/{note_identifier}/comments/{identifier}
• POST /case/notes/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/notes/{note_identifier}/comments/{identifier}

Evidence comments:

• GET /case/evidences/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/evidences/{evidence_identifier}/comments
• POST /case/evidences/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/evidences/{evidence_identifier}/comments
• GET /case/evidences/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/evidences/{evidence_identifier}/comments/{identifier}
• POST /case/evidences/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/evidences/{evidence_identifier}/comments/{identifier}
• POST /case/evidences/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/evidences/{evidence_identifier}/comments/{identifier}

IoC comments:

• GET /case/ioc/<int:cur_id>/comments/list => GET /api/v2/cases/{case_identifier}/iocs/{ioc_identifier}/comments
• POST /case/ioc/<int:cur_id>/comments/add => POST /api/v2/cases/{case_identifier}/iocs/{ioc_identifier}/comments
• GET /case/ioc/<int:cur_id>/comments/<int:com_id> => GET /api/v2/cases/{case_identifier}/iocs/{ioc_identifier}/comments/{identifier}
• POST /case/ioc/<int:cur_id>/comments/<int:com_id>/edit => PUT /api/v2/cases/{case_identifier}/iocs/{ioc_identifier}/comments/{identifier}
• POST /case/ioc/<int:cur_id>/comments/<int:com_id>/delete => DELETE /api/v2/cases/{case_identifier}/iocs/{ioc_identifier}/comments/{identifier}

Activities:

• GET /activities/list => GET /api/v2/activities
• GET /activities/list-all => GET /api/v2/activities
• GET /case/activities/list => GET /api/v2/activities

Alert filters:

• GET /filters/<string:filter_type>/list => GET /api/v2/alerts-filters
• POST /filters/add => POST /api/v2/alerts-filters
• GET /filters/<int:filter_id> => GET /api/v2/alerts-filters/{identifier}
• POST /filters/update/<int:filter_id> => PUT /api/v2/alerts-filters/{identifier}
• POST /filters/delete/<int:filter_id> => DELETE /api/v2/alerts-filters/{identifier}

Note directories:

• GET /case/notes/directories/filter => GET /api/v2/cases/{case_identifier}/notes-directories

Customers:

• GET /manage/customers/list => GET /api/v2/manage/customers
• POST /manage/customers/add => POST /api/v2/manage/customers
• GET /manage/customers/<int:client_id> => GET /api/v2/manage/customers/{identifier}
• POST /manage/customers/update/<int:client_id> => PUT /api/v2/manage/customers/{identifier}
• POST /manage/customers/delete/<int:client_id> => DELETE /api/v2/manage/customers/{identifier}

Customers contacts:

• POST /manage/customers/<int:client_id>/contacts/add => POST /api/v2/customers/{customer_identifier}/contacts
• POST /manage/customers/<int:client_id>/contacts/<int:contact_id>/update => PUT /api/v2/customers/{customer_identifier}/contacts/{identifier}
• POST /manage/customers/<int:client_id>/contacts/<int:contact_id>/delete => DELETE /api/v2/customers/{customer_identifier}/contacts/{identifier}

Customers cases:

• GET /manage/customers/<int:client_id>/cases => GET /api/v2/customers/{customer_identifier}/cases

Notes revisions:

• GET /case/notes/<int:cur_id>/revisions/list => GET /api/v2/cases/{case_identifier}/notes/{note_identifier}/revisions
• GET /case/notes/<int:cur_id>/revisions/<int:revision_id> => GET /api/v2/cases/{case_identifier}/notes/{note_identifier}/revisions/{identifier}
• POST /case/notes/<int:cur_id>/revisions/<int:revision_id>/delete => DELETE /api/v2/cases/{case_identifier}/notes/{note_identifier}/revisions/{identifier}

Alert statuses:

• GET /manage/alert-status/list => GET /api/v2/manage/alert-status(es?)
• GET /manage/alert-status/<int:classification_id> => GET /api/v2/manage/alert-status(es?)/{identifier}

Severities:

• GET /manage/severities/list => GET /api/v2/manage/severities
• GET /manage/severities/<int:severity_id> => GET /api/v2/manage/severities/{identifier}

Event categories:

• GET /manage/event-categories/list => GET /api/v2/manage/event-categories
• GET /manage/event-categories/<int:cur_id> => GET /api/v2/manage/event-categories/{identifier}

Alert resolutions:

• GET /manage/alert-resolutions/list => GET /api/v2/manage/alert-resolutions
• GET /manage/alert-resolutions/<int:resolution_id> => GET /api/v2/alert-resolutions/{identifier}

Analysis statuses:

• GET /manage/analysis-status/list => GET /api/v2/manage/analysis-status(es?)
• GET /manage/analysis-status/<int:cur_id> => GET /api/v2/manage/analysis-status(es?)/{identifier}

Asset types:

• GET /manage/asset-type/list => GET /api/v2/manage/asset-types
• POST /manage/asset-type/add => POST /api/v2/manage/asset-types
• GET /manage/asset-type/<int:cur_id> => GET /api/v2/manage/asset-types/{identifier}
• POST /manage/asset-type/update/<int:cur_id> => PUT /api/v2/manage/asset-types/{identifier}
• POST /manage/asset-type/delete/<int:cur_id> => DELETE /api/v2/manage/asset-types/{identifier}

Case classifications:

• GET /manage/case-classifications/list => GET /api/v2/cases-classifications
• POST /manage/case-classifications/add => POST /api/v2/cases-classifications
• POST /manage/case-classifications/<int:classification_id> => GET /api/v2/cases-classifications/{identifier}
• GET /manage/case-classifications/update/<int:classification_id> => PUT /api/v2/cases-classifications/{identifier}
• POST /manage/case-classifications/delete/<int:classification_id> => DELETE /api/v2/cases-classifications/{identifier}

Evidence types:

• GET /manage/evidence-types/list => GET /api/v2/evidence-types
• POST /manage/evidence-types/add => POST /api/v2/evidence-types
• GET /manage/evidence-types/<int:evidence_type_id> => GET /api/v2/evidence-types/{identifier}
• POST /manage/evidence-types/update/<int:evidence_type_id> => PUT /api/v2/evidence-types/{identifier}
• POST /manage/evidence-types/delete/<int:evidence_type_id> => DELETE /api/v2/evidence-types/{identifier}

Traffic Light Protocol:

• GET /manage/tlp/list => GET /api/v2/manage/tlp or GET /api/v2/manage/tlp-colors
• GET /manage/tlp/<int:cur_id> => GET /api/v2/manage/tlp/{identifier} or GET /api/v2/manage/tlp-colors/{identifier}

Events:

• GET /case/timeline/events/list => GET /api/v2/cases/{case_identifier}/events

Case templates:

• GET /manage/case-templates/list => GET /api/v2/manage/case-templates
• POST /manage/case-templates/add => POST /api/v2/manage/case-templates
• POST /manage/case-templates/update/<int:cur_id> => PUT /api/v2/manage/case-templates/{identifier}
• POST /manage/case-templates/delete/<int:case_template_id> => DELETE /api/v2/manage/case-templates/{identifier}

Profile:

• GET /user/whoami => GET /api/v2/me (remove GET api/v2/auth/whoami)
• POST /context/set => PUT /api/v2/me
• POST /user/update => PUT /api/v2/me
• GET /user/theme/set/{string} => PUT /api/v2/me
• GET /user/deletion-prompt/set/{string} => PUT /api/v2/me
• GET /user/mini-sidebar/set/{string} => PUT /api/v2/me
• GET /user/token/renew => PUT /api/v2/me/api-key
• GET /user/tasks/list => GET /api/v2/me/tasks : returns all case tasks assigned to the current user
• GET /user/reviews/list => GET /api/v2/me/case-reviews : returns all cases with a review request for the current user

[lowlandghost]

Would there maybe be benefit of having the various types, classes, etc be under a "taxonomy" path vs bare? I.e. /api/v2/taxonomy/alerts/filters, /api/v2/taxonomy/assets/types?

[c8y3]

In this comment, I will put the list of endpoints to debate for removal.

• GET /user/refresh-permissions: why should it be necessary to refresh the user permissions? Aren't they always up to date?
• GET /user/cases/list: it should be possible to retrieve the list of cases owned by the owner with request GET /api/v2/cases?case_owner_id={user_identifier}
• GET /api/v2/context/search-cases: maybe GET /api/v2/cases is sufficient?
• POST /alerts/escalate/<int:alert_id>: could create a new case and then merge the alert in the new case
• POST /alerts/batch/update: could send several consecutive unitary requests instead
• POST /alerts/batch/delete
• POST /alerts/batch/merge
• POST /alerts/batch/escalate
• GET /dashboard/case_charts: could do a request on cases filtered by the open date, the data could then be computed in the interface
• GET /context/search-cases: remove and use (maybe enrich) GET /api/v2/cases instead

[lowlandghost]

I think the refresh permissions might apply to LDAP/OIDC?

Other ones, look ok to remove

[whikernel]

Hey!
I agree with @lowlandghost for 2 and 3.

For the first one, it's because the case access is deducted from a number of parameters: see the graph here https://docs.dfir-iris.org/latest/operations/access_control/

To avoid calculating this every time (really db consuming), it's cached in the EffectivePersmission table. In some cases (don't remember which exactly - but I think like auto follow on groups), the recalculation of all accesses for all users is too time consuming and thus is done later. It can be forced by clicking on the refresh permissions on the user profile.
Now ideally this should be removed and the calculation of the perms should be deferred via Celery. I didn't had the time to do it :/

Cheers!

[c8y3]

Here are some other endpoints to debate for removal:

• GET /sitemap
• GET /api/ping
• POST /user/tasks/status/update: it should be possible to do the update the task with PUT /api/v2/cases/{case_identifier}/tasks/{identifier}
• POST /case/tasks/status/update/<int:cur_id>: it should be possible to do the update the task with PUT /api/v2/cases/{case_identifier}/tasks/{identifier}
• GET /case/tasks/state
• GET /case/timeline/state
• GET /case/assets/state
• GET /case/evidences/state
• GET /case/ioc/state
• GET /case/notes/state
• GET /api/versions
• POST /manage/alert-status/search: seems unused in the current interface
• POST /manage/severities/search: seems unused in the current interface
• POST /manage/event-categories/search: seems unused in the current interface
• POST /manage/alert-resolutions/search: seems unused in the current interface
• POST /manage/analysis-status/search: seems unused in the current interface
• POST /manage/asset-types/search: seems unused in the current interface
• POST /manage/case-classifications/search: seems unused in the current interface
• POST /manage/evidence-types/search: seems unused in the current interface
• POST /manage/ioc-types/search: seems unused in the current interface
• GET /datastore/file/info/<int:cur_id>: seems unused in the current interface

[c8y3]

Datastore

The datastore endpoints return a tree. There are no limits on the size of the tree. It is assumed they are relatively small. Could this still be some kind of vulnerability? An arbitrary limit on the maximum number of nodes in a tree could be applied at node creation.

• GET /datastore/list/tree => GET /api/v2/cases/{case_identifier}/datastore
• GET /datastore/list/filter => GET /api/v2/cases/{case_identifier}/datastore with a query parameter (q=?)

Datastore folders

• POST /datastore/folder/add => POST /api/v2/cases/{case_identifier}/datastore/folders
• POST /datastore/folder/move/<int:cur_id> => PUT /api/v2/cases/{case_identifier}/datastore/folders/{identifier}
• POST /datastore/folder/rename/<int:cur_id> => PUT /api/v2/cases/{case_identifier}/datastore/folders/{identifier}
• POST /datastore/folder/delete/<int:cur_id> => DELETE /api/v2/cases/{case_identifier}/datastore/folders/{identifier}

Datastore files: in this version, the URL to upload the file metadata and the file content (as pure binary) are distinct. The cons, is that file upload requires two requests (and trying to get a file content before it is uploaded is an error).

• POST /datastore/file/add/<int:cur_id> (multipart/formdata) => POST /api/v2/cases/{case_identifier}/datastore/files (for the metadata) and PUT /api/v2/cases/{case_identifier}/datastore/files/{identifier}/content (for the file content itself)
• POST /datastore/file/add-interactive (json) => same
• POST /datastore/file/update/<int:cur_id> (multipart/formdata) => PUT /api/v2/cases/{case_identifier}/datastore/files/{identifier} (for the metadata) and PUT /api/v2/cases/{case_identifier}/datastore/files/{identifier}/content (for the file content itself)
• POST /datastore/file/move/<int:cur_id> => PUT /api/v2/cases/{case_identifier}/datastore/files/{identifier}
• GET /datastore/file/view/<int:cur_id> => GET /api/v2/cases/{case_identifier}/datastore/files/{identifier}/content
• POST /datastore/file/delete/<int:cur_id> => DELETE /api/v2/cases/{case_identifier}/datastore/files/{identifier}

Alternatively, one could use multipart requests. But, this is not uniform with the rest of the API, where JSON is uniformly used. Also, there are no endpoints to retrieve a file metadata (not necessary for now in the current interface)

• POST /datastore/file/add/<int:cur_id> (multipart/formdata) => POST /api/v2/cases/{case_identifier}/datastore/files (multipart/formadata)
• POST /datastore/file/add-interactive (json) => same
• POST /datastore/file/update/<int:cur_id> (multipart/formdata) => PUT /api/v2/cases/{case_identifier}/datastore/files/{identifier} (multipart/formadata)
• POST /datastore/file/move/<int:cur_id> => same
• GET /datastore/file/view/<int:cur_id> => GET /api/v2/cases/{case_identifier}/datastore/files/{identifier} (get the file content, but there is no way to retrieve the metadata)
• POST /datastore/file/delete/<int:cur_id> => DELETE /api/v2/cases/{case_identifier}/datastore/files/{identifier}

See https://tyk.io/blog/api-design-guidance-file-upload/ and https://stackoverflow.com/questions/3938569/how-do-i-upload-a-file-with-metadata-using-a-rest-web-service.

[c8y3]

More TODOs

Alerts

• POST /alerts/merge/<int:alert_id>: links an alert to an existing case => PUT /api/v2/alerts/{alert_identifier}/cases { case_identifier: identifier}
• POST /alerts/unmerge/<int:alert_id> => DELETE /api/v2/alerts/{alert_identifier}/cases/{case_identifier}

Case states

• GET /manage/case-states/list => GET /api/v2/manage/case-states
• POST /manage/case-states/add => POST /api/v2/manage/case-states
• GET /manage/case-states/<int:state_id> => GET /api/v2/manage/case-states/{identifier}
• POST /manage/case-states/update/<int:state_id> => PUT /api/v2/manage/case-states/{identifier}
• POST /manage/case-states/delete/<int:state_id> => DELETE /api/v2/manage/case-states/{identifier}

IoC types

• GET /manage/ioc-types/list => GET /api/v2/manage/ioc-types
• POST /manage/ioc-types/add => POST /api/v2/manage/ioc-types
• GET /manage/ioc-types/<int:cur_id> => GET /api/v2/manage/ioc-types/{identifier}
• POST /manage/ioc-types/update/<int:cur_id> => PUT /api/v2/manage/ioc-types/{identifier}
• POST /manage/ioc-types/delete/<int:cur_id> => DELETE /api/v2/manage/ioc-types/{identifier}

Task status

• GET /manage/task-status/list => GET /api/v2/manage/task-status
• GET /manage/task-status/<int:cur_id> => GET /api/v2/manage/task-status/{identifier}

Attributes

• GET /manage/attributes/list => GET /api/v2/manage/attributes
• POST /manage/attributes/update/<int:cur_id> => PUT /api/v2/manage/attributes/{identifier}

Database

• GET /manage/server/backups/make-db => POST api/v2/manage/database-backups

Assets

• GET /manage/assets/filter => GET /api/v2/customers/{identifier}/assets

Hooks

• GET /dim/hooks/options/<hook_type>/list => GET /api/v2/hooks?type=<TYPE>
• GET /manage/modules/hooks/list => GET /api/v2/hooks
• POST /dim/hooks/call => POST /api/v2/hooks-run

Cases

• GET /overview/filter => enrich GET /api/v2/cases
• POST /search => GET /api/v2/notes+GET /api/v2/iocs+GET /api/v2/comments

[c8y3]

Even more TODOs

• GET /dim/tasks/list/<int:count> : list of celery tasks executed by the workers => GET /api/v2/worker-tasks

• GET /manage/groups/list => GET /api/v2/manage/groups

• POST /manage/groups/<int:cur_id>/members/update => PUT /api/v2/manage/groups/{group_identifier}, field group_members_identifiers with list of identifiers in the request and field group_members with fields user/name/id in the response

• POST /manage/users/<int:cur_id>/groups/update => PUT /api/v2/manage/users/{user_identifier}

• POST /manage/groups/<int:cur_id>/members/delete/<int:cur_id_2> => PUT /api/v2/manage/groups/{group_identifier}

• POST /manage/groups/<int:cur_id>/cases-access/update => PUT /api/v2/manage/groups/{group_identifier}, add field group_cases_accesses=[{access_level: 4, case_id: 2}] in the request body (and keep field group_cases_access in the response, with the same fields as today)

• POST /manage/groups/<int:cur_id>/cases-access/delete => PUT /api/v2/manage/groups/{group_identifier}

• POST /case/access/set-group sets the access level to the case for the users of the group => PUT /api/v2/manage/cases/{case_identifier}/accesses with [{access_level: 4, group_identifier: 2}] in the request body

• GET /manage/modules/list => GET /api/v2/manage/modules

• POST /manage/modules/add => POST /api/v2/manage/modules

• POST /manage/modules/remove/<int:module_id> => DELETE /api/v2/manage/modules/{module_identifier}

• GET /manage/users/list => GET /api/v2/manage/users