You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
-[6. Issues Tracking and Contribution](#Issues-Tracking-and-Contribution)
26
-
-[7. Licenses](#Licenses)
27
-
-[8. Authors](#Authors)
22
+
-[4.1. Installation](#Installation)
23
+
-[5. Issues Tracking and Contribution](#Issues-Tracking-and-Contribution)
24
+
-[6. Licenses](#Licenses)
25
+
-[7. Authors](#Authors)
28
26
29
27
30
28
@@ -39,7 +37,8 @@ Digital Investigation Platform
39
37
40
38
## What is Kuiper?
41
39
Kuiper is a digital investigation platform that provides a capabilities for the investigation team and individuals to parse, search, visualize collected evidences (evidences could be collected by fast traige script like [Hoarder](https://github.com/muteb/Hoarder)). In additional, collaborate with other team members on the same platform by tagging artifacts and present it as a timeline, as well as setting rules for automating the detection. The main purpose of this project is to aid in streamlining digital investigation activities and allow advanced analytics capabilities with the ability to handle a large amounts of data.
@@ -88,82 +87,30 @@ With a large number of cases and a large number of team members, it becomes hard
88
87
89
88
Kuiper use the following components:
90
89
91
-
**Flask:** A web framework written in Python, used as the primary web application component.
92
-
93
-
**Elasticsearch:** A distributed, open source search and analytics engine, used as the primary database to store parser results.
94
-
95
-
**MongoDB:** A database that stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema, used to store Kuiper web application configurations and information about parsed files.
96
-
97
-
**Redis:** A in-memory data structure store, used as a database, cache and message broker, used as a message broker to relay tasks to celery workers.
98
-
99
-
**Celery:** A asynchronous task queue/job queue based on distributed message passing, used as the main processing engine to process relayed tasks from redis.
100
-
101
-
102
-
## Parsers
90
+
-**Flask:** A web framework written in Python, used as the primary web application component.
103
91
104
-
The following are parsers used in Kuiper project, some are custom built, and others have been modified to output the results in a compliant format in order to integrate it with Kuiper and some have been heavily modified to make it compatible with Kuiper.
92
+
-**Elasticsearch:** A distributed, open source search and analytics engine, used as the primary database to store parser results.
-**MongoDB:** A database that stores data in JSON-like documents that can vary in structure, offering a dynamic, flexible schema, used to store Kuiper web application configurations and information about parsed files.
145
95
96
+
-**Redis:** A in-memory data structure store, used as a database, cache and message broker, used as a message broker to relay tasks to celery workers.
146
97
98
+
-**Celery:** A asynchronous task queue/job queue based on distributed message passing, used as the main processing engine to process relayed tasks from redis.
147
99
148
-
149
-
150
-
To add your own parser on Kuiper, read documentation [Add Custom Parser](https://github.com/DFIRKuiper/Kuiper/wiki/Add-Custom-Parser)
- If you want to use RAM more than 64GB to increase Elasticsearch performence, it is recommended to use multiple nodes for Elasticsearch cluster instead in different machines
113
+
- For parsing, Celery generate workers based on CPU cores (worker per core), each core parse one machine at a time and when the machine finished, the other queued machines will start parsing, if you have large number of machines to process in the same time you have to increase the cores number
167
114
168
115
## Installation
169
116
@@ -194,11 +141,6 @@ If everything runs correctly now you should be able to use Kuiper through the li
194
141
195
142
Happy hunting :).
196
143
197
-
# TODO List:
198
-
199
-
- Add Wiki
200
-
- Fix issues with parsers
201
-
202
144
# Issues Tracking and Contribution
203
145
204
146
We are happy to receive any issues, contribution, and ideas.
0 commit comments