Skip to content

[Feature] Support NDJson and CSV #262

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Mar 14, 2023
Merged

[Feature] Support NDJson and CSV #262

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
619d525
support ndjson, csv
ahmednfwela Mar 11, 2023
3c563ae
dart format
ahmednfwela Mar 11, 2023
02dcf72
Merge branch 'main' of github.com:meilisearch/meilisearch-dart into f…
ahmednfwela Mar 14, 2023
23765f1
use macros for documentation
ahmednfwela Mar 14, 2023
484d858
use relative imports
ahmednfwela Mar 14, 2023
68266da
format
ahmednfwela Mar 14, 2023
aeaa4bd
reworked document tests
ahmednfwela Mar 14, 2023
073dfbf
format
ahmednfwela Mar 14, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions lib/src/http_request.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,20 +29,23 @@ abstract class HttpRequest {
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType,
});

/// POST method
Future<Response<T>> postMethod<T>(
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType,
});

/// PUT method
Future<Response<T>> putMethod<T>(
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType,
});

/// DELETE method
Expand Down
13 changes: 12 additions & 1 deletion lib/src/http_request_impl.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,6 @@ class HttpRequestImpl implements HttpRequest {
if (_kIsWeb) Version.qualifiedVersionWeb
].join(',')
},
contentType: 'application/json',
responseType: ResponseType.json,
connectTimeout: connectTimeout ?? Duration(seconds: 5),
),
Expand Down Expand Up @@ -61,12 +60,16 @@ class HttpRequestImpl implements HttpRequest {
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType = Headers.jsonContentType,
}) async {
try {
return await dio.post<T>(
path,
data: data,
queryParameters: queryParameters,
options: Options(
contentType: contentType,
),
);
} on DioError catch (e) {
return throwException(e);
Expand All @@ -78,12 +81,16 @@ class HttpRequestImpl implements HttpRequest {
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType = Headers.jsonContentType,
}) async {
try {
return await dio.patch<T>(
path,
data: data,
queryParameters: queryParameters,
options: Options(
contentType: contentType,
),
);
} on DioError catch (e) {
return throwException(e);
Expand All @@ -95,12 +102,16 @@ class HttpRequestImpl implements HttpRequest {
String path, {
Object? data,
Map<String, Object?>? queryParameters,
String contentType = Headers.jsonContentType,
}) async {
try {
return await dio.put<T>(
path,
data: data,
queryParameters: queryParameters,
options: Options(
contentType: contentType,
),
);
} on DioError catch (e) {
return throwException(e);
Expand Down
103 changes: 99 additions & 4 deletions lib/src/index.dart
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,35 +54,130 @@ abstract class MeiliSearchIndex {
Future<Result<Map<String, Object?>>> getDocuments({DocumentsQuery? params});

/// Add a list of documents by given [documents] and optional [primaryKey] parameter.
/// If index is not exists tries to create a new index and adds documents.
///
/// If index does not exist, it tries to create a new index and adds documents.
Future<Task> addDocuments(
List<Map<String, Object?>> documents, {
String? primaryKey,
});

/// Add a list of documents by given [documents] and optional [primaryKey] parameter
///
/// * the passed [documents] must be a valid JSON string representing an array of objects.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<Task> addDocumentsJson(
String documents, {
Copy link
Member

@brunoocasali brunoocasali Mar 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The best way to represent the JSON docs is to receive it as string?

Copy link
Collaborator Author

@ahmednfwela ahmednfwela Mar 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The current addDocuments already receives a parsed json, all these new methods mainly target file contents, so all of them must be raw strings.
or is that not the case ?

String? primaryKey,
});

/// Add a list of documents by given [documents] and optional [primaryKey] parameter
///
/// * The passed [documents] must be a valid CSV string, where each line corresponds to an object.
/// * If index does not exist tries to create a new index and adds documents.
Future<Task> addDocumentsCsv(
String documents, {
String? primaryKey,
});

/// Add a list of documents by given [documents] and optional [primaryKey] parameter
///
/// the passed [documents] must be a valid Newline Delimited Json (NdJson) string, where each line corresponds to an object.
/// If index does not exist tries to create a new index and adds documents.
Future<Task> addDocumentsNdjson(
String documents, {
String? primaryKey,
});

/// Add a list of documents in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
/// If index is not exists tries to create a new index and adds documents.
/// If index does not exist tries to create a new index and adds documents.
Future<List<Task>> addDocumentsInBatches(
List<Map<String, Object?>> documents, {
int batchSize = 1000,
String? primaryKey,
});

/// Add a list of documents in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid CSV string, where each line corresponds to an object.
/// * If index does not exist tries to create a new index and adds documents.
Future<List<Task>> addDocumentsCsvInBatches(
String documents, {
String? primaryKey,
int batchSize = 1000,
});

/// Add a list of documents in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid Newline Delimited Json (NdJson) string, where each line corresponds to an object.
/// * If index does not exist tries to create a new index and adds documents.
Future<List<Task>> addDocumentsNdjsonInBatches(
String documents, {
String? primaryKey,
int batchSize = 1000,
});

/// Add a list of documents or update them if they already exist by given [documents] and optional [primaryKey] parameter.
/// If index is not exists tries to create a new index and adds documents.
/// If index does not exist tries to create a new index and adds documents.
Future<Task> updateDocuments(
List<Map<String, Object?>> documents, {
String? primaryKey,
});

/// Add a list of documents or update them if they already exist by given [documents] and optional [primaryKey] parameter.
///
/// * the passed [documents] must be a valid JSON string representing an array of objects.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<Task> updateDocumentsJson(
String documents, {
String? primaryKey,
});

/// Add a list of documents or update them if they already exist by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid Newline Delimited Json (NdJson) string, where each line corresponds to an object.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<Task> updateDocumentsNdjson(
String documents, {
String? primaryKey,
});

/// Add a list of documents or update them if they already exist by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid CSV string, where each line corresponds to an object.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<Task> updateDocumentsCsv(
String documents, {
String? primaryKey,
});

/// Add a list of documents or update them if they already exist in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
/// If index is not exists tries to create a new index and adds documents.
/// If index does not exist tries to create a new index and adds documents.
Future<List<Task>> updateDocumentsInBatches(
List<Map<String, Object?>> documents, {
int batchSize = 1000,
String? primaryKey,
});

/// Add a list of documents or update them if they already exist in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid CSV string, where each line corresponds to an object.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<List<Task>> updateDocumentsCsvInBatches(
String documents, {
String? primaryKey,
int batchSize = 1000,
});

/// Add a list of documents or update them if they already exist in batches of size [batchSize] by given [documents] and optional [primaryKey] parameter.
///
/// * The passed [documents] must be a valid Newline Delimited Json (NdJson) string, where each line corresponds to an object.
/// * If index does not exist, it tries to create a new index and adds documents.
Future<List<Task>> updateDocumentsNdjsonInBatches(
String documents, {
String? primaryKey,
int batchSize = 1000,
});

/// Delete one document by given [id].
Future<Task> deleteDocument(Object id);

Expand Down
Loading