bulk-translate 0.25.2

Third-party providers hosting↗️

A tiny Python no-string package for performing translation of a massive CSV/JSONL files that natively provides support of pre-annotated fixed-spans that are invariant for translator.

Description

📘 More on spans

📘 bulk-translate features

The out-of-the box features of the bulk-translate are:

• ✅ Support of the spans for annotation / optional translation.
• ✅ Native Implementation of two translation modes:
  • fast-mode: exploits extra chars that could be used for grouping all the text parts into single batch with further deconstruction.
  • accurate: performs individual translation of each text part.
• ✅ No strings: you're free to adopt any LM / LLM backend.
  • Support googletrans by default.

Installation

From PyPI:

pip install bulk-translate

or latest version from here:

pip install git+https://github.com/nicolay-r/bulk-translate

Usage

API

👉 Follow this notebook tutorial at nlp-thirdgate

Command Line / Shell

NOTE: Spans supports only in JSON-lines format.

NOTE: Requires source_iter package installation.

For the following test.tsv example data with annotated entities enclosed in square brackets:

python -m bulk_translate.translate \
    --src "test/data/test.tsv" \
    --schema '{"translated":"{text}"}' \
    --adapter "dynamic:models/googletrans_310a.py:GoogleTranslateModel" \
    --output "test-translated.jsonl" \
    --batch-size 10 \
    %%m \
    --src "auto" \
    --dest "ru"

Powered by

The pipeline construction components were taken from AREkit [github]