Skip to content
Merged
Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
4365858
chore: [java] add readme for migrated split repos
suztomo Oct 31, 2022
fc1aadf
fix tag
suztomo Oct 31, 2022
77a1b11
chore: removed irrelevant empty lines
suztomo Oct 31, 2022
ce7621f
reverted empty line deletion
suztomo Oct 31, 2022
ae8bd3c
remove unnecessary spaces
suztomo Oct 31, 2022
d8e5406
remove unnecessary new line
suztomo Oct 31, 2022
b0fc6ba
format
suztomo Oct 31, 2022
bdfc641
whitespace
suztomo Oct 31, 2022
87fabb2
whitespace
suztomo Oct 31, 2022
b77e482
format
suztomo Nov 1, 2022
58e06e3
issue templates
suztomo Nov 1, 2022
a110ba9
treatment for monorepo
suztomo Nov 1, 2022
e7df66d
negate in jinja2
suztomo Nov 1, 2022
bcc42fe
Merge remote-tracking branch 'origin/master' into java_monorepo_migra…
suztomo Nov 1, 2022
1b6809b
monorepo variable comment
suztomo Nov 1, 2022
a866416
split_repo flag
suztomo Nov 1, 2022
3668884
revert reature request template
suztomo Nov 1, 2022
4b61704
templating
suztomo Nov 1, 2022
812da6c
syntax fix
suztomo Nov 1, 2022
3007c14
format issue
suztomo Nov 1, 2022
e958b53
Merge remote-tracking branch 'origin/master' into java_monorepo_migra…
suztomo Nov 1, 2022
0b03d61
chore: remove my username
suztomo Nov 1, 2022
0e78ac7
refactor: created migrated_split_repo variable
suztomo Nov 1, 2022
f589e1d
format
suztomo Nov 1, 2022
3d1ed74
readme
suztomo Nov 1, 2022
1da3731
Merge branch 'master' into java_monorepo_migration_note
chingor13 Nov 1, 2022
106300a
Merge remote-tracking branch 'origin/master' into java_monorepo_migra…
suztomo Nov 2, 2022
89707ad
not to pass environment variable
suztomo Nov 2, 2022
c0716f4
get for dictionary
suztomo Nov 2, 2022
ba906ef
keyerror
suztomo Nov 2, 2022
b3946d4
negate monorepo
suztomo Nov 2, 2022
305dde4
Merge branch 'master' into java_monorepo_migration_note
suztomo Nov 4, 2022
528c020
Merge branch 'master' into java_monorepo_migration_note
suztomo Nov 4, 2022
f394051
Merge branch 'master' into java_monorepo_migration_note
suztomo Nov 7, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions docker/owlbot/java/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,38 @@ docker run --rm -v $(pwd):/workspace --user "$(id -u):$(id -g)" gcr.io/repo-auto

### Building the image

#### Local Docker

From the root of the synthtool repository, run:

```bash
suztomo@suztomo:~/synthtool$ docker build -f docker/owlbot/java/Dockerfile .
...
Removing intermediate container e6d071e39d1b
---> a7d7e0c80b00
Successfully built a7d7e0c80b00
```

"a7d7e0c80b00" is the ID of the container image build. Try running the
postprocessor image with a target repository.
Here is an example with java-aiplatform repository below:

```bash
suztomo@suztomo:~/java-aiplatform$ git checkout -b test_postprocessor origin/main
branch 'test_postprocessor' set up to track 'origin/main'.
Switched to a new branch 'test_postprocessor'
suztomo@suztomo:~/java-aiplatform$ docker run --rm -v $(pwd):/workspace a7d7e0c80b00
...
Reformatting source...
...done
suztomo@suztomo:~/java-aiplatform$ git diff
... (shows the generated file differences) ...
```

This manual confirmation identifies syntax errors in Python scripts and the
templates.

#### Cloud Build
This image is built via Cloud Build. From the root of this repository, run:

```bash
Expand All @@ -32,3 +64,20 @@ To rebuild the golden test fixtures:
1. Delete the `golden` directory.
2. Copy the `input` directory recursively to `golden`
3. [Run the latest owlbot image](#running-locally) against the `golden` directory.

### Lint error

When you modify Python scripts, you may encounter lint errors
Kokoro build:

```
nox > black --check synthtool tests
would reformat synthtool/languages/java.py

Oh no! 💥 💔 💥
1 file would be reformatted, 78 files would be left unchanged.
```

In this case, install [nox](https://nox.thea.codes/en/stable/) and run
`nox -s lint` to reproduce the lint problems.

2 changes: 2 additions & 0 deletions docker/owlbot/java/bin/entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ if [ "$(ls */.OwlBot.yaml|wc -l)" -gt 1 ];then
# Monorepo (googleapis/google-cloud-java) has multiple OwlBot.yaml config
# files in the modules.
echo "Processing monorepo"
# Tell the python scripts that we're processing monorepo
export MONOREPO=true
if [ -d owl-bot-staging ]; then
# The content of owl-bot-staging is controlled by Owlbot.yaml files in
# each module in the monorepo
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@ name: Bug report
about: Create a report to help us improve

---

{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager']) %}
:bus: This library has moved to
[google-cloud-java/{{ metadata['repo']['repo_short'] }}](
https://github.com/googleapis/google-cloud-java/tree/main/{{ metadata['repo']['repo_short'] }}).
This repository will be archived in the future.
{% endif %}
Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,13 @@ name: Feature request
about: Suggest an idea for this library

---

{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager']) %}
:bus: This library has moved to
[google-cloud-java/{{ metadata['repo']['repo_short'] }}](
https://github.com/googleapis/google-cloud-java/tree/main/{{ metadata['repo']['repo_short'] }}).
This repository will be archived in the future.
{% endif %}
Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ env_vars: {

env_vars: {
key: "ENABLE_FLAKYBOT"
value: "true"
{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager'])
%} value: "false"{% else %} value: "true"{% endif %}
}

env_vars: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,9 @@ env_vars: {

env_vars: {
key: "ENABLE_FLAKYBOT"
value: "true"
{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager'])
%} value: "false"{% else %} value: "true"{% endif %}
}

env_vars: {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,7 @@ env_vars: {

env_vars: {
key: "ENABLE_FLAKYBOT"
value: "true"
{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager'])
%} value: "false"{% else %} value: "true"{% endif %}
}
10 changes: 9 additions & 1 deletion synthtool/gcp/templates/java_library/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,15 @@ Java idiomatic client for [{{metadata['repo']['name_pretty']}}][product-docs].
> Note: This client is a work-in-progress, and may occasionally
> make backwards-incompatible changes.
{% endif %}

{% if split_repo and (metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager'])) %}
:bus: In October 2022, this library has moved to
[google-cloud-java/{{ metadata['repo']['repo_short'] }}](
https://github.com/googleapis/google-cloud-java/tree/main/{{ metadata['repo']['repo_short'] }}).
This repository will be archived in the future.
Future releases will appear in the new repository (https://github.com/googleapis/google-cloud-java/releases).
The Maven artifact coordinates (`{{ group_id }}:{{ artifact_id }}`) remain the same.
{% endif %}
## Quickstart

{% if 'snippets' in metadata and metadata['snippets'][metadata['repo']['api_shortname'] + '_install_with_bom'] -%}
Expand Down
8 changes: 6 additions & 2 deletions synthtool/gcp/templates/java_library/renovate.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
{
"extends": [
{% if metadata['repo']['library_type'] == 'GAPIC_AUTO'
or (metadata['repo']['repo_short'] and metadata['repo']['repo_short'] in ['java-translate', 'java-dns', 'java-notification', 'java-resourcemanager'])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider moving this logic to the template variable collector

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to migrated_split_repo variable in java.py.

%}{
"enabled": false,
{% else %}{
{% endif %} "extends": [
":separateMajorReleases",
":combinePatchMinorReleases",
":ignoreUnstable",
Expand Down
3 changes: 3 additions & 0 deletions synthtool/languages/java.py
Original file line number Diff line number Diff line change
Expand Up @@ -485,6 +485,9 @@ def common_templates(
**kwargs: Additional options for CommonTemplates.java_library()
"""
kwargs["metadata"] = _common_template_metadata()
# MONOREPO environment variable is set in entrypoint.sh
kwargs["monorepo"] = os.environ.get("MONOREPO") == "true"
kwargs["split_repo"] = os.environ.get("MONOREPO") != "true"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading environment variables in the logic feels error prone. It feels like this should value should be explicitly provided to this function

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The challenge is that this java.common_templates() method is called owlbot.py in individual repositories, not the shell script.

To address your concern, I think of providing a keyword argument to java.common_templates() method in owlbot.py:

suztomo@suztomo:~/java-aiplatform$ git diff
diff --git a/owlbot.py b/owlbot.py
index cac43dbb..0dc60a13 100644
--- a/owlbot.py
+++ b/owlbot.py
@@ -27,5 +27,6 @@ java.common_templates(
         ".kokoro/nightly/samples.cfg",
         ".kokoro/presubmit/samples.cfg",
         ".github/CODEOWNERS"
-    ]
+    ],
+    migrated_split_repo=True
 )

This requires me to edit all owlbot.py in ~120 repositories. (feasible but time-consuming)

Do you think of a better way to detect whether the postprocessor is processing monorepo in this Python script?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than modifying owlbot.py in ~120 split repositories, passing a special value for 1 monorepo is easy. I'll go for this path.

templates = gcp.CommonTemplates(template_path=template_path).java_library(**kwargs)

# skip README generation on Kokoro (autosynth)
Expand Down