chore(migration): Migrate code from googleapis/python-runtimeconfig into packages/google-cloud-runtimeconfig #15544
Conversation
- dns - error_reporting - language - logging - monitoring - resource_manager - runtimeconfig - speech - translate - vision Also updating bounds on these in `google-cloud` uber-package.
The gcloud-common repo moved to https://github.com/GoogleCloudPlatform/google-cloud-common
* Remove api.core packages from google.cloud.core, make google.cloud.core depend on api_core. * s/google.api.core/google.api_core/g and nox updates * Fixing core tests, addressing review feedback * Fix bigquery
Also - updating all dependencies of `grpcio` to `>= 1.7.0`. This was due to an issue [1] with `1.6.0`. - updating the version of `google-api-core` (also to be released, This is required since the bounds on `grpcio` of `google-cloud-core==0.28.0` and `google-api-core==0.1.0` are mutually exclusive.) - Updating `google-api-core` CHANGELOG for release. - Updating packages to depend on `google-cloud-core>=0.28.0`. - Installing `nox -s lint` deps locally for vision. [1]: grpc/grpc#12455
* Making release for most packages. Every package except those that have already been released (`google-cloud-core`, `google-api-core`, `google-cloud-bigquery`): - `google-cloud` - `google-cloud-bigtable` - `google-cloud-datastore` - `google-cloud-dns` - `google-cloud-error-reporting` - `google-cloud-firestore` - `google-cloud-language` - `google-cloud-logging` - `google-cloud-monitoring` - `google-cloud-resource-manager` - `google-cloud-runtimeconfig` - `google-cloud-spanner` - `google-cloud-speech` - `google-cloud-storage` - `google-cloud-trace` - `google-cloud-translate` - `google-cloud-videointelligence` - `google-cloud-vision` * Adding changelog files for each package.
This is to make it clear the code is between releases. Any code that relies on a **new** feature (e.g. of `google-api-core`) will then be able to **explicitly** make this clear by using the lower bound of the `devN` version. Fixes #4208. See: https://snarky.ca/how-i-manage-package-version-numbers/
Also obeying an 80-column limit for the content and adding a missing "``virtualenv``" in the phrase "``pip`` and ``virtualenv``" in some of the docs.
* Making a `nox -s default` session for all packages. * Using "default" `nox` session on AppVeyor. This was 32-bit or 64-bit Python can be used, depending on which is the active `python` / the active `nox.exe`.
* remove 3.4 from unit test runs * add 3.7 to most packages. PubSub, Monitoring, BigQuery not enabled * Fix #5292 by draining queue in a way compatible with SimpleQueue and Queue
- Move contents of 'docs/runtimeconfig/' to 'runtimeconfig/docs'. - Rename 'runtimeconfig/docs/usage.rst' to 'runtimeconfig/docs/index.rst', and add a static redirect page, 'runtimeconfig/docs/usage.html'. - Harmonize / DRY 'runtimeconfig/README.rst' / 'runtimeconfig/docs/index.rst'. Toward #5912
* Remove badges for deprecated umbrella 'google-cloud' package. * Clarify support levels. - Add explicit section to support linking from sub-package README badges. - Move explanatory text for a support level above the list of packages at that level. * Normalize use of support-level badges in READMEs. - Note that 'error_reporting/README.rst' and 'monitoring/README.rst' are undergoing other edits; they are left out here to avoid conflicts. * Use 'General Avaialblity' for support level. Fix linkx in related API READMEs. * Fix links for alpha support in API READMEs. * Fix links for beta support in API READMEs.
* blacken bigtable * blacken logging * blacken translate * blacken runtimeconfig * blacken dns
Summary of ChangesHello @parthea, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request completes the migration of the Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request migrates the python-runtimeconfig library into the monorepo, encompassing its source code, documentation, and CI/CD configuration. A security audit revealed critical vulnerabilities in CI/CD and utility scripts, specifically insecure deserialization and command injection due to unquoted variables in shell scripts, which must be addressed to secure the CI environment and maintain documentation integrity. Additionally, the migration introduced incorrect paths and URLs in configuration and documentation files, and there are opportunities for code simplification and consistency improvements.
I am having trouble creating individual review comments. Click here to see my feedback.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.12/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.13/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.9/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.github/workflows/docs.yml (11-13)
The versions for actions/checkout and actions/setup-python appear to be incorrect. The latest major versions are v4 for actions/checkout and v5 for actions/setup-python. Using non-existent versions like @v6 will cause this workflow to fail.
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5packages/google-cloud-runtimeconfig/.github/workflows/lint.yml (11-13)
The versions for actions/checkout and actions/setup-python appear to be incorrect. The latest major versions are v4 for actions/checkout and v5 for actions/setup-python. Using non-existent versions like @v6 will cause this workflow to fail.
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5packages/google-cloud-runtimeconfig/.kokoro/continuous/common.cfg (17-27)
The paths for build_file and TRAMPOLINE_BUILD_FILE seem to be incorrect for this monorepo structure. They appear to be copied from the old standalone repository.
build_fileshould point to the script within thepackages/google-cloud-runtimeconfigdirectory.TRAMPOLINE_BUILD_FILE's value should reflect the monorepo structure, likely starting withgithub/google-cloud-python/packages/google-cloud-runtimeconfig/....
build_file: "packages/google-cloud-runtimeconfig/.kokoro/trampoline.sh"
# Configure the docker image for kokoro-trampoline.
env_vars: {
key: "TRAMPOLINE_IMAGE"
value: "gcr.io/cloud-devrel-kokoro-resources/python-multi"
}
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/google-cloud-python/packages/google-cloud-runtimeconfig/.kokoro/build.sh"
}
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.9/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/presubmit/common.cfg (17-27)
The paths for build_file and TRAMPOLINE_BUILD_FILE seem incorrect for a monorepo structure. They should be updated to reflect the new location of the files within packages/google-cloud-runtimeconfig.
build_file: "packages/google-cloud-runtimeconfig/.kokoro/trampoline.sh"
# Configure the docker image for kokoro-trampoline.
env_vars: {
key: "TRAMPOLINE_IMAGE"
value: "gcr.io/cloud-devrel-kokoro-resources/python-multi"
}
env_vars: {
key: "TRAMPOLINE_BUILD_FILE"
value: "github/google-cloud-python/packages/google-cloud-runtimeconfig/.kokoro/build.sh"
}
packages/google-cloud-runtimeconfig/.kokoro/samples/lint/common.cfg (17-34)
The paths for TRAMPOLINE_BUILD_FILE and build_file appear to be from the old standalone repository and are incorrect for the new monorepo structure. They need to be updated to point to the correct locations within the packages/google-cloud-runtimeconfig directory.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.10/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure. They should be updated to reflect the new location under packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.10/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.11/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.11/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.12/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.8/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.8/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.7/periodic-head.cfg (9-11)
The TRAMPOLINE_BUILD_FILE path is incorrect for the monorepo structure. It should be updated to point to the script's new location within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.7/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/.kokoro/samples/python3.13/common.cfg (23-40)
The paths for TRAMPOLINE_BUILD_FILE and build_file are incorrect for the monorepo structure and should be updated to point to the files within packages/google-cloud-runtimeconfig.
packages/google-cloud-runtimeconfig/scripts/readme-gen/readme_gen.py (58)
The use of yaml.load() is insecure as it can lead to arbitrary code execution when parsing untrusted YAML content. Since this script reads files from the repository that could be modified in a pull request, it poses a security risk. It is recommended to use yaml.safe_load() instead.
config = yaml.safe_load(f)
packages/google-cloud-runtimeconfig/.kokoro/populate-secrets.sh (28-37)
The environment variable SECRET_MANAGER_KEYS is used unquoted in a shell command, which can lead to command injection if the variable contains shell metacharacters. Additionally, the key variable is used unquoted in the docker run command and output redirection, which can lead to argument injection or path traversal.
for key in $(echo "${SECRET_MANAGER_KEYS}" | sed "s/,/ /g")
do
msg "Retrieving secret ${key}"
docker run --entrypoint=gcloud \
--volume="${KOKORO_GFILE_DIR}:${KOKORO_GFILE_DIR}" \
gcr.io/google.com/cloudsdktool/cloud-sdk \
secrets versions access latest \
--project cloud-devrel-kokoro-resources \
--secret "${key}" > \
"${SECRET_LOCATION}/${key}"
packages/google-cloud-runtimeconfig/.kokoro/noxfile.py (92)
The list of Python versions to test against is missing '3.14'. The unittest.yml workflow includes it, and it's generally good practice to keep these lists consistent.
ALL_VERSIONS = ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14"]
packages/google-cloud-runtimeconfig/.kokoro/build.sh (57)
The NOX_SESSION environment variable is used unquoted in the command python3 -m nox -s ${NOX_SESSION:-}. This can lead to command or argument injection if the variable's value is influenced by an attacker. Quoting the variable prevents the shell from performing word splitting and glob expansion on its value.
python3 -m nox -s "${NOX_SESSION:-}"
packages/google-cloud-runtimeconfig/.repo-metadata.json (10)
The repo field points to the old standalone repository. In a monorepo, this should be updated to point to the monorepo itself, which is googleapis/google-cloud-python.
"repo": "googleapis/google-cloud-python",
packages/google-cloud-runtimeconfig/CONTRIBUTING.rst (52)
This git remote command points to the old standalone repository. It should be updated to point to the google-cloud-python monorepo.
$ git remote add upstream [email protected]:googleapis/google-cloud-python.git
packages/google-cloud-runtimeconfig/CONTRIBUTING.rst (212)
This link points to a file in the old standalone repository. It should be updated to point to the correct location within the google-cloud-python monorepo.
``https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-runtimeconfig/CONTRIBUTING.rst``)
packages/google-cloud-runtimeconfig/CONTRIBUTING.rst (245)
This link to noxfile.py points to the old standalone repository. It should be updated to point to the file within the google-cloud-python monorepo.
.. _config: https://github.com/googleapis/google-cloud-python/blob/main/packages/google-cloud-runtimeconfig/noxfile.py
packages/google-cloud-runtimeconfig/README.rst (56)
The link to the samples/ folder points to the old standalone repository. This should be updated to point to the correct location within the google-cloud-python monorepo.
.. _samples/: https://github.com/googleapis/google-cloud-python/tree/main/packages/google-cloud-runtimeconfig/samples
packages/google-cloud-runtimeconfig/google/cloud/runtimeconfig/config.py (116)
There is a typo in the type hint. It should be google.cloud.runtimeconfig.client.Client.
:type client: :class:`google.cloud.runtimeconfig.client.Client`
packages/google-cloud-runtimeconfig/google/cloud/runtimeconfig/variable.py (208-217)
The logic for parsing the timestamp can be simplified by using google.api_core.datetime_helpers.from_rfc3339, which can handle timestamps with or without nanoseconds gracefully and returns the appropriate datetime or DatetimeWithNanoseconds object.
value = self._properties.get("updateTime")
if value is not None:
return datetime_helpers.from_rfc3339(value)
return Nonepackages/google-cloud-runtimeconfig/noxfile.py (392)
The check for cpp implementation support should include Python 3.14, as it's not supported there either.
if protobuf_implementation == "cpp" and session.python in ("3.11", "3.12", "3.13", "3.14"):
parthea
added
the
do not merge
parthea
deleted the
migration.python-runtimeconfig.migration.2026-02-09_17-15-38.migrate
branch
parthea
removed
the
do not merge
20 participants
See #10940.
This PR should be merged with a merge-commit, not a squash-commit, in order to preserve the git history.