Skip to content

GH-49273: [Python] Move stub docstring script to _build_utils with gr…#49357

Open
vanshaj2023 wants to merge 8 commits intoapache:mainfrom
vanshaj2023:fix-stub-docstrings-build-utils
Open

GH-49273: [Python] Move stub docstring script to _build_utils with gr…#49357
vanshaj2023 wants to merge 8 commits intoapache:mainfrom
vanshaj2023:fix-stub-docstrings-build-utils

Conversation

@vanshaj2023
Copy link

@vanshaj2023 vanshaj2023 commented Feb 21, 2026

Rationale for this change

Follow-up to #49235 and #49273. The update_stub_docstrings.py script was added to MANIFEST.in to fix nightly sdist builds, but it used a fragile importlib.util.spec_from_file_location pattern and forced libcst as a hard build dependency. This gives the script a proper home and makes builds more robust for users on exotic platforms who may not have libcst.

What changes are included in this PR?

  • Move scripts/update_stub_docstrings.py to pyarrow/_build_utils/update_stub_docstrings.py
  • _build_utils is auto-included in sdist via existing graft pyarrow, excluded from wheels via setuptools.packages.find.exclude
  • Replace importlib.util.spec_from_file_location in setup.py with a normal import wrapped in try/except ImportError
  • Make libcst a soft build dependency in pyproject.toml (CI still installs it via requirements-build.txt)
  • Add PYARROW_SKIP_STUB_DOCSTRINGS env var for explicit opt-out
  • Keep scripts/update_stub_docstrings.py as a thin CLI wrapper

Are these changes tested?

No new tests. Verified that all modified files pass flake8, graceful degradation catches ImportError cleanly, and CI requirements files still include libcst so official wheels get docstrings.

Are there any user-facing changes?

No. Users building from sdist without libcst will now get stubs without docstrings instead of a build failure.

@rok
Copy link
Member

rok commented Feb 22, 2026

Thanks for working on this @vanshaj2023. I've kicked off a CI-workflow to see if it works on our CI.
That said - we have ongoing work that'll change the way we build wheels python #49357 and I have a draft proposal for docstring injection change to go with it. That'll mean #49273 might change significantly. I we probably best wait for review on #49259 to finish and then proceed with your fix on top. Feel free to review #49259 to help things move along.

@vanshaj2023
Copy link
Author

Thanks @rok
I'll wait for #49259 to merge and then update this PR on top of it. Happy to review #49259 too.

vanshaj2023 and others added 6 commits February 24, 2026 15:35
…-core backend

- Add build_wheel wrapper to _build_backend/__init__.py that copies .pyi stubs
  into the wheel and injects docstrings from the built pyarrow runtime
- Make libcst optional (graceful degradation when not installed)
- Add wheel.exclude to prevent pyarrow/_build_utils from being packaged
- Support PYARROW_SKIP_STUB_DOCSTRINGS=1 env var to skip injection
@vanshaj2023
Copy link
Author

Hi @rok,
I've updated this PR on top of #49259 as discussed.
Please review these changes.

@rok
Copy link
Member

rok commented Mar 11, 2026

Hey @vanshaj2023, thanks for the update. Meanwhile we refactored the build system #49259 and docstring injection script is getting refactored as well #49453, let's revisit #49273 (and this PR) once #49453 is merged. Sorry for the uncertainty here, but this is a rapidly evolving part of the project.

@vanshaj2023
Copy link
Author

Thanks for the heads up @rok, no worries at all.
I understand this area is actively evolving. I'll hold off on this PR until #49453 is merged and then reassess whether #49273 still needs additional changes.
Happy to help review #49453 in the meantime if that would be useful!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants