Skip to content

chore(migration): Migrate code from googleapis/python-bigquery-magics into packages/bigquery-magics#16005

Draft
parthea wants to merge 93 commits intomainfrom
migration.python-bigquery-magics.migration.2026-03-02_16-59-45.migrate
Draft

chore(migration): Migrate code from googleapis/python-bigquery-magics into packages/bigquery-magics#16005
parthea wants to merge 93 commits intomainfrom
migration.python-bigquery-magics.migration.2026-03-02_16-59-45.migrate

Conversation

@parthea
Copy link
Contributor

@parthea parthea commented Mar 2, 2026

See #14911.

This PR should be merged with a merge-commit, not a squash-commit, in order to preserve the git history.

tswast and others added 30 commits January 19, 2024 16:22
* chore: copy magics unit tests from google-cloud-bigquery

* update tests to use bigquery_magics module

* remove auth from unit tests

* fix coverage
* chore: synchronize with Python templates

* add owlbot configuration

* remove unknown paths from linter

* use templated noxfile

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* multiple teams: see: googleapis/synthtool#946

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: grant api-bigquery and api-bigquery-dataframe teams write access to repo

* add python 3.7

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* chore: copy magics unit tests from google-cloud-bigquery

* include google-cloud-testutils in unittests

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* use bqstorage extras

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* use extras by python for everything

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* update tests to use bigquery_magics module

* remove auth from unit tests

* fix coverage

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@d50980e
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:5a4c19d17e597b92d786e569be101e636c9c2817731f80a5adec56b2aa8fe070

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: google-cloud-policy-bot[bot] <80869356+google-cloud-policy-bot[bot]@users.noreply.github.com>
* chore: add the sample test for magics

* re-arrange directory and make nox -s pass

* chore: add the sample test for magics

* chore: copy `%%bigquery` magics tests from google-cloud-bigquery (#8)

* chore: copy magics unit tests from google-cloud-bigquery

* update tests to use bigquery_magics module

* remove auth from unit tests

* fix coverage

* re-arrange directory and make nox -s pass

* delete files

* test the version

* chore: add the sample test for magics

* re-arrange directory and make nox -s pass

* chore: add the sample test for magics

* chore: copy `%%bigquery` magics tests from google-cloud-bigquery (#8)

* chore: copy magics unit tests from google-cloud-bigquery

* update tests to use bigquery_magics module

* remove auth from unit tests

* fix coverage

* re-arrange directory and make nox -s pass

* delete files

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* delete files

* address comments

* fix the merging conflicts

* set local version of variables

* address comments

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Tim Sweña (Swast) <swast@google.com>
Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* docs: clean up docs and let the nox pass

* update the commit

* clean up the README.rst to pass lint_setup_py

* Trigger Build

* address comments
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* chore: add samples and kokoro tests as required checks

* update tests
* docs: update the IPython Magics docs
Update lingering links to google-cloud-bigquery.
Source-Link: googleapis/synthtool@a37f74c
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:d3de8a02819f65001effcbd3ea76ce97e9bcff035c7a89457f40f892c87c5b32

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
)

Source-Link: googleapis/synthtool@14d8b28
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:5651442a6336971a2fb2df40fb56b3337df67cafa14c0809cc89cb34ccee1b8e

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@9ae0785
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:52210e0e0559f5ea8c52be148b33504022e1faef4e95fbe4b32d68022af2fa7e

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore(deps): update all dependencies

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
* chore: include bigquery-magics version number in the user agent

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Import version from bigquery_magics.version package to avoid circular
dependency.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: https://togithub.com/googleapis/synthtool/commit/bef813d194de29ddf3576eda60148b6b3dcc93d9
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:94bb690db96e6242b2567a4860a94d48fa48696d092e51b0884a1a2c0a79a407
* feat: depends on pydata-google-auth to fetch credentials

If the credentials are not set, the user will be guided through the OAuth flow.

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* Add pydata-google-auth deps to 3.7 testing constraints.

* fix typo

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
* refactor: use dataclass for Context to reduce boilerplate

* Fix test cover workflow. Ref: googleapis/python-bigquery-dataframes#948

* Fix field name

* fix docstring indentation
#54)

* refactor: break down _cell_magic() into several helper functions to improve readability

* Fix import

* fix test failures

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* fix type hint for pre 3.9 compatibility

* remove unused variable in test

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@71a7297
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:e8dcfd7cbfd8beac3a3ff8d3f3185287ea0625d859168cc80faccfc9a7a00455

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
* chore(deps): update all dependencies

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Co-authored-by: Anthonios Partheniou <partheniou@google.com>
…igframes' to support query results larger than 10 GB (#58)

* define 'engine' field in the conext

* use bigframes API when the context.engine says so

* remove unnecessary deps

* relax pip deps

* undo pip deps

* make bigframes an optional dependency

* fix format

* use 'is' for None checks

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* parametrize bigframes installation

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* define unit_bf with bf installation to make owlbot happy

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* parametrize bf installation and disable owlbot enforcement on the noxfile

* chang owlbot file to test optional bf deps

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
release-please bot and others added 27 commits August 21, 2025 15:54
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
Source-Link: googleapis/synthtool@16790a3
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:543e209e7c1c1ffe720eb4db1a3f045a75099304fb19aa11a47dc717b8aae2a9

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
)

* chore: add missing kokoro configs following up to 3.7 and 3.8 drop

* Update .kokoro/presubmit/prerelease-deps-3.10.cfg

* add psutil to system test deps
Source-Link: googleapis/synthtool@6702a34
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-python:latest@sha256:fbbc8db67afd8b7d71bf694c5081a32da0c528eba166fbcffb3b6e56ddf907d5

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
This PR updates the librarian language container image sha.
This PR effectively moves ownership for this repo to the python language
team, and removes api-bigquery as the defacto code owner.
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.7.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:c8612d3fffb3f6a32353b2d1abd16b61e87811866f7ec9d65b59b02eb452a620
<details><summary>bigquery-magics: 0.11.0</summary>

##
[0.11.0](googleapis/python-bigquery-magics@v0.10.3...v0.11.0)
(2025-12-15)

### Features

* support python 3.14 (#165)
([8ee73ecf](googleapis/python-bigquery-magics@8ee73ecf))

</details>
…n on Colab (#209)

…d Spanner. This avoids clashes if the same notebook uses visualizers
for both BigQuery graphs and Spanner graphs, which can cause
malfunctions in the visualizers.

Thank you for opening a Pull Request! Before submitting your PR, there
are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/python-bigquery-magics/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕
)

Removes the `%%bqsql` magic alias.

Changes:
- Removed `ipython.register_magic_function` call for `bqsql` in
`bigquery_magics/__init__.py`.
- Removed `%%bqsql` references from `bigquery_magics/bigquery.py`
docstring.
- Removed `bqsql` from `test_extension_load` parametrization in
`tests/unit/bigquery/test_bigquery.py`.
- Removed `%%bqsql` references and examples from `README.rst` and
`docs/magics.rst`.

---
*PR created automatically by Jules for task
[18023458691533748703](https://jules.google.com/task/18023458691533748703)
started by @tswast*

---------

Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com>
Co-authored-by: tswast <247555+tswast@users.noreply.github.com>
We now only include query results directly in the html when the query
results are less than 100 KB. For larger query results, we store only
the reference to the destination table in the HTML, and have the python
code re-read the query results from the destination table during the
callback.

Also, added a hard limit of 5 MB in the query result size, beyond which,
graph visualization is not supported altogether.

Thank you for opening a Pull Request! Before submitting your PR, there
are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/python-bigquery-magics/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕
Thank you for opening a Pull Request! Before submitting your PR, there
are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/python-bigquery-magics/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
feat: update image to
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:1a2a85ab507aea26d787c06cc7979decb117164c81dd78a745982dfda80d4f68
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.8.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:1a2a85ab507aea26d787c06cc7979decb117164c81dd78a745982dfda80d4f68
<details><summary>bigquery-magics: 0.12.0</summary>

##
[0.12.0](googleapis/python-bigquery-magics@v0.11.0...v0.12.0)
(2026-02-10)

### Features

* support schema view (#211)
([8e1883ee](googleapis/python-bigquery-magics@8e1883ee))

* remove bqsql magic to make that name available for bigframes (#210)
([c46c94af](googleapis/python-bigquery-magics@c46c94af))

### Bug Fixes

* reduce conflicts between Spanner and BigQuery graph visualization on
Colab (#209)
([7dca7b13](googleapis/python-bigquery-magics@7dca7b13))

</details>
…frame-team (#218)

This PR replaces the old api-bigquery-dataframe with bigquery-dataframe-team,
and yoshi-python/python-core-client-libraries with cloud-sdk-python-team.

b/478003109
1) Disallow backticks in the graph name, as the code doesn't handle them
properly 2) Add try/catch so a failed information_schema query only
disables schema view, not the entire visualizer 3) Remove the default
table view when the graph visualizer is used.

Thank you for opening a Pull Request! Before submitting your PR, there
are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/python-bigquery-magics/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕
…#220)

Thank you for opening a Pull Request! Before submitting your PR, there
are a few things you can do to make sure it goes smoothly:
- [ ] Make sure to open an issue as a
[bug/issue](https://github.com/googleapis/python-bigquery-magics/issues/new/choose)
before writing your code! That way we can discuss the change, evaluate
designs, and agree on the general idea
- [ ] Ensure the tests and linter pass
- [ ] Code coverage does not decrease (if any source code was changed)
- [ ] Appropriate docs were updated (if necessary)

Fixes #<issue_number_goes_here> 🦕
feat: update image to
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:160860d189ff1c2f7515638478823712fa5b243e27ccc33a2728669fa1e2ed0c
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.8.3
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/python-librarian-generator@sha256:160860d189ff1c2f7515638478823712fa5b243e27ccc33a2728669fa1e2ed0c
<details><summary>bigquery-magics: v0.12.1</summary>

##
[v0.12.1](googleapis/python-bigquery-magics@v0.12.0...v0.12.1)
(2026-02-25)

### Bug Fixes

* usability improvement to the graph visualizer (#215)
([7655ada7](googleapis/python-bigquery-magics@7655ada7))

</details>
…s/main' into migration.python-bigquery-magics.migration.2026-03-02_16-59-45.migrate
@google-cla
Copy link

google-cla bot commented Mar 2, 2026

Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

View this failed invocation of the CLA check for more information.

For the most up to date status, view the checks section at the bottom of the pull request.

@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request completes the migration of the bigquery-magics project into a new sub-package within the monorepo. This involved moving all source code, documentation, configuration, and CI/CD pipelines to ensure the project operates seamlessly in its new location. The primary goal was to consolidate the project while preserving its complete git history, as explicitly requested.

Highlights

  • Codebase Migration: The entire bigquery-magics codebase has been migrated from googleapis/python-bigquery-magics to a new location within the monorepo at packages/bigquery-magics.
  • Configuration and CI/CD Setup: All associated configuration files, including .coveragerc, .flake8, .gitignore, .pre-commit-config.yaml, .repo-metadata.json, and .trampolinerc, have been added to the new package directory. Extensive Kokoro and GitHub Actions workflows for documentation, linting, and unit/system tests across various Python versions (3.7-3.14) have also been established.
  • Documentation and Contributing Guidelines: Comprehensive documentation files, including CHANGELOG.md, CODE_OF_CONDUCT.md, CONTRIBUTING.md, LICENSE, README.rst, and SECURITY.md, along with Sphinx configuration and templates, have been moved or created to support the new package structure.
  • Core Logic and Utilities: The core Python source code for bigquery_magics, including modules for version helpers, BigQuery magic functionality, configuration context, client creation, environment detection, graph visualization server, line argument parsing, and Python string formatting for SQL, has been relocated.
  • Sample Code and Testing Infrastructure: Sample snippets, their associated helper utilities, Pytest configurations, and Nox configurations for testing samples have been integrated into the new package structure.
Changelog
  • packages/bigquery-magics/CHANGELOG.md
    • Added changelog entries for versions up to 0.12.1.
    • Version 0.12.1 includes a usability improvement to the graph visualizer.
    • Version 0.12.0 introduced schema view support and removed the bqsql magic.
    • Version 0.11.0 added support for Python 3.14.
    • Version 0.10.3 fixed property fields in graph visualization to be strings.
    • Version 0.10.2 dropped support for Python 3.7 and 3.8, and improved spanner-graph-notebook compatibility.
Activity
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@parthea parthea added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Mar 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Indicates a pull request not ready for merge, due to either quality or timing.

Projects

None yet

Development

Successfully merging this pull request may close these issues.