-
Notifications
You must be signed in to change notification settings - Fork 63
feat: Add BigQuery ObjectRef functions to bigframes.bigquery.obj
#2380
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
This change introduces support for BigQuery ObjectRef functions: - `OBJ.FETCH_METADATA` - `OBJ.GET_ACCESS_URL` - `OBJ.MAKE_REF` These are exposed via a new `bigframes.bigquery.obj` module.
jialuoo
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these test failures related to this PR? I’ve also added Trevor as a reviewer since he probably have more context on the background of these changes than I do.
|
|
||
| """ | ||
| ObjectRef functions defined from | ||
| https://cloud.google.com/bigquery/docs/reference/standard-sql/object-ref-functions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
the link doesn't work
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
| # limitations under the License. | ||
|
|
||
| """This module exposes `BigQuery ObjectRef | ||
| <https://cloud.google.com/bigquery/docs/object-table-object-ref-functions>`_ functions. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
link doesn't work.
| SELECT | ||
| *, | ||
| OBJ.GET_ACCESS_URL(`bfcol_4`) AS `bfcol_7` | ||
| OBJ.GET_ACCESS_URL(`bfcol_4`, 'R') AS `bfcol_7` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We may still need minimum system tests. Otherwise we may fail to catch errors like this. WDYT?
|
|
||
| @log_adapter.method_logger(custom_base_name="bigquery_obj") | ||
| def make_ref( | ||
| uri_or_json: series.Series, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It can also be a single value in SQL. E.g. OBJ.MAKE_REF("gs://something", "us.my_connection"). It can be useful in local experiments. Do we want to support the case?
| def fetch_metadata( | ||
| objectref: series.Series, | ||
| ) -> series.Series: | ||
| """The OBJ.FETCH_METADATA function returns Cloud Storage metadata for a partially populated ObjectRef value. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should add the preview disclaimer.
This change introduces support for BigQuery ObjectRef functions:
OBJ.FETCH_METADATAOBJ.GET_ACCESS_URLOBJ.MAKE_REFThese are exposed via a new
bigframes.bigquery.objmodule.Thank you for opening a Pull Request! Before submitting your PR, there are a few things you can do to make sure it goes smoothly:
Replaces #2340
🦕